690 lines
24 KiB
Markdown
690 lines
24 KiB
Markdown
# SolidJS Migration Plan
|
||
|
||
## Decision Record
|
||
|
||
| Decision | Choice |
|
||
|----------|--------|
|
||
| Framework | SolidJS (reactivity + components) |
|
||
| Build | Vite + vite-plugin-solid |
|
||
| Apps | Immersive only, extensible for future layouts |
|
||
| Core integration | Deep — ML/synth/audio modeled as SolidJS stores/signals |
|
||
| Output reactivity | Batch at frame rate (one signal update per rAF) |
|
||
| Event system | Unified signal bus with topics (replaces EventBus + CustomEvents) |
|
||
| Migration strategy | Big bang rewrite, old code as reference |
|
||
| Routing | None — signals only, URL params for config |
|
||
| Canvas | Components with refs, internal render loops |
|
||
| Mobile | Desktop-first, mobile later |
|
||
| Layout | Headless UI primitives (Drawer, Overlay, Panel, Dock) |
|
||
|
||
---
|
||
|
||
## Architecture Overview
|
||
|
||
```
|
||
src/
|
||
├── index.html # Single entry point
|
||
├── App.tsx # Root component, provider tree
|
||
├── vite.config.ts
|
||
│
|
||
├── core/ # Framework-agnostic engines (transplanted)
|
||
│ ├── wasm/ # WASM binaries + Emscripten glue (unchanged)
|
||
│ │ ├── nisps.wasm
|
||
│ │ ├── nisps.js
|
||
│ │ └── nisps-wasm-worker.js
|
||
│ ├── iml.ts # WasmIML wrapper (thin adaptation)
|
||
│ ├── dataset.ts # Training dataset (FIFO ring buffer)
|
||
│ ├── synth/ # Synth engines (transplanted, minimal changes)
|
||
│ │ ├── engine-interface.ts
|
||
│ │ ├── c15-adapter.ts
|
||
│ │ ├── c15-bridge.ts
|
||
│ │ ├── additive-engine.ts
|
||
│ │ ├── fm-engine.ts
|
||
│ │ ├── faust-engine-base.ts
|
||
│ │ ├── param-map.ts # 126 C15 params (data, unchanged)
|
||
│ │ └── presets.ts # Synth presets (data, unchanged)
|
||
│ ├── audio/
|
||
│ │ ├── arpeggiator.ts
|
||
│ │ ├── arpeggiator-worker.js
|
||
│ │ ├── audio-canvas.ts
|
||
│ │ └── midi-io.ts # MIDIInput + MIDIOutput merged
|
||
│ ├── eoc/ # Effects chain (transplanted)
|
||
│ │ ├── eoc-chain.ts
|
||
│ │ ├── eoc-module.ts
|
||
│ │ └── modules/ # Individual effect modules
|
||
│ └── shapeseq/ # Sequencer (transplanted, minimal changes)
|
||
│ ├── sequencer.ts
|
||
│ ├── chain.ts
|
||
│ ├── clock.ts
|
||
│ ├── pattern.ts
|
||
│ ├── primitives.ts
|
||
│ └── ...
|
||
│
|
||
├── bus/ # Signal bus
|
||
│ └── signal-bus.ts # createSignalBus(), topic(), match(), emit()
|
||
│
|
||
├── stores/ # SolidJS stores (the reactive state layer)
|
||
│ ├── ml-store.ts # IML instances, outputs, loss, training state
|
||
│ ├── input-store.ts # Joystick position, input mode, pipeline config
|
||
│ ├── output-store.ts # Output mode, overrides, routing config
|
||
│ ├── synth-store.ts # Active engine, arpeggiator, volume, presets
|
||
│ ├── eoc-store.ts # EOC chain state, nisps mode, modules
|
||
│ ├── midi-store.ts # MIDI CC map, devices, output state
|
||
│ ├── session-store.ts # Persistence (save/load localStorage)
|
||
│ └── ui-store.ts # Drawer state, active panels, help seen
|
||
│
|
||
├── hooks/ # Reactive glue (SolidJS "hooks" / composables)
|
||
│ ├── useInference.ts # Per-frame inference loop
|
||
│ ├── useTraining.ts # Train/thumbs-up/thumbs-down actions
|
||
│ ├── useOutputRouting.ts # Route outputs → synth/visual/midi/audio-canvas
|
||
│ ├── useInputPipeline.ts # Deadzone, zoom, curve, smoothing, momentum
|
||
│ ├── useOutputPipeline.ts # Slew rate, smoothing, freeze gate
|
||
│ ├── usePersistence.ts # Auto-save/load, URL param parsing
|
||
│ ├── useAudioContext.ts # Lazy AudioContext creation, resume on gesture
|
||
│ ├── useGamepad.ts # Gamepad polling
|
||
│ └── useKeyboard.ts # Keyboard shortcuts
|
||
│
|
||
├── primitives/ # Headless UI primitives
|
||
│ ├── Drawer.tsx # Slide-in panel (headless)
|
||
│ ├── Overlay.tsx # Floating positioned element
|
||
│ ├── Panel.tsx # Collapsible content section
|
||
│ ├── Dock.tsx # Icon bar with drawer triggers
|
||
│ ├── PillToggle.tsx # Segmented toggle (output mode, input mode)
|
||
│ ├── Slider.tsx # Range input with label/value display
|
||
│ └── Canvas.tsx # Canvas wrapper with ref + resize observer
|
||
│
|
||
├── components/ # Feature components
|
||
│ ├── app/
|
||
│ │ └── ImmersiveLayout.tsx # Main layout shell
|
||
│ ├── input/
|
||
│ │ ├── Joystick.tsx # Virtual joystick (pointer events)
|
||
│ │ ├── JoyMap.tsx # Zoom minimap + trails (canvas)
|
||
│ │ └── InputModeToggle.tsx
|
||
│ ├── output/
|
||
│ │ ├── FlowField.tsx # Particle visualizer (canvas)
|
||
│ │ ├── SynthVisualizer.tsx # Param bar chart (canvas)
|
||
│ │ ├── Heatmap.tsx # Parameter heatmap grid
|
||
│ │ └── OutputModeToggle.tsx
|
||
│ ├── training/
|
||
│ │ ├── TrainingControls.tsx # Add/Train/Clear/Randomize buttons
|
||
│ │ ├── RLControls.tsx # Thumbs up/down, noise display
|
||
│ │ ├── LossPlot.tsx # Loss history (canvas)
|
||
│ │ └── StatusLine.tsx # Example count, loss, mode
|
||
│ ├── synth/
|
||
│ │ ├── SynthControls.tsx # Start/stop, volume, arp controls
|
||
│ │ ├── PresetSelector.tsx # Synth preset chips
|
||
│ │ ├── EngineSwitcher.tsx # Engine dropdown
|
||
│ │ └── ParamEditor.tsx # Per-param override popup
|
||
│ ├── eoc/
|
||
│ │ ├── EOCPanel.tsx # Effects chain UI
|
||
│ │ ├── EOCModule.tsx # Single effect module card
|
||
│ │ └── EOCJoystick.tsx # Independent mode joystick
|
||
│ ├── midi/
|
||
│ │ ├── MIDIInputPanel.tsx # Device select, CC mapping
|
||
│ │ ├── MIDICCPanel.tsx # CC output config
|
||
│ │ └── MIDIPresets.tsx # CC preset management
|
||
│ ├── shapeseq/
|
||
│ │ ├── ShapeSeqPanel.tsx # Sequencer controls
|
||
│ │ ├── StepVisualizer.tsx # Step display (canvas)
|
||
│ │ └── ChainBuilder.tsx # Primitive chain editor
|
||
│ ├── controls/
|
||
│ │ ├── ControlSurface.tsx # Boldness/Memory/Precision axes
|
||
│ │ ├── InputHeatmap.tsx # 2D input space heatmap (canvas)
|
||
│ │ └── EngineParams.tsx # Spread, noise, decay sliders
|
||
│ └── debug/
|
||
│ ├── DevPanel.tsx # Debug tools
|
||
│ ├── WeightHealth.tsx # Weight magnitude histogram (canvas)
|
||
│ └── GradientFlow.tsx # Per-layer gradient analysis (canvas)
|
||
│
|
||
├── actions/ # Imperative operations (not reactive)
|
||
│ ├── resize-mlp.ts # Resize MLP, warm-start weights
|
||
│ ├── apply-overrides.ts # Override application logic
|
||
│ └── export-import.ts # Session export/import
|
||
│
|
||
└── assets/
|
||
├── c15/ # C15 WASM synth binary
|
||
└── faust/ # Faust DSP files
|
||
```
|
||
|
||
---
|
||
|
||
## Store Design (Deep Integration)
|
||
|
||
### ML Store (`stores/ml-store.ts`)
|
||
|
||
```typescript
|
||
import { createStore, produce } from "solid-js/store";
|
||
import { createSignal } from "solid-js";
|
||
|
||
// The hot path: batched per-frame, NOT per-parameter
|
||
const [outputs, setOutputs] = createSignal(new Float32Array(126));
|
||
const [rawParams, setRawParams] = createSignal(new Float32Array(126));
|
||
|
||
const [mlState, setMlState] = createStore({
|
||
// IML instances (not reactive themselves — opaque handles)
|
||
imlJoy: null as WasmIML | null,
|
||
imlHand: null as WasmIML | null,
|
||
activeIml: 'joy' as 'joy' | 'hand',
|
||
|
||
// Reactive state derived from IML
|
||
outputCount: 126,
|
||
loss: null as number | null,
|
||
lossHistory: [] as number[],
|
||
exampleCount: 0,
|
||
isTraining: false,
|
||
layerStats: null,
|
||
|
||
// RL config
|
||
spreadLevel: 0.6,
|
||
noiseLevel: 0.05,
|
||
rlDecay: 0.97,
|
||
learningRate: 0.1,
|
||
maxIterations: 50,
|
||
|
||
// Undo
|
||
undoStack: [] as Float32Array[],
|
||
});
|
||
|
||
// The IML instances live OUTSIDE the store (mutable, non-proxy-safe).
|
||
// The store tracks their *state* reactively.
|
||
// After each inference: setOutputs(iml.getOutputs())
|
||
// After each train: setMlState({ loss, exampleCount, lossHistory })
|
||
```
|
||
|
||
**Key design choice**: `Float32Array` outputs are a signal, not a store property. Stores use proxies which don't play well with typed arrays. A signal holding the array reference, replaced each frame, gives us batch reactivity cheaply.
|
||
|
||
### Input Store (`stores/input-store.ts`)
|
||
|
||
```typescript
|
||
const [inputState, setInputState] = createStore({
|
||
mode: 'joystick' as 'joystick' | 'hands',
|
||
joyX: 0.5,
|
||
joyY: 0.5,
|
||
isDragging: false,
|
||
followMode: false,
|
||
|
||
// Input pipeline config
|
||
pipeline: {
|
||
deadzone: 0.02,
|
||
zoom: 1.0,
|
||
zoomAnchorX: 0.5,
|
||
zoomAnchorY: 0.5,
|
||
anchorMode: 'auto' as 'auto' | 'sticky' | 'center',
|
||
curve: 1.0, // 1.0 = linear
|
||
smoothing: 0.0, // 0.0 = none
|
||
momentumZoom: false,
|
||
},
|
||
});
|
||
```
|
||
|
||
### Output Store (`stores/output-store.ts`)
|
||
|
||
```typescript
|
||
const [outputState, setOutputState] = createStore({
|
||
mode: 'visual' as 'visual' | 'synth' | 'midi-cc' | 'audio-canvas',
|
||
|
||
// Per-mode overrides (arrays of { min, max, curve, muted, fixedValue })
|
||
visualOverrides: [] as ParamOverride[],
|
||
synthOverrides: {
|
||
type: 'grouped' as 'grouped' | 'flat',
|
||
groups: [] as GroupOverride[], // C15 grouped
|
||
flat: [] as ParamOverride[], // Faust flat
|
||
},
|
||
midiCCOverrides: [] as ParamOverride[],
|
||
audioCanvasOverrides: [] as ParamOverride[],
|
||
|
||
// Output pipeline config
|
||
pipeline: {
|
||
globalCurve: 1.0,
|
||
smoothing: 0.0,
|
||
slewRate: 1.0,
|
||
freezeGate: false,
|
||
},
|
||
});
|
||
```
|
||
|
||
### Synth Store (`stores/synth-store.ts`)
|
||
|
||
```typescript
|
||
const [synthState, setSynthState] = createStore({
|
||
engineId: 'shaper-feedback' as string,
|
||
engine: null as SynthEngine | null, // Opaque handle
|
||
isRunning: false,
|
||
volume: 0.7,
|
||
|
||
// Arpeggiator
|
||
arp: {
|
||
enabled: false,
|
||
tempo: 120,
|
||
progression: 'major' as string,
|
||
octaves: 1,
|
||
offset: 0,
|
||
},
|
||
|
||
// Current preset
|
||
presetId: null as string | null,
|
||
});
|
||
```
|
||
|
||
---
|
||
|
||
## Signal Bus Design (`bus/signal-bus.ts`)
|
||
|
||
```typescript
|
||
import { createSignal, createMemo } from "solid-js";
|
||
|
||
type Topic<T = any> = {
|
||
(): T | undefined; // Read (reactive)
|
||
fire: (data: T) => void; // Write (imperative)
|
||
};
|
||
|
||
export function createSignalBus() {
|
||
const topics = new Map<string, ReturnType<typeof createSignal>>();
|
||
|
||
function topic<T = any>(name: string): Topic<T> {
|
||
if (!topics.has(name)) {
|
||
const [get, set] = createSignal<T | undefined>(undefined, { equals: false });
|
||
topics.set(name, [get, set]);
|
||
}
|
||
const [get, set] = topics.get(name)!;
|
||
const accessor = () => get() as T | undefined;
|
||
accessor.fire = (data: T) => set(() => data);
|
||
return accessor as Topic<T>;
|
||
}
|
||
|
||
function emit<T = any>(name: string, data: T) {
|
||
topic<T>(name).fire(data);
|
||
}
|
||
|
||
function match(pattern: string) {
|
||
// 'seq.*' → derived signal merging all seq.* topics
|
||
const prefix = pattern.replace('*', '');
|
||
return createMemo(() => {
|
||
let latest: { name: string; data: any } | undefined;
|
||
for (const [name, [get]] of topics) {
|
||
if (name.startsWith(prefix)) {
|
||
const val = get();
|
||
if (val !== undefined) {
|
||
latest = { name, data: val };
|
||
}
|
||
}
|
||
}
|
||
return latest;
|
||
});
|
||
}
|
||
|
||
return { topic, emit, match };
|
||
}
|
||
```
|
||
|
||
**Usage equals: false** is critical — it means every `emit()` triggers subscribers even if the data is identical. This matches event semantics (every `seq.step` matters, even if the step number repeats).
|
||
|
||
---
|
||
|
||
## Inference Loop (`hooks/useInference.ts`)
|
||
|
||
```typescript
|
||
import { createEffect, onCleanup } from "solid-js";
|
||
|
||
export function useInference(mlStore, inputStore, outputStore, bus) {
|
||
let rafId: number;
|
||
|
||
function tick() {
|
||
const iml = mlStore.activeIml === 'joy' ? mlStore.imlJoy : mlStore.imlHand;
|
||
if (!iml) { rafId = requestAnimationFrame(tick); return; }
|
||
|
||
// Set inputs
|
||
iml.setInput(0, inputStore.joyX);
|
||
iml.setInput(1, inputStore.joyY);
|
||
|
||
// Forward pass
|
||
iml.process();
|
||
const raw = iml.getOutputs();
|
||
|
||
// Batch update — single signal write per frame
|
||
setOutputs(new Float32Array(raw));
|
||
|
||
// Route to active output
|
||
routeOutputs(raw, outputStore, synthStore, bus);
|
||
|
||
// Fire bus event (for non-UI subscribers like sequencer)
|
||
bus.emit('ml.inference', { outputs: raw });
|
||
|
||
rafId = requestAnimationFrame(tick);
|
||
}
|
||
|
||
rafId = requestAnimationFrame(tick);
|
||
onCleanup(() => cancelAnimationFrame(rafId));
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## Output Routing (`hooks/useOutputRouting.ts`)
|
||
|
||
```typescript
|
||
function routeOutputs(raw: Float32Array, outputStore, synthStore, bus) {
|
||
const mode = outputStore.mode;
|
||
const overrides = getOverridesForMode(mode, outputStore);
|
||
|
||
// Apply overrides (pure function, no side effects)
|
||
const processed = applyOverrides(raw, overrides);
|
||
|
||
switch (mode) {
|
||
case 'visual':
|
||
bus.emit('output.visual', processed);
|
||
break;
|
||
case 'synth':
|
||
// Throttled: only send to engine at 20fps
|
||
bus.emit('output.synth', processed);
|
||
break;
|
||
case 'midi-cc':
|
||
bus.emit('output.midi', processed);
|
||
break;
|
||
case 'audio-canvas':
|
||
bus.emit('output.audiocanvas', processed);
|
||
break;
|
||
}
|
||
|
||
// Always update heatmap (unthrottled — it's just DOM)
|
||
bus.emit('output.heatmap', raw);
|
||
}
|
||
```
|
||
|
||
Each output component subscribes to its own bus topic. The FlowField listens to `output.visual`, the SynthVisualizer to `output.synth`, etc. Throttling for synth param sends lives inside the synth subscriber, not in the routing.
|
||
|
||
---
|
||
|
||
## Component Examples
|
||
|
||
### Joystick Component
|
||
|
||
```tsx
|
||
const Joystick = () => {
|
||
const { inputState, setInputState } = useInputStore();
|
||
const bus = useBus();
|
||
|
||
const onPointerDown = (e: PointerEvent) => {
|
||
(e.target as Element).setPointerCapture(e.pointerId);
|
||
const rect = (e.target as Element).getBoundingClientRect();
|
||
const x = (e.clientX - rect.left) / rect.width;
|
||
const y = 1 - (e.clientY - rect.top) / rect.height;
|
||
setInputState({ joyX: x, joyY: y, isDragging: true });
|
||
};
|
||
|
||
const onPointerMove = (e: PointerEvent) => {
|
||
if (!inputState.isDragging) return;
|
||
const rect = (e.target as Element).getBoundingClientRect();
|
||
const x = Math.max(0, Math.min(1, (e.clientX - rect.left) / rect.width));
|
||
const y = Math.max(0, Math.min(1, 1 - (e.clientY - rect.top) / rect.height));
|
||
setInputState({ joyX: x, joyY: y });
|
||
};
|
||
|
||
const onPointerUp = () => setInputState({ isDragging: false });
|
||
|
||
return (
|
||
<div
|
||
class="joystick"
|
||
onPointerDown={onPointerDown}
|
||
onPointerMove={onPointerMove}
|
||
onPointerUp={onPointerUp}
|
||
>
|
||
<div
|
||
class="joystick-dot"
|
||
style={{
|
||
left: `${inputState.joyX * 100}%`,
|
||
bottom: `${inputState.joyY * 100}%`,
|
||
}}
|
||
/>
|
||
</div>
|
||
);
|
||
};
|
||
```
|
||
|
||
### FlowField (Canvas Component)
|
||
|
||
```tsx
|
||
const FlowField = () => {
|
||
let canvasRef: HTMLCanvasElement;
|
||
const bus = useBus();
|
||
const visualData = bus.topic<Float32Array>('output.visual');
|
||
|
||
onMount(() => {
|
||
const ctx = canvasRef.getContext('2d')!;
|
||
const vis = new FlowFieldVisualizer(ctx); // Transplanted engine
|
||
|
||
// Internal render loop — reads signal each frame
|
||
let raf: number;
|
||
const loop = () => {
|
||
const data = visualData();
|
||
if (data) vis.setParams(data);
|
||
vis.draw();
|
||
raf = requestAnimationFrame(loop);
|
||
};
|
||
raf = requestAnimationFrame(loop);
|
||
onCleanup(() => cancelAnimationFrame(raf));
|
||
});
|
||
|
||
return <canvas ref={canvasRef!} class="flow-field" />;
|
||
};
|
||
```
|
||
|
||
### Heatmap (Reactive DOM)
|
||
|
||
```tsx
|
||
const Heatmap = () => {
|
||
const bus = useBus();
|
||
const heatmapData = bus.topic<Float32Array>('output.heatmap');
|
||
const { outputState } = useOutputStore();
|
||
const paramMeta = () => getParamMeta(outputState.mode);
|
||
|
||
return (
|
||
<div class="heatmap-grid">
|
||
<For each={paramMeta()}>
|
||
{(param, i) => (
|
||
<HeatmapCell
|
||
index={i()}
|
||
param={param}
|
||
value={() => heatmapData()?.[i()] ?? 0}
|
||
/>
|
||
)}
|
||
</For>
|
||
</div>
|
||
);
|
||
};
|
||
|
||
const HeatmapCell = (props) => {
|
||
const width = () => `${(props.value() * 100).toFixed(1)}%`;
|
||
return (
|
||
<div class="heatmap-cell" title={props.param.name}>
|
||
<div class="heatmap-bar" style={{ width: width() }} />
|
||
</div>
|
||
);
|
||
};
|
||
```
|
||
|
||
---
|
||
|
||
## Provider Tree (`App.tsx`)
|
||
|
||
```tsx
|
||
const App = () => {
|
||
return (
|
||
<BusProvider>
|
||
<MLProvider>
|
||
<InputProvider>
|
||
<OutputProvider>
|
||
<SynthProvider>
|
||
<EOCProvider>
|
||
<SessionProvider>
|
||
<ImmersiveLayout />
|
||
</SessionProvider>
|
||
</EOCProvider>
|
||
</SynthProvider>
|
||
</OutputProvider>
|
||
</InputProvider>
|
||
</MLProvider>
|
||
</BusProvider>
|
||
);
|
||
};
|
||
```
|
||
|
||
Each provider creates its store and exposes it via context. The `SessionProvider` handles persistence (auto-save on interval, load on mount, URL param parsing).
|
||
|
||
---
|
||
|
||
## Migration Phases
|
||
|
||
### Phase 1: Skeleton + Core Loop
|
||
|
||
**Goal**: Joystick → MLP → FlowField working in SolidJS.
|
||
|
||
1. `npm create vite@latest playground-solid -- --template solid-ts`
|
||
2. Set up project structure (`core/`, `stores/`, `components/`, `bus/`)
|
||
3. Transplant WASM files and `nisps-wasm.js` into `core/wasm/`
|
||
4. Implement `createSignalBus()`
|
||
5. Implement `ml-store.ts` and `input-store.ts` (minimal)
|
||
6. Implement `useInference` hook
|
||
7. Build `Joystick` component
|
||
8. Build `FlowField` component (transplant `FlowFieldVisualizer` class)
|
||
9. Wire it up in `App.tsx` with minimal `ImmersiveLayout`
|
||
10. Verify: drag joystick → see particles respond
|
||
|
||
**Validates**: SolidJS + WASM integration, signal bus, frame-rate batching, canvas components.
|
||
|
||
### Phase 2: Training + RL
|
||
|
||
**Goal**: Full ML interaction loop.
|
||
|
||
1. Implement `useTraining` hook (add example, train async, thumbs up/down)
|
||
2. Implement `output-store.ts` (visual overrides only for now)
|
||
3. Build `TrainingControls` (Add/Train/Clear/Randomize)
|
||
4. Build `RLControls` (thumbs up/down, noise display)
|
||
5. Build `LossPlot` (canvas)
|
||
6. Build `StatusLine`
|
||
7. Implement undo stack in `ml-store`
|
||
8. Implement `useOutputRouting` (visual mode only)
|
||
|
||
**Validates**: Async training with UI updates, RL feedback loop, undo.
|
||
|
||
### Phase 3: Synth Integration
|
||
|
||
**Goal**: Synth output mode working.
|
||
|
||
1. Transplant `c15-adapter.ts`, `c15-bridge.ts`, `param-map.ts`, `presets.ts`
|
||
2. Implement `synth-store.ts`
|
||
3. Implement `useAudioContext` (lazy creation, gesture resume)
|
||
4. Build `SynthControls` (start/stop, volume)
|
||
5. Build `SynthVisualizer` (canvas, param bars)
|
||
6. Build `PresetSelector`
|
||
7. Build `EngineSwitcher` (C15, Additive, FM)
|
||
8. Implement synth output routing with throttling
|
||
9. Implement `output-store` grouped overrides for C15
|
||
10. Build `Heatmap` + `ParamEditor` popup
|
||
|
||
**Validates**: Multi-engine support, override system, audio integration.
|
||
|
||
### Phase 4: Layout + UI Primitives
|
||
|
||
**Goal**: Full immersive UI.
|
||
|
||
1. Build headless primitives: `Drawer`, `Overlay`, `Dock`, `PillToggle`, `Slider`
|
||
2. Build `ImmersiveLayout` (fullscreen canvas + floating controls + drawer stack)
|
||
3. Build `ControlSurface` (Boldness/Memory/Precision compound axes)
|
||
4. Implement `useInputPipeline` (deadzone, zoom, curve, smoothing)
|
||
5. Build `JoyMap` (zoom minimap with trails, canvas)
|
||
6. Build `InputHeatmap` (2D color field, canvas)
|
||
7. Implement `useKeyboard` (shortcuts: 1/2/Z etc.)
|
||
8. Wire drawer system (dock icons → drawer toggles)
|
||
|
||
**Validates**: Layout primitives, compound axis system, input pipeline.
|
||
|
||
### Phase 5: Remaining Features
|
||
|
||
**Goal**: Feature parity with a-immersive.
|
||
|
||
1. **MIDI**: `MIDIInputPanel`, `MIDICCPanel`, `MIDIPresets`, midi-cc output mode
|
||
2. **Audio Canvas**: Transplant, wire as output mode
|
||
3. **EOC Chain**: `EOCPanel`, `EOCModule`, EOC joystick, nisps modes (Shared/Linked/Independent)
|
||
4. **Arpeggiator**: Controls, worker integration
|
||
5. **Output Pipeline**: `useOutputPipeline` (slew, smoothing, freeze)
|
||
6. **Weight Health**: `WeightHealth`, `GradientFlow` (canvas)
|
||
7. **Session Presets**: Save/load full state, URL sharing
|
||
8. **Persistence**: Auto-save, localStorage round-trip
|
||
9. **Debug probe**: `window.__nisps` when `?debug=1`
|
||
|
||
### Phase 6: ShapeSeq
|
||
|
||
**Goal**: Sequencer as optional subsystem.
|
||
|
||
1. Transplant sequencer core (already modular)
|
||
2. Build `ShapeSeqPanel`, `StepVisualizer`, `ChainBuilder`
|
||
3. Wire to signal bus (`seq.*` topics)
|
||
4. Lazy-load when enabled via URL param
|
||
|
||
### Phase 7: Polish + Tests
|
||
|
||
1. Port Playwright e2e tests (update selectors for new DOM)
|
||
2. Add component-level tests (vitest + solid-testing-library)
|
||
3. Responsive CSS pass
|
||
4. Performance profiling (ensure 60fps inference + rendering)
|
||
5. Accessibility pass on interactive elements
|
||
|
||
---
|
||
|
||
## Key Risks & Mitigations
|
||
|
||
| Risk | Mitigation |
|
||
|------|------------|
|
||
| WASM + SolidJS reactivity overhead | Typed arrays stay as signals (not store properties). Batch per frame. Profile early in Phase 1. |
|
||
| 126 params × 20fps = 2520 synth param sends/sec | Keep existing throttle (50ms interval, 0.002 dead zone) in synth subscriber, not in routing. |
|
||
| Store proxy overhead on large arrays | Use `createSignal` for `Float32Array` outputs, not `createStore`. Stores only for structured config. |
|
||
| Signal bus wildcard `match()` perf | Memoize prefix scans. Most topics are fixed at startup. Profile if >50 topics. |
|
||
| Canvas components fighting rAF | Each canvas owns its loop. No shared orchestrator unless profiling shows frame contention. |
|
||
| MLP resize destroys training data | Same as current: warn user, warm-start weights for joystick IML. Store handles the state transition. |
|
||
| `equals: false` on bus signals | Required for event semantics but means every emit triggers all subscribers. Keep topic count bounded. |
|
||
|
||
---
|
||
|
||
## What Gets Transplanted vs Rewritten
|
||
|
||
### Transplanted (minimal changes, mostly just TS types)
|
||
- `nisps-wasm.js` + worker + WASM binary
|
||
- `dataset.ts` (FIFO ring buffer)
|
||
- `FlowFieldVisualizer` class (canvas rendering)
|
||
- `c15-bridge.ts`, `c15-adapter.ts`
|
||
- `param-map.ts`, `presets.ts` (pure data)
|
||
- `additive-engine.ts`, `fm-engine.ts`, `faust-engine-base.ts`
|
||
- `arpeggiator.ts` + worker
|
||
- `eoc-chain.ts`, `eoc-module.ts`, all effect modules
|
||
- ShapeSeq core (`sequencer.ts`, `chain.ts`, `clock.ts`, `pattern.ts`, `primitives.ts`)
|
||
- `audio-canvas.ts`
|
||
- `input-pipeline.ts` logic (becomes `useInputPipeline` hook wrapping same math)
|
||
- `output-pipeline.ts` logic (becomes `useOutputPipeline` hook)
|
||
- `control-surface.ts` compound axis logic (data tables + interpolation)
|
||
|
||
### Rewritten from scratch
|
||
- All DOM manipulation (→ JSX components)
|
||
- Event wiring (→ signal bus + reactive effects)
|
||
- State management (scattered module-scope vars → stores)
|
||
- Override application (→ `apply-overrides.ts` pure function)
|
||
- Persistence (→ `SessionProvider` with `createEffect` auto-save)
|
||
- Layout/CSS (→ new CSS with headless primitives)
|
||
- Init/boot sequence (→ provider tree + `onMount` hooks)
|
||
|
||
### Deleted (not ported)
|
||
- `b-app.js`, `c-app.js`, `app.js` (consolidated into one app)
|
||
- `b-workbench.html`, `c-journey.html`, `index.html` (single entry point)
|
||
- `iml.js`, `mlp.js`, `layer.js`, `node.js` (legacy JS ML engine, WASM only)
|
||
- `event-bus.js` (replaced by signal bus)
|
||
- All `wire*()` functions (replaced by component-local event handlers)
|
||
- DOM-string-building functions (`buildHeatmap`, `buildDrawerHTML`, etc.)
|
||
|
||
---
|
||
|
||
## Open Questions
|
||
|
||
1. **TypeScript strictness**: Full strict mode from day one, or gradual? (Transplanted JS modules will need type annotations.)
|
||
2. **CSS approach**: Plain CSS files per component? CSS modules? Vanilla Extract? (Plain CSS is simplest and matches the current approach.)
|
||
3. **Testing during migration**: Run old Playwright tests against old code in parallel, or wait for Phase 7?
|
||
4. **WASM loading**: Vite handles `.wasm` imports natively, but the Emscripten glue (`nisps.js`) may need special config. Spike this in Phase 1.
|
||
5. **Faust DSP loading**: Currently fetched at runtime. Keep as-is or bundle?
|