# SolidJS Migration Plan ## Decision Record | Decision | Choice | |----------|--------| | Framework | SolidJS (reactivity + components) | | Build | Vite + vite-plugin-solid | | Apps | Immersive only, extensible for future layouts | | Core integration | Deep — ML/synth/audio modeled as SolidJS stores/signals | | Output reactivity | Batch at frame rate (one signal update per rAF) | | Event system | Unified signal bus with topics (replaces EventBus + CustomEvents) | | Migration strategy | Big bang rewrite, old code as reference | | Routing | None — signals only, URL params for config | | Canvas | Components with refs, internal render loops | | Mobile | Desktop-first, mobile later | | Layout | Headless UI primitives (Drawer, Overlay, Panel, Dock) | --- ## Architecture Overview ``` src/ ├── index.html # Single entry point ├── App.tsx # Root component, provider tree ├── vite.config.ts │ ├── core/ # Framework-agnostic engines (transplanted) │ ├── wasm/ # WASM binaries + Emscripten glue (unchanged) │ │ ├── nisps.wasm │ │ ├── nisps.js │ │ └── nisps-wasm-worker.js │ ├── iml.ts # WasmIML wrapper (thin adaptation) │ ├── dataset.ts # Training dataset (FIFO ring buffer) │ ├── synth/ # Synth engines (transplanted, minimal changes) │ │ ├── engine-interface.ts │ │ ├── c15-adapter.ts │ │ ├── c15-bridge.ts │ │ ├── additive-engine.ts │ │ ├── fm-engine.ts │ │ ├── faust-engine-base.ts │ │ ├── param-map.ts # 126 C15 params (data, unchanged) │ │ └── presets.ts # Synth presets (data, unchanged) │ ├── audio/ │ │ ├── arpeggiator.ts │ │ ├── arpeggiator-worker.js │ │ ├── audio-canvas.ts │ │ └── midi-io.ts # MIDIInput + MIDIOutput merged │ ├── eoc/ # Effects chain (transplanted) │ │ ├── eoc-chain.ts │ │ ├── eoc-module.ts │ │ └── modules/ # Individual effect modules │ └── shapeseq/ # Sequencer (transplanted, minimal changes) │ ├── sequencer.ts │ ├── chain.ts │ ├── clock.ts │ ├── pattern.ts │ ├── primitives.ts │ └── ... │ ├── bus/ # Signal bus │ └── signal-bus.ts # createSignalBus(), topic(), match(), emit() │ ├── stores/ # SolidJS stores (the reactive state layer) │ ├── ml-store.ts # IML instances, outputs, loss, training state │ ├── input-store.ts # Joystick position, input mode, pipeline config │ ├── output-store.ts # Output mode, overrides, routing config │ ├── synth-store.ts # Active engine, arpeggiator, volume, presets │ ├── eoc-store.ts # EOC chain state, nisps mode, modules │ ├── midi-store.ts # MIDI CC map, devices, output state │ ├── session-store.ts # Persistence (save/load localStorage) │ └── ui-store.ts # Drawer state, active panels, help seen │ ├── hooks/ # Reactive glue (SolidJS "hooks" / composables) │ ├── useInference.ts # Per-frame inference loop │ ├── useTraining.ts # Train/thumbs-up/thumbs-down actions │ ├── useOutputRouting.ts # Route outputs → synth/visual/midi/audio-canvas │ ├── useInputPipeline.ts # Deadzone, zoom, curve, smoothing, momentum │ ├── useOutputPipeline.ts # Slew rate, smoothing, freeze gate │ ├── usePersistence.ts # Auto-save/load, URL param parsing │ ├── useAudioContext.ts # Lazy AudioContext creation, resume on gesture │ ├── useGamepad.ts # Gamepad polling │ └── useKeyboard.ts # Keyboard shortcuts │ ├── primitives/ # Headless UI primitives │ ├── Drawer.tsx # Slide-in panel (headless) │ ├── Overlay.tsx # Floating positioned element │ ├── Panel.tsx # Collapsible content section │ ├── Dock.tsx # Icon bar with drawer triggers │ ├── PillToggle.tsx # Segmented toggle (output mode, input mode) │ ├── Slider.tsx # Range input with label/value display │ └── Canvas.tsx # Canvas wrapper with ref + resize observer │ ├── components/ # Feature components │ ├── app/ │ │ └── ImmersiveLayout.tsx # Main layout shell │ ├── input/ │ │ ├── Joystick.tsx # Virtual joystick (pointer events) │ │ ├── JoyMap.tsx # Zoom minimap + trails (canvas) │ │ └── InputModeToggle.tsx │ ├── output/ │ │ ├── FlowField.tsx # Particle visualizer (canvas) │ │ ├── SynthVisualizer.tsx # Param bar chart (canvas) │ │ ├── Heatmap.tsx # Parameter heatmap grid │ │ └── OutputModeToggle.tsx │ ├── training/ │ │ ├── TrainingControls.tsx # Add/Train/Clear/Randomize buttons │ │ ├── RLControls.tsx # Thumbs up/down, noise display │ │ ├── LossPlot.tsx # Loss history (canvas) │ │ └── StatusLine.tsx # Example count, loss, mode │ ├── synth/ │ │ ├── SynthControls.tsx # Start/stop, volume, arp controls │ │ ├── PresetSelector.tsx # Synth preset chips │ │ ├── EngineSwitcher.tsx # Engine dropdown │ │ └── ParamEditor.tsx # Per-param override popup │ ├── eoc/ │ │ ├── EOCPanel.tsx # Effects chain UI │ │ ├── EOCModule.tsx # Single effect module card │ │ └── EOCJoystick.tsx # Independent mode joystick │ ├── midi/ │ │ ├── MIDIInputPanel.tsx # Device select, CC mapping │ │ ├── MIDICCPanel.tsx # CC output config │ │ └── MIDIPresets.tsx # CC preset management │ ├── shapeseq/ │ │ ├── ShapeSeqPanel.tsx # Sequencer controls │ │ ├── StepVisualizer.tsx # Step display (canvas) │ │ └── ChainBuilder.tsx # Primitive chain editor │ ├── controls/ │ │ ├── ControlSurface.tsx # Boldness/Memory/Precision axes │ │ ├── InputHeatmap.tsx # 2D input space heatmap (canvas) │ │ └── EngineParams.tsx # Spread, noise, decay sliders │ └── debug/ │ ├── DevPanel.tsx # Debug tools │ ├── WeightHealth.tsx # Weight magnitude histogram (canvas) │ └── GradientFlow.tsx # Per-layer gradient analysis (canvas) │ ├── actions/ # Imperative operations (not reactive) │ ├── resize-mlp.ts # Resize MLP, warm-start weights │ ├── apply-overrides.ts # Override application logic │ └── export-import.ts # Session export/import │ └── assets/ ├── c15/ # C15 WASM synth binary └── faust/ # Faust DSP files ``` --- ## Store Design (Deep Integration) ### ML Store (`stores/ml-store.ts`) ```typescript import { createStore, produce } from "solid-js/store"; import { createSignal } from "solid-js"; // The hot path: batched per-frame, NOT per-parameter const [outputs, setOutputs] = createSignal(new Float32Array(126)); const [rawParams, setRawParams] = createSignal(new Float32Array(126)); const [mlState, setMlState] = createStore({ // IML instances (not reactive themselves — opaque handles) imlJoy: null as WasmIML | null, imlHand: null as WasmIML | null, activeIml: 'joy' as 'joy' | 'hand', // Reactive state derived from IML outputCount: 126, loss: null as number | null, lossHistory: [] as number[], exampleCount: 0, isTraining: false, layerStats: null, // RL config spreadLevel: 0.6, noiseLevel: 0.05, rlDecay: 0.97, learningRate: 0.1, maxIterations: 50, // Undo undoStack: [] as Float32Array[], }); // The IML instances live OUTSIDE the store (mutable, non-proxy-safe). // The store tracks their *state* reactively. // After each inference: setOutputs(iml.getOutputs()) // After each train: setMlState({ loss, exampleCount, lossHistory }) ``` **Key design choice**: `Float32Array` outputs are a signal, not a store property. Stores use proxies which don't play well with typed arrays. A signal holding the array reference, replaced each frame, gives us batch reactivity cheaply. ### Input Store (`stores/input-store.ts`) ```typescript const [inputState, setInputState] = createStore({ mode: 'joystick' as 'joystick' | 'hands', joyX: 0.5, joyY: 0.5, isDragging: false, followMode: false, // Input pipeline config pipeline: { deadzone: 0.02, zoom: 1.0, zoomAnchorX: 0.5, zoomAnchorY: 0.5, anchorMode: 'auto' as 'auto' | 'sticky' | 'center', curve: 1.0, // 1.0 = linear smoothing: 0.0, // 0.0 = none momentumZoom: false, }, }); ``` ### Output Store (`stores/output-store.ts`) ```typescript const [outputState, setOutputState] = createStore({ mode: 'visual' as 'visual' | 'synth' | 'midi-cc' | 'audio-canvas', // Per-mode overrides (arrays of { min, max, curve, muted, fixedValue }) visualOverrides: [] as ParamOverride[], synthOverrides: { type: 'grouped' as 'grouped' | 'flat', groups: [] as GroupOverride[], // C15 grouped flat: [] as ParamOverride[], // Faust flat }, midiCCOverrides: [] as ParamOverride[], audioCanvasOverrides: [] as ParamOverride[], // Output pipeline config pipeline: { globalCurve: 1.0, smoothing: 0.0, slewRate: 1.0, freezeGate: false, }, }); ``` ### Synth Store (`stores/synth-store.ts`) ```typescript const [synthState, setSynthState] = createStore({ engineId: 'shaper-feedback' as string, engine: null as SynthEngine | null, // Opaque handle isRunning: false, volume: 0.7, // Arpeggiator arp: { enabled: false, tempo: 120, progression: 'major' as string, octaves: 1, offset: 0, }, // Current preset presetId: null as string | null, }); ``` --- ## Signal Bus Design (`bus/signal-bus.ts`) ```typescript import { createSignal, createMemo } from "solid-js"; type Topic = { (): T | undefined; // Read (reactive) fire: (data: T) => void; // Write (imperative) }; export function createSignalBus() { const topics = new Map>(); function topic(name: string): Topic { if (!topics.has(name)) { const [get, set] = createSignal(undefined, { equals: false }); topics.set(name, [get, set]); } const [get, set] = topics.get(name)!; const accessor = () => get() as T | undefined; accessor.fire = (data: T) => set(() => data); return accessor as Topic; } function emit(name: string, data: T) { topic(name).fire(data); } function match(pattern: string) { // 'seq.*' → derived signal merging all seq.* topics const prefix = pattern.replace('*', ''); return createMemo(() => { let latest: { name: string; data: any } | undefined; for (const [name, [get]] of topics) { if (name.startsWith(prefix)) { const val = get(); if (val !== undefined) { latest = { name, data: val }; } } } return latest; }); } return { topic, emit, match }; } ``` **Usage equals: false** is critical — it means every `emit()` triggers subscribers even if the data is identical. This matches event semantics (every `seq.step` matters, even if the step number repeats). --- ## Inference Loop (`hooks/useInference.ts`) ```typescript import { createEffect, onCleanup } from "solid-js"; export function useInference(mlStore, inputStore, outputStore, bus) { let rafId: number; function tick() { const iml = mlStore.activeIml === 'joy' ? mlStore.imlJoy : mlStore.imlHand; if (!iml) { rafId = requestAnimationFrame(tick); return; } // Set inputs iml.setInput(0, inputStore.joyX); iml.setInput(1, inputStore.joyY); // Forward pass iml.process(); const raw = iml.getOutputs(); // Batch update — single signal write per frame setOutputs(new Float32Array(raw)); // Route to active output routeOutputs(raw, outputStore, synthStore, bus); // Fire bus event (for non-UI subscribers like sequencer) bus.emit('ml.inference', { outputs: raw }); rafId = requestAnimationFrame(tick); } rafId = requestAnimationFrame(tick); onCleanup(() => cancelAnimationFrame(rafId)); } ``` --- ## Output Routing (`hooks/useOutputRouting.ts`) ```typescript function routeOutputs(raw: Float32Array, outputStore, synthStore, bus) { const mode = outputStore.mode; const overrides = getOverridesForMode(mode, outputStore); // Apply overrides (pure function, no side effects) const processed = applyOverrides(raw, overrides); switch (mode) { case 'visual': bus.emit('output.visual', processed); break; case 'synth': // Throttled: only send to engine at 20fps bus.emit('output.synth', processed); break; case 'midi-cc': bus.emit('output.midi', processed); break; case 'audio-canvas': bus.emit('output.audiocanvas', processed); break; } // Always update heatmap (unthrottled — it's just DOM) bus.emit('output.heatmap', raw); } ``` Each output component subscribes to its own bus topic. The FlowField listens to `output.visual`, the SynthVisualizer to `output.synth`, etc. Throttling for synth param sends lives inside the synth subscriber, not in the routing. --- ## Component Examples ### Joystick Component ```tsx const Joystick = () => { const { inputState, setInputState } = useInputStore(); const bus = useBus(); const onPointerDown = (e: PointerEvent) => { (e.target as Element).setPointerCapture(e.pointerId); const rect = (e.target as Element).getBoundingClientRect(); const x = (e.clientX - rect.left) / rect.width; const y = 1 - (e.clientY - rect.top) / rect.height; setInputState({ joyX: x, joyY: y, isDragging: true }); }; const onPointerMove = (e: PointerEvent) => { if (!inputState.isDragging) return; const rect = (e.target as Element).getBoundingClientRect(); const x = Math.max(0, Math.min(1, (e.clientX - rect.left) / rect.width)); const y = Math.max(0, Math.min(1, 1 - (e.clientY - rect.top) / rect.height)); setInputState({ joyX: x, joyY: y }); }; const onPointerUp = () => setInputState({ isDragging: false }); return (
); }; ``` ### FlowField (Canvas Component) ```tsx const FlowField = () => { let canvasRef: HTMLCanvasElement; const bus = useBus(); const visualData = bus.topic('output.visual'); onMount(() => { const ctx = canvasRef.getContext('2d')!; const vis = new FlowFieldVisualizer(ctx); // Transplanted engine // Internal render loop — reads signal each frame let raf: number; const loop = () => { const data = visualData(); if (data) vis.setParams(data); vis.draw(); raf = requestAnimationFrame(loop); }; raf = requestAnimationFrame(loop); onCleanup(() => cancelAnimationFrame(raf)); }); return ; }; ``` ### Heatmap (Reactive DOM) ```tsx const Heatmap = () => { const bus = useBus(); const heatmapData = bus.topic('output.heatmap'); const { outputState } = useOutputStore(); const paramMeta = () => getParamMeta(outputState.mode); return (
{(param, i) => ( heatmapData()?.[i()] ?? 0} /> )}
); }; const HeatmapCell = (props) => { const width = () => `${(props.value() * 100).toFixed(1)}%`; return (
); }; ``` --- ## Provider Tree (`App.tsx`) ```tsx const App = () => { return ( ); }; ``` Each provider creates its store and exposes it via context. The `SessionProvider` handles persistence (auto-save on interval, load on mount, URL param parsing). --- ## Migration Phases ### Phase 1: Skeleton + Core Loop **Goal**: Joystick → MLP → FlowField working in SolidJS. **Also**: validate all risky integration points early. 1. `npm create vite@latest playground-solid -- --template solid-ts` 2. Set up project structure (`core/`, `stores/`, `components/`, `bus/`) 3. **Configure Vite**: COOP/COEP headers, WASM asset handling (spike `public/` vs `locateFile`) 4. Transplant WASM files (`nisps.wasm`, `nisps.js`, worker) into `core/wasm/` 5. **Spike**: Verify WASM loads and runs inference in Vite dev server (before building any UI) 6. **Spike**: Verify C15 WASM + SharedArrayBuffer works with COOP/COEP headers 7. Implement `createSignalBus()` 8. Implement `ml-store.ts` and `input-store.ts` (minimal) 9. Implement reactive inference (`createEffect` on input signals, not rAF) 10. Build `Joystick` component 11. Build `FlowField` component (transplant `FlowFieldVisualizer` class, own rAF loop) 12. Wire it up in `App.tsx` with minimal `ImmersiveLayout` 13. Verify: drag joystick → see particles respond 14. Add worker `dispose()` to IML, verify cleanup in `onCleanup()` **Validates**: SolidJS + WASM integration, COOP/COEP, signal bus, reactive inference, canvas components, worker lifecycle. ### Phase 2: Training + RL **Goal**: Full ML interaction loop. 1. Implement `useTraining` hook (add example, train async, thumbs up/down) 2. Implement `output-store.ts` (visual overrides only for now) 3. Build `TrainingControls` (Add/Train/Clear/Randomize) 4. Build `RLControls` (thumbs up/down, noise display) 5. Build `LossPlot` (canvas) 6. Build `StatusLine` 7. Implement undo stack in `ml-store` 8. Implement `useOutputRouting` (visual mode only) **Validates**: Async training with UI updates, RL feedback loop, undo. ### Phase 3: Synth Integration **Goal**: Synth output mode working. 1. Transplant `c15-adapter.ts`, `c15-bridge.ts`, `param-map.ts`, `presets.ts` 2. Implement `synth-store.ts` 3. Implement `useAudioContext` (lazy creation, gesture resume) 4. Build `SynthControls` (start/stop, volume) 5. Build `SynthVisualizer` (canvas, param bars) 6. Build `PresetSelector` 7. Build `EngineSwitcher` (C15, Additive, FM) 8. Implement synth output routing with throttling 9. Implement `output-store` grouped overrides for C15 10. Build `Heatmap` + `ParamEditor` popup **Validates**: Multi-engine support, override system, audio integration. ### Phase 4: Layout + UI Primitives **Goal**: Full immersive UI. 1. Build headless primitives: `Drawer`, `Overlay`, `Dock`, `PillToggle`, `Slider` 2. Build `ImmersiveLayout` (fullscreen canvas + floating controls + drawer stack) 3. Build `ControlSurface` (Boldness/Memory/Precision compound axes) 4. Implement `useInputPipeline` (deadzone, zoom, curve, smoothing) 5. Build `JoyMap` (zoom minimap with trails, canvas) 6. Build `InputHeatmap` (2D color field, canvas) 7. Implement `useKeyboard` (shortcuts: 1/2/Z etc.) 8. Wire drawer system (dock icons → drawer toggles) **Validates**: Layout primitives, compound axis system, input pipeline. ### Phase 5: Remaining Features **Goal**: Feature parity with a-immersive. 1. **MIDI**: `MIDIInputPanel`, `MIDICCPanel`, `MIDIPresets`, midi-cc output mode 2. **Audio Canvas**: Transplant, wire as output mode 3. **EOC Chain**: `EOCPanel`, `EOCModule`, EOC joystick, nisps modes (Shared/Linked/Independent) 4. **Arpeggiator**: Controls, worker integration 5. **Output Pipeline**: `useOutputPipeline` (slew, smoothing, freeze) 6. **Weight Health**: `WeightHealth`, `GradientFlow` (canvas) 7. **Session Presets**: Save/load full state, URL sharing 8. **Persistence**: Auto-save, localStorage round-trip 9. **Debug probe**: `window.__nisps` when `?debug=1` ### Phase 6: ShapeSeq **Goal**: Sequencer as optional subsystem. 1. Transplant sequencer core (already modular) 2. Build `ShapeSeqPanel`, `StepVisualizer`, `ChainBuilder` 3. Wire to signal bus (`seq.*` topics) 4. Lazy-load when enabled via URL param ### Phase 7: Polish + Tests 1. Port Playwright e2e tests (update selectors for new DOM) 2. Add component-level tests (vitest + solid-testing-library) 3. Responsive CSS pass 4. Performance profiling (ensure 60fps inference + rendering) 5. Accessibility pass on interactive elements --- ## Fresh Eyes: What the Plan Was Missing ### 1. COOP/COEP Headers & SharedArrayBuffer (Blocker) The C15 synth engine requires `SharedArrayBuffer` for its audio ring buffer. This means the server **must** send: ``` Cross-Origin-Opener-Policy: same-origin Cross-Origin-Embedder-Policy: require-corp ``` The current codebase uses `serve-coop.py` for this. **Vite dev server must be configured** with these headers or C15 synth mode will fail silently. This should be validated in Phase 1, not Phase 3. ```typescript // vite.config.ts export default defineConfig({ plugins: [solidPlugin()], server: { headers: { 'Cross-Origin-Opener-Policy': 'same-origin', 'Cross-Origin-Embedder-Policy': 'require-corp', }, }, }); ``` ### 2. Worker Lifecycle (Currently Leaked) Workers are **never terminated** in the current code: - `nisps-wasm-worker.js` — created lazily on first `trainAsync()`, lives forever - `arpeggiator-worker.js` — created on arpeggiator init, lives forever In a SolidJS app where components mount/unmount, workers must be cleaned up in `onCleanup()`. The current code leaks them because the page never changes — the SolidJS version will need explicit termination, especially if engine switching or mode changes recreate IML instances. **Add to architecture**: `core/iml.ts` must expose a `dispose()` method that terminates the worker. `synth-store` cleanup must terminate the arpeggiator worker. ### 3. AudioContext Lifecycle Gotchas - **C15Bridge creates its own AudioContext internally**; Faust engines require one passed in. This API inconsistency means `useAudioContext()` can't own the single context — C15 creates its own. - **No `dispose()` on C15Bridge** — relies on GC of the AudioContext. Engine switching disconnects audio nodes (line 1121 in a-app.js) but doesn't explicitly close the context. - **AudioWorklet module loading** via `addModule()` — browsers deduplicate, but if the context is recreated, modules must be re-registered. **Add to architecture**: The `useAudioContext` hook needs to handle two patterns: "I own the context" (Faust) vs "the engine owns the context" (C15). Consider normalizing this in the transplant — make C15Bridge accept an external AudioContext. ### 4. Debug Probe Synchrony Contract The Playwright tests depend on `window.__nisps` methods being **synchronous**: - `setInputs(x, y)` → synchronously runs inference + routes outputs + updates heatmap - `thumbsUp()` → synchronously calls `addExample()` before returning - `saveState()` → synchronously writes to localStorage - `train()` → synchronously trains and returns loss (not the async variant) In SolidJS, store updates are batched by default. If `setInputs()` writes to a signal but the effect that runs inference hasn't flushed yet, the probe will return stale data. **The probe must bypass SolidJS reactivity** and call imperative methods directly on the IML instance. **Add to Phase 5 (debug probe)**: Build the probe as a direct imperative bridge to the IML/store internals, not as a reactive consumer. Use `batch()` or `untrack()` where needed. ### 5. MIDI CC Map — Engine-Scoped Dynamic Storage Keys MIDI CC maps are stored with engine-scoped localStorage keys: `nisps-midi-cc-map:${activeEngine.id}`. When the engine switches: 1. Current map is saved to the old engine's key 2. New map is loaded from the new engine's key 3. `midiCCMap` and `midiCCOverrides` arrays are mutated in-place In SolidJS, in-place array mutation (`arr.length = 0; arr.push(...)`) won't trigger reactivity. The `midi-store` must use `setStore(produce(...))` or replace arrays entirely. The engine-scoped key pattern needs to be replicated in `session-store.ts`. ### 6. EOC Chain — Mutable Audio Graph + MLP Resize Cascade When EOC modules are added/removed in "Shared" mode: 1. MLP output count changes → MLP must be destroyed and recreated 2. Training examples are lost (different output dimensionality) 3. Audio graph nodes must be reconnected 4. Heatmap must be rebuilt This is a **cascade of side effects** triggered by a single user action. In the current code it's handled by a `window.addEventListener('eoc:change', ...)` handler that orchestrates everything imperatively. In SolidJS, this should be modeled as: EOC module list is a store → derived signal computes total output count → `createEffect` watches output count and triggers MLP resize when it changes. But the MLP resize is async (WASM allocation) and has a confirmation dialog ("this will clear examples"). **Effects can't show dialogs**. **Proposed pattern**: EOC store exposes a `pendingResize` signal. A component watches it and shows the confirmation UI. On confirm, an action triggers the actual resize. Don't try to make this fully reactive — keep it as an explicit action flow. ### 7. WASM + Emscripten Glue Loading in Vite The WASM is loaded via Emscripten's `nisps.js` glue file, which does its own `fetch()` of `nisps.wasm` using a relative path. Vite's asset handling will hash filenames in production builds, breaking the hardcoded path. **Options**: - Configure Vite to copy WASM files to `public/` (no hashing, always available at known path) - Modify the Emscripten glue to accept a custom `locateFile` override - Use Vite's `?url` import to get the resolved asset path and pass it to the WASM loader This must be spiked in Phase 1. Same issue applies to C15 WASM (`c15/c15_engine.wasm`), C15 parameters (`c15/parameters.json`), and Faust DSP files. ### 8. Session Presets Are Shared Across Apps `nisps-session-presets` localStorage key is **shared across all three current apps**. Since we're consolidating to one app, this is fine — but the key should be documented, and the migration should handle importing presets saved by the old app. ### 9. Lazy Loading Needs Suspense Boundaries Three features are lazily loaded: - **ShapeSeq**: dynamic `import()` when `?shapeseq=1` - **Hand tracking**: imports MediaPipe from CDN (`cdn.jsdelivr.net`) — external dependency that can fail - **Audio Canvas**: created on first switch to audio-canvas mode SolidJS `lazy()` + `` handles this naturally, but: - MediaPipe CDN fetch failure needs a fallback UI (not just a blank screen) - ShapeSeq lazy loading should show a loading state, not block the whole app - Audio Canvas creation involves AudioContext (requires gesture) — can't be wrapped in Suspense naively ### 10. CSS Animations Are Stateful Several CSS classes trigger animations that encode UI state: - `.follow-pulse` — 1.5s infinite pulse (follow mode active) - `.btn-flash` / `.rl-flash` — 0.2s feedback flash - `.drawerSlideIn` — drawer appearance If SolidJS re-renders a component (e.g., `` toggling), CSS animations restart from the beginning. For the pulse animation this is fine, but for the flash animations, a re-render mid-flash would cause visual glitches. **Mitigation**: Use `classList` toggling on stable DOM nodes rather than conditional rendering for animation-bearing elements. Or use the Web Animations API for imperative control. ### 11. `window.__nispsEoc` Is Unconditional Unlike `window.__nisps` (gated by `?debug=1`), `window.__nispsEoc` is **always exposed** (line 1528 in a-app.js). It provides `trainingTarget` getter/setter and `imlEoc` reference. If external code depends on this, it needs to be preserved in the SolidJS version unconditionally. ### 12. Inference Should NOT Be in rAF The plan puts inference in a `requestAnimationFrame` loop. But inference only needs to run **when inputs change** (joystick drag, gamepad poll, hand tracking frame). Running it every frame when the joystick is idle wastes CPU. **Better pattern**: Run inference reactively — `createEffect` watching `inputState.joyX` and `inputState.joyY`. When they change, run inference and update outputs. The canvas render loops (FlowField, SynthVisualizer) still run on rAF for smooth animation, but they just read the latest outputs signal — they don't trigger inference. Exception: gamepad polling needs a rAF loop to read the Gamepad API, but that loop should only set input signals, not run inference directly. --- ## Key Risks & Mitigations | Risk | Mitigation | |------|------------| | WASM + SolidJS reactivity overhead | Typed arrays stay as signals (not store properties). Batch per frame. Profile early in Phase 1. | | 126 params × 20fps = 2520 synth param sends/sec | Keep existing throttle (50ms interval, 0.002 dead zone) in synth subscriber, not in routing. | | Store proxy overhead on large arrays | Use `createSignal` for `Float32Array` outputs, not `createStore`. Stores only for structured config. | | Signal bus wildcard `match()` perf | Memoize prefix scans. Most topics are fixed at startup. Profile if >50 topics. | | Canvas components fighting rAF | Each canvas owns its loop. No shared orchestrator unless profiling shows frame contention. | | MLP resize destroys training data | Same as current: warn user, warm-start weights for joystick IML. Store handles the state transition. | | `equals: false` on bus signals | Required for event semantics but means every emit triggers all subscribers. Keep topic count bounded. | | SharedArrayBuffer / COOP+COEP | Configure Vite dev server headers in Phase 1. Validate C15 works before Phase 3. | | WASM asset paths broken by Vite hashing | Spike in Phase 1: use `public/` dir or `locateFile` override for all WASM/JSON assets. | | Worker leak on component unmount | Add `dispose()` to IML and arpeggiator. Call in `onCleanup()`. | | EOC resize cascade needs confirmation dialog | Model as pending action, not reactive effect. Component shows dialog, action triggers resize. | | Debug probe expects synchronous execution | Build probe as imperative bridge, bypass SolidJS batching with `batch()`/`untrack()`. | | CSS animation restart on re-render | Use `classList` on stable nodes, not ``/`` for animated elements. | | Idle inference wastes CPU | Make inference reactive to input changes, not rAF-driven. Canvas loops stay on rAF. | --- ## What Gets Transplanted vs Rewritten ### Transplanted (minimal changes, mostly just TS types) - `nisps-wasm.js` + worker + WASM binary - `dataset.ts` (FIFO ring buffer) - `FlowFieldVisualizer` class (canvas rendering) - `c15-bridge.ts`, `c15-adapter.ts` - `param-map.ts`, `presets.ts` (pure data) - `additive-engine.ts`, `fm-engine.ts`, `faust-engine-base.ts` - `arpeggiator.ts` + worker - `eoc-chain.ts`, `eoc-module.ts`, all effect modules - ShapeSeq core (`sequencer.ts`, `chain.ts`, `clock.ts`, `pattern.ts`, `primitives.ts`) - `audio-canvas.ts` - `input-pipeline.ts` logic (becomes `useInputPipeline` hook wrapping same math) - `output-pipeline.ts` logic (becomes `useOutputPipeline` hook) - `control-surface.ts` compound axis logic (data tables + interpolation) ### Rewritten from scratch - All DOM manipulation (→ JSX components) - Event wiring (→ signal bus + reactive effects) - State management (scattered module-scope vars → stores) - Override application (→ `apply-overrides.ts` pure function) - Persistence (→ `SessionProvider` with `createEffect` auto-save) - Layout/CSS (→ new CSS with headless primitives) - Init/boot sequence (→ provider tree + `onMount` hooks) ### Deleted (not ported) - `b-app.js`, `c-app.js`, `app.js` (consolidated into one app) - `b-workbench.html`, `c-journey.html`, `index.html` (single entry point) - `iml.js`, `mlp.js`, `layer.js`, `node.js` (legacy JS ML engine, WASM only) - `event-bus.js` (replaced by signal bus) - All `wire*()` functions (replaced by component-local event handlers) - DOM-string-building functions (`buildHeatmap`, `buildDrawerHTML`, etc.) --- ## Open Questions 1. **TypeScript strictness**: Full strict mode from day one, or gradual? (Transplanted JS modules will need type annotations.) 2. **CSS approach**: Plain CSS files per component? CSS modules? Vanilla Extract? (Plain CSS is simplest and matches the current approach.) 3. **Testing during migration**: Run old Playwright tests against old code in parallel, or wait for Phase 7? 4. **WASM loading**: Vite handles `.wasm` imports natively, but the Emscripten glue (`nisps.js`) may need special config. Spike this in Phase 1. 5. **Faust DSP loading**: Currently fetched at runtime. Keep as-is or bundle?