Add SolidJS migration plan
This commit is contained in:
parent
b72914813c
commit
47b6b1466b
1 changed files with 690 additions and 0 deletions
690
playground/PLAN-solidjs-migration.md
Normal file
690
playground/PLAN-solidjs-migration.md
Normal file
|
|
@ -0,0 +1,690 @@
|
||||||
|
# SolidJS Migration Plan
|
||||||
|
|
||||||
|
## Decision Record
|
||||||
|
|
||||||
|
| Decision | Choice |
|
||||||
|
|----------|--------|
|
||||||
|
| Framework | SolidJS (reactivity + components) |
|
||||||
|
| Build | Vite + vite-plugin-solid |
|
||||||
|
| Apps | Immersive only, extensible for future layouts |
|
||||||
|
| Core integration | Deep — ML/synth/audio modeled as SolidJS stores/signals |
|
||||||
|
| Output reactivity | Batch at frame rate (one signal update per rAF) |
|
||||||
|
| Event system | Unified signal bus with topics (replaces EventBus + CustomEvents) |
|
||||||
|
| Migration strategy | Big bang rewrite, old code as reference |
|
||||||
|
| Routing | None — signals only, URL params for config |
|
||||||
|
| Canvas | Components with refs, internal render loops |
|
||||||
|
| Mobile | Desktop-first, mobile later |
|
||||||
|
| Layout | Headless UI primitives (Drawer, Overlay, Panel, Dock) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Architecture Overview
|
||||||
|
|
||||||
|
```
|
||||||
|
src/
|
||||||
|
├── index.html # Single entry point
|
||||||
|
├── App.tsx # Root component, provider tree
|
||||||
|
├── vite.config.ts
|
||||||
|
│
|
||||||
|
├── core/ # Framework-agnostic engines (transplanted)
|
||||||
|
│ ├── wasm/ # WASM binaries + Emscripten glue (unchanged)
|
||||||
|
│ │ ├── nisps.wasm
|
||||||
|
│ │ ├── nisps.js
|
||||||
|
│ │ └── nisps-wasm-worker.js
|
||||||
|
│ ├── iml.ts # WasmIML wrapper (thin adaptation)
|
||||||
|
│ ├── dataset.ts # Training dataset (FIFO ring buffer)
|
||||||
|
│ ├── synth/ # Synth engines (transplanted, minimal changes)
|
||||||
|
│ │ ├── engine-interface.ts
|
||||||
|
│ │ ├── c15-adapter.ts
|
||||||
|
│ │ ├── c15-bridge.ts
|
||||||
|
│ │ ├── additive-engine.ts
|
||||||
|
│ │ ├── fm-engine.ts
|
||||||
|
│ │ ├── faust-engine-base.ts
|
||||||
|
│ │ ├── param-map.ts # 126 C15 params (data, unchanged)
|
||||||
|
│ │ └── presets.ts # Synth presets (data, unchanged)
|
||||||
|
│ ├── audio/
|
||||||
|
│ │ ├── arpeggiator.ts
|
||||||
|
│ │ ├── arpeggiator-worker.js
|
||||||
|
│ │ ├── audio-canvas.ts
|
||||||
|
│ │ └── midi-io.ts # MIDIInput + MIDIOutput merged
|
||||||
|
│ ├── eoc/ # Effects chain (transplanted)
|
||||||
|
│ │ ├── eoc-chain.ts
|
||||||
|
│ │ ├── eoc-module.ts
|
||||||
|
│ │ └── modules/ # Individual effect modules
|
||||||
|
│ └── shapeseq/ # Sequencer (transplanted, minimal changes)
|
||||||
|
│ ├── sequencer.ts
|
||||||
|
│ ├── chain.ts
|
||||||
|
│ ├── clock.ts
|
||||||
|
│ ├── pattern.ts
|
||||||
|
│ ├── primitives.ts
|
||||||
|
│ └── ...
|
||||||
|
│
|
||||||
|
├── bus/ # Signal bus
|
||||||
|
│ └── signal-bus.ts # createSignalBus(), topic(), match(), emit()
|
||||||
|
│
|
||||||
|
├── stores/ # SolidJS stores (the reactive state layer)
|
||||||
|
│ ├── ml-store.ts # IML instances, outputs, loss, training state
|
||||||
|
│ ├── input-store.ts # Joystick position, input mode, pipeline config
|
||||||
|
│ ├── output-store.ts # Output mode, overrides, routing config
|
||||||
|
│ ├── synth-store.ts # Active engine, arpeggiator, volume, presets
|
||||||
|
│ ├── eoc-store.ts # EOC chain state, nisps mode, modules
|
||||||
|
│ ├── midi-store.ts # MIDI CC map, devices, output state
|
||||||
|
│ ├── session-store.ts # Persistence (save/load localStorage)
|
||||||
|
│ └── ui-store.ts # Drawer state, active panels, help seen
|
||||||
|
│
|
||||||
|
├── hooks/ # Reactive glue (SolidJS "hooks" / composables)
|
||||||
|
│ ├── useInference.ts # Per-frame inference loop
|
||||||
|
│ ├── useTraining.ts # Train/thumbs-up/thumbs-down actions
|
||||||
|
│ ├── useOutputRouting.ts # Route outputs → synth/visual/midi/audio-canvas
|
||||||
|
│ ├── useInputPipeline.ts # Deadzone, zoom, curve, smoothing, momentum
|
||||||
|
│ ├── useOutputPipeline.ts # Slew rate, smoothing, freeze gate
|
||||||
|
│ ├── usePersistence.ts # Auto-save/load, URL param parsing
|
||||||
|
│ ├── useAudioContext.ts # Lazy AudioContext creation, resume on gesture
|
||||||
|
│ ├── useGamepad.ts # Gamepad polling
|
||||||
|
│ └── useKeyboard.ts # Keyboard shortcuts
|
||||||
|
│
|
||||||
|
├── primitives/ # Headless UI primitives
|
||||||
|
│ ├── Drawer.tsx # Slide-in panel (headless)
|
||||||
|
│ ├── Overlay.tsx # Floating positioned element
|
||||||
|
│ ├── Panel.tsx # Collapsible content section
|
||||||
|
│ ├── Dock.tsx # Icon bar with drawer triggers
|
||||||
|
│ ├── PillToggle.tsx # Segmented toggle (output mode, input mode)
|
||||||
|
│ ├── Slider.tsx # Range input with label/value display
|
||||||
|
│ └── Canvas.tsx # Canvas wrapper with ref + resize observer
|
||||||
|
│
|
||||||
|
├── components/ # Feature components
|
||||||
|
│ ├── app/
|
||||||
|
│ │ └── ImmersiveLayout.tsx # Main layout shell
|
||||||
|
│ ├── input/
|
||||||
|
│ │ ├── Joystick.tsx # Virtual joystick (pointer events)
|
||||||
|
│ │ ├── JoyMap.tsx # Zoom minimap + trails (canvas)
|
||||||
|
│ │ └── InputModeToggle.tsx
|
||||||
|
│ ├── output/
|
||||||
|
│ │ ├── FlowField.tsx # Particle visualizer (canvas)
|
||||||
|
│ │ ├── SynthVisualizer.tsx # Param bar chart (canvas)
|
||||||
|
│ │ ├── Heatmap.tsx # Parameter heatmap grid
|
||||||
|
│ │ └── OutputModeToggle.tsx
|
||||||
|
│ ├── training/
|
||||||
|
│ │ ├── TrainingControls.tsx # Add/Train/Clear/Randomize buttons
|
||||||
|
│ │ ├── RLControls.tsx # Thumbs up/down, noise display
|
||||||
|
│ │ ├── LossPlot.tsx # Loss history (canvas)
|
||||||
|
│ │ └── StatusLine.tsx # Example count, loss, mode
|
||||||
|
│ ├── synth/
|
||||||
|
│ │ ├── SynthControls.tsx # Start/stop, volume, arp controls
|
||||||
|
│ │ ├── PresetSelector.tsx # Synth preset chips
|
||||||
|
│ │ ├── EngineSwitcher.tsx # Engine dropdown
|
||||||
|
│ │ └── ParamEditor.tsx # Per-param override popup
|
||||||
|
│ ├── eoc/
|
||||||
|
│ │ ├── EOCPanel.tsx # Effects chain UI
|
||||||
|
│ │ ├── EOCModule.tsx # Single effect module card
|
||||||
|
│ │ └── EOCJoystick.tsx # Independent mode joystick
|
||||||
|
│ ├── midi/
|
||||||
|
│ │ ├── MIDIInputPanel.tsx # Device select, CC mapping
|
||||||
|
│ │ ├── MIDICCPanel.tsx # CC output config
|
||||||
|
│ │ └── MIDIPresets.tsx # CC preset management
|
||||||
|
│ ├── shapeseq/
|
||||||
|
│ │ ├── ShapeSeqPanel.tsx # Sequencer controls
|
||||||
|
│ │ ├── StepVisualizer.tsx # Step display (canvas)
|
||||||
|
│ │ └── ChainBuilder.tsx # Primitive chain editor
|
||||||
|
│ ├── controls/
|
||||||
|
│ │ ├── ControlSurface.tsx # Boldness/Memory/Precision axes
|
||||||
|
│ │ ├── InputHeatmap.tsx # 2D input space heatmap (canvas)
|
||||||
|
│ │ └── EngineParams.tsx # Spread, noise, decay sliders
|
||||||
|
│ └── debug/
|
||||||
|
│ ├── DevPanel.tsx # Debug tools
|
||||||
|
│ ├── WeightHealth.tsx # Weight magnitude histogram (canvas)
|
||||||
|
│ └── GradientFlow.tsx # Per-layer gradient analysis (canvas)
|
||||||
|
│
|
||||||
|
├── actions/ # Imperative operations (not reactive)
|
||||||
|
│ ├── resize-mlp.ts # Resize MLP, warm-start weights
|
||||||
|
│ ├── apply-overrides.ts # Override application logic
|
||||||
|
│ └── export-import.ts # Session export/import
|
||||||
|
│
|
||||||
|
└── assets/
|
||||||
|
├── c15/ # C15 WASM synth binary
|
||||||
|
└── faust/ # Faust DSP files
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Store Design (Deep Integration)
|
||||||
|
|
||||||
|
### ML Store (`stores/ml-store.ts`)
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { createStore, produce } from "solid-js/store";
|
||||||
|
import { createSignal } from "solid-js";
|
||||||
|
|
||||||
|
// The hot path: batched per-frame, NOT per-parameter
|
||||||
|
const [outputs, setOutputs] = createSignal(new Float32Array(126));
|
||||||
|
const [rawParams, setRawParams] = createSignal(new Float32Array(126));
|
||||||
|
|
||||||
|
const [mlState, setMlState] = createStore({
|
||||||
|
// IML instances (not reactive themselves — opaque handles)
|
||||||
|
imlJoy: null as WasmIML | null,
|
||||||
|
imlHand: null as WasmIML | null,
|
||||||
|
activeIml: 'joy' as 'joy' | 'hand',
|
||||||
|
|
||||||
|
// Reactive state derived from IML
|
||||||
|
outputCount: 126,
|
||||||
|
loss: null as number | null,
|
||||||
|
lossHistory: [] as number[],
|
||||||
|
exampleCount: 0,
|
||||||
|
isTraining: false,
|
||||||
|
layerStats: null,
|
||||||
|
|
||||||
|
// RL config
|
||||||
|
spreadLevel: 0.6,
|
||||||
|
noiseLevel: 0.05,
|
||||||
|
rlDecay: 0.97,
|
||||||
|
learningRate: 0.1,
|
||||||
|
maxIterations: 50,
|
||||||
|
|
||||||
|
// Undo
|
||||||
|
undoStack: [] as Float32Array[],
|
||||||
|
});
|
||||||
|
|
||||||
|
// The IML instances live OUTSIDE the store (mutable, non-proxy-safe).
|
||||||
|
// The store tracks their *state* reactively.
|
||||||
|
// After each inference: setOutputs(iml.getOutputs())
|
||||||
|
// After each train: setMlState({ loss, exampleCount, lossHistory })
|
||||||
|
```
|
||||||
|
|
||||||
|
**Key design choice**: `Float32Array` outputs are a signal, not a store property. Stores use proxies which don't play well with typed arrays. A signal holding the array reference, replaced each frame, gives us batch reactivity cheaply.
|
||||||
|
|
||||||
|
### Input Store (`stores/input-store.ts`)
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const [inputState, setInputState] = createStore({
|
||||||
|
mode: 'joystick' as 'joystick' | 'hands',
|
||||||
|
joyX: 0.5,
|
||||||
|
joyY: 0.5,
|
||||||
|
isDragging: false,
|
||||||
|
followMode: false,
|
||||||
|
|
||||||
|
// Input pipeline config
|
||||||
|
pipeline: {
|
||||||
|
deadzone: 0.02,
|
||||||
|
zoom: 1.0,
|
||||||
|
zoomAnchorX: 0.5,
|
||||||
|
zoomAnchorY: 0.5,
|
||||||
|
anchorMode: 'auto' as 'auto' | 'sticky' | 'center',
|
||||||
|
curve: 1.0, // 1.0 = linear
|
||||||
|
smoothing: 0.0, // 0.0 = none
|
||||||
|
momentumZoom: false,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
### Output Store (`stores/output-store.ts`)
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const [outputState, setOutputState] = createStore({
|
||||||
|
mode: 'visual' as 'visual' | 'synth' | 'midi-cc' | 'audio-canvas',
|
||||||
|
|
||||||
|
// Per-mode overrides (arrays of { min, max, curve, muted, fixedValue })
|
||||||
|
visualOverrides: [] as ParamOverride[],
|
||||||
|
synthOverrides: {
|
||||||
|
type: 'grouped' as 'grouped' | 'flat',
|
||||||
|
groups: [] as GroupOverride[], // C15 grouped
|
||||||
|
flat: [] as ParamOverride[], // Faust flat
|
||||||
|
},
|
||||||
|
midiCCOverrides: [] as ParamOverride[],
|
||||||
|
audioCanvasOverrides: [] as ParamOverride[],
|
||||||
|
|
||||||
|
// Output pipeline config
|
||||||
|
pipeline: {
|
||||||
|
globalCurve: 1.0,
|
||||||
|
smoothing: 0.0,
|
||||||
|
slewRate: 1.0,
|
||||||
|
freezeGate: false,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
### Synth Store (`stores/synth-store.ts`)
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
const [synthState, setSynthState] = createStore({
|
||||||
|
engineId: 'shaper-feedback' as string,
|
||||||
|
engine: null as SynthEngine | null, // Opaque handle
|
||||||
|
isRunning: false,
|
||||||
|
volume: 0.7,
|
||||||
|
|
||||||
|
// Arpeggiator
|
||||||
|
arp: {
|
||||||
|
enabled: false,
|
||||||
|
tempo: 120,
|
||||||
|
progression: 'major' as string,
|
||||||
|
octaves: 1,
|
||||||
|
offset: 0,
|
||||||
|
},
|
||||||
|
|
||||||
|
// Current preset
|
||||||
|
presetId: null as string | null,
|
||||||
|
});
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Signal Bus Design (`bus/signal-bus.ts`)
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { createSignal, createMemo } from "solid-js";
|
||||||
|
|
||||||
|
type Topic<T = any> = {
|
||||||
|
(): T | undefined; // Read (reactive)
|
||||||
|
fire: (data: T) => void; // Write (imperative)
|
||||||
|
};
|
||||||
|
|
||||||
|
export function createSignalBus() {
|
||||||
|
const topics = new Map<string, ReturnType<typeof createSignal>>();
|
||||||
|
|
||||||
|
function topic<T = any>(name: string): Topic<T> {
|
||||||
|
if (!topics.has(name)) {
|
||||||
|
const [get, set] = createSignal<T | undefined>(undefined, { equals: false });
|
||||||
|
topics.set(name, [get, set]);
|
||||||
|
}
|
||||||
|
const [get, set] = topics.get(name)!;
|
||||||
|
const accessor = () => get() as T | undefined;
|
||||||
|
accessor.fire = (data: T) => set(() => data);
|
||||||
|
return accessor as Topic<T>;
|
||||||
|
}
|
||||||
|
|
||||||
|
function emit<T = any>(name: string, data: T) {
|
||||||
|
topic<T>(name).fire(data);
|
||||||
|
}
|
||||||
|
|
||||||
|
function match(pattern: string) {
|
||||||
|
// 'seq.*' → derived signal merging all seq.* topics
|
||||||
|
const prefix = pattern.replace('*', '');
|
||||||
|
return createMemo(() => {
|
||||||
|
let latest: { name: string; data: any } | undefined;
|
||||||
|
for (const [name, [get]] of topics) {
|
||||||
|
if (name.startsWith(prefix)) {
|
||||||
|
const val = get();
|
||||||
|
if (val !== undefined) {
|
||||||
|
latest = { name, data: val };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return latest;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
return { topic, emit, match };
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Usage equals: false** is critical — it means every `emit()` triggers subscribers even if the data is identical. This matches event semantics (every `seq.step` matters, even if the step number repeats).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Inference Loop (`hooks/useInference.ts`)
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { createEffect, onCleanup } from "solid-js";
|
||||||
|
|
||||||
|
export function useInference(mlStore, inputStore, outputStore, bus) {
|
||||||
|
let rafId: number;
|
||||||
|
|
||||||
|
function tick() {
|
||||||
|
const iml = mlStore.activeIml === 'joy' ? mlStore.imlJoy : mlStore.imlHand;
|
||||||
|
if (!iml) { rafId = requestAnimationFrame(tick); return; }
|
||||||
|
|
||||||
|
// Set inputs
|
||||||
|
iml.setInput(0, inputStore.joyX);
|
||||||
|
iml.setInput(1, inputStore.joyY);
|
||||||
|
|
||||||
|
// Forward pass
|
||||||
|
iml.process();
|
||||||
|
const raw = iml.getOutputs();
|
||||||
|
|
||||||
|
// Batch update — single signal write per frame
|
||||||
|
setOutputs(new Float32Array(raw));
|
||||||
|
|
||||||
|
// Route to active output
|
||||||
|
routeOutputs(raw, outputStore, synthStore, bus);
|
||||||
|
|
||||||
|
// Fire bus event (for non-UI subscribers like sequencer)
|
||||||
|
bus.emit('ml.inference', { outputs: raw });
|
||||||
|
|
||||||
|
rafId = requestAnimationFrame(tick);
|
||||||
|
}
|
||||||
|
|
||||||
|
rafId = requestAnimationFrame(tick);
|
||||||
|
onCleanup(() => cancelAnimationFrame(rafId));
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Output Routing (`hooks/useOutputRouting.ts`)
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
function routeOutputs(raw: Float32Array, outputStore, synthStore, bus) {
|
||||||
|
const mode = outputStore.mode;
|
||||||
|
const overrides = getOverridesForMode(mode, outputStore);
|
||||||
|
|
||||||
|
// Apply overrides (pure function, no side effects)
|
||||||
|
const processed = applyOverrides(raw, overrides);
|
||||||
|
|
||||||
|
switch (mode) {
|
||||||
|
case 'visual':
|
||||||
|
bus.emit('output.visual', processed);
|
||||||
|
break;
|
||||||
|
case 'synth':
|
||||||
|
// Throttled: only send to engine at 20fps
|
||||||
|
bus.emit('output.synth', processed);
|
||||||
|
break;
|
||||||
|
case 'midi-cc':
|
||||||
|
bus.emit('output.midi', processed);
|
||||||
|
break;
|
||||||
|
case 'audio-canvas':
|
||||||
|
bus.emit('output.audiocanvas', processed);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Always update heatmap (unthrottled — it's just DOM)
|
||||||
|
bus.emit('output.heatmap', raw);
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Each output component subscribes to its own bus topic. The FlowField listens to `output.visual`, the SynthVisualizer to `output.synth`, etc. Throttling for synth param sends lives inside the synth subscriber, not in the routing.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Component Examples
|
||||||
|
|
||||||
|
### Joystick Component
|
||||||
|
|
||||||
|
```tsx
|
||||||
|
const Joystick = () => {
|
||||||
|
const { inputState, setInputState } = useInputStore();
|
||||||
|
const bus = useBus();
|
||||||
|
|
||||||
|
const onPointerDown = (e: PointerEvent) => {
|
||||||
|
(e.target as Element).setPointerCapture(e.pointerId);
|
||||||
|
const rect = (e.target as Element).getBoundingClientRect();
|
||||||
|
const x = (e.clientX - rect.left) / rect.width;
|
||||||
|
const y = 1 - (e.clientY - rect.top) / rect.height;
|
||||||
|
setInputState({ joyX: x, joyY: y, isDragging: true });
|
||||||
|
};
|
||||||
|
|
||||||
|
const onPointerMove = (e: PointerEvent) => {
|
||||||
|
if (!inputState.isDragging) return;
|
||||||
|
const rect = (e.target as Element).getBoundingClientRect();
|
||||||
|
const x = Math.max(0, Math.min(1, (e.clientX - rect.left) / rect.width));
|
||||||
|
const y = Math.max(0, Math.min(1, 1 - (e.clientY - rect.top) / rect.height));
|
||||||
|
setInputState({ joyX: x, joyY: y });
|
||||||
|
};
|
||||||
|
|
||||||
|
const onPointerUp = () => setInputState({ isDragging: false });
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div
|
||||||
|
class="joystick"
|
||||||
|
onPointerDown={onPointerDown}
|
||||||
|
onPointerMove={onPointerMove}
|
||||||
|
onPointerUp={onPointerUp}
|
||||||
|
>
|
||||||
|
<div
|
||||||
|
class="joystick-dot"
|
||||||
|
style={{
|
||||||
|
left: `${inputState.joyX * 100}%`,
|
||||||
|
bottom: `${inputState.joyY * 100}%`,
|
||||||
|
}}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
### FlowField (Canvas Component)
|
||||||
|
|
||||||
|
```tsx
|
||||||
|
const FlowField = () => {
|
||||||
|
let canvasRef: HTMLCanvasElement;
|
||||||
|
const bus = useBus();
|
||||||
|
const visualData = bus.topic<Float32Array>('output.visual');
|
||||||
|
|
||||||
|
onMount(() => {
|
||||||
|
const ctx = canvasRef.getContext('2d')!;
|
||||||
|
const vis = new FlowFieldVisualizer(ctx); // Transplanted engine
|
||||||
|
|
||||||
|
// Internal render loop — reads signal each frame
|
||||||
|
let raf: number;
|
||||||
|
const loop = () => {
|
||||||
|
const data = visualData();
|
||||||
|
if (data) vis.setParams(data);
|
||||||
|
vis.draw();
|
||||||
|
raf = requestAnimationFrame(loop);
|
||||||
|
};
|
||||||
|
raf = requestAnimationFrame(loop);
|
||||||
|
onCleanup(() => cancelAnimationFrame(raf));
|
||||||
|
});
|
||||||
|
|
||||||
|
return <canvas ref={canvasRef!} class="flow-field" />;
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
### Heatmap (Reactive DOM)
|
||||||
|
|
||||||
|
```tsx
|
||||||
|
const Heatmap = () => {
|
||||||
|
const bus = useBus();
|
||||||
|
const heatmapData = bus.topic<Float32Array>('output.heatmap');
|
||||||
|
const { outputState } = useOutputStore();
|
||||||
|
const paramMeta = () => getParamMeta(outputState.mode);
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div class="heatmap-grid">
|
||||||
|
<For each={paramMeta()}>
|
||||||
|
{(param, i) => (
|
||||||
|
<HeatmapCell
|
||||||
|
index={i()}
|
||||||
|
param={param}
|
||||||
|
value={() => heatmapData()?.[i()] ?? 0}
|
||||||
|
/>
|
||||||
|
)}
|
||||||
|
</For>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
|
||||||
|
const HeatmapCell = (props) => {
|
||||||
|
const width = () => `${(props.value() * 100).toFixed(1)}%`;
|
||||||
|
return (
|
||||||
|
<div class="heatmap-cell" title={props.param.name}>
|
||||||
|
<div class="heatmap-bar" style={{ width: width() }} />
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Provider Tree (`App.tsx`)
|
||||||
|
|
||||||
|
```tsx
|
||||||
|
const App = () => {
|
||||||
|
return (
|
||||||
|
<BusProvider>
|
||||||
|
<MLProvider>
|
||||||
|
<InputProvider>
|
||||||
|
<OutputProvider>
|
||||||
|
<SynthProvider>
|
||||||
|
<EOCProvider>
|
||||||
|
<SessionProvider>
|
||||||
|
<ImmersiveLayout />
|
||||||
|
</SessionProvider>
|
||||||
|
</EOCProvider>
|
||||||
|
</SynthProvider>
|
||||||
|
</OutputProvider>
|
||||||
|
</InputProvider>
|
||||||
|
</MLProvider>
|
||||||
|
</BusProvider>
|
||||||
|
);
|
||||||
|
};
|
||||||
|
```
|
||||||
|
|
||||||
|
Each provider creates its store and exposes it via context. The `SessionProvider` handles persistence (auto-save on interval, load on mount, URL param parsing).
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Migration Phases
|
||||||
|
|
||||||
|
### Phase 1: Skeleton + Core Loop
|
||||||
|
|
||||||
|
**Goal**: Joystick → MLP → FlowField working in SolidJS.
|
||||||
|
|
||||||
|
1. `npm create vite@latest playground-solid -- --template solid-ts`
|
||||||
|
2. Set up project structure (`core/`, `stores/`, `components/`, `bus/`)
|
||||||
|
3. Transplant WASM files and `nisps-wasm.js` into `core/wasm/`
|
||||||
|
4. Implement `createSignalBus()`
|
||||||
|
5. Implement `ml-store.ts` and `input-store.ts` (minimal)
|
||||||
|
6. Implement `useInference` hook
|
||||||
|
7. Build `Joystick` component
|
||||||
|
8. Build `FlowField` component (transplant `FlowFieldVisualizer` class)
|
||||||
|
9. Wire it up in `App.tsx` with minimal `ImmersiveLayout`
|
||||||
|
10. Verify: drag joystick → see particles respond
|
||||||
|
|
||||||
|
**Validates**: SolidJS + WASM integration, signal bus, frame-rate batching, canvas components.
|
||||||
|
|
||||||
|
### Phase 2: Training + RL
|
||||||
|
|
||||||
|
**Goal**: Full ML interaction loop.
|
||||||
|
|
||||||
|
1. Implement `useTraining` hook (add example, train async, thumbs up/down)
|
||||||
|
2. Implement `output-store.ts` (visual overrides only for now)
|
||||||
|
3. Build `TrainingControls` (Add/Train/Clear/Randomize)
|
||||||
|
4. Build `RLControls` (thumbs up/down, noise display)
|
||||||
|
5. Build `LossPlot` (canvas)
|
||||||
|
6. Build `StatusLine`
|
||||||
|
7. Implement undo stack in `ml-store`
|
||||||
|
8. Implement `useOutputRouting` (visual mode only)
|
||||||
|
|
||||||
|
**Validates**: Async training with UI updates, RL feedback loop, undo.
|
||||||
|
|
||||||
|
### Phase 3: Synth Integration
|
||||||
|
|
||||||
|
**Goal**: Synth output mode working.
|
||||||
|
|
||||||
|
1. Transplant `c15-adapter.ts`, `c15-bridge.ts`, `param-map.ts`, `presets.ts`
|
||||||
|
2. Implement `synth-store.ts`
|
||||||
|
3. Implement `useAudioContext` (lazy creation, gesture resume)
|
||||||
|
4. Build `SynthControls` (start/stop, volume)
|
||||||
|
5. Build `SynthVisualizer` (canvas, param bars)
|
||||||
|
6. Build `PresetSelector`
|
||||||
|
7. Build `EngineSwitcher` (C15, Additive, FM)
|
||||||
|
8. Implement synth output routing with throttling
|
||||||
|
9. Implement `output-store` grouped overrides for C15
|
||||||
|
10. Build `Heatmap` + `ParamEditor` popup
|
||||||
|
|
||||||
|
**Validates**: Multi-engine support, override system, audio integration.
|
||||||
|
|
||||||
|
### Phase 4: Layout + UI Primitives
|
||||||
|
|
||||||
|
**Goal**: Full immersive UI.
|
||||||
|
|
||||||
|
1. Build headless primitives: `Drawer`, `Overlay`, `Dock`, `PillToggle`, `Slider`
|
||||||
|
2. Build `ImmersiveLayout` (fullscreen canvas + floating controls + drawer stack)
|
||||||
|
3. Build `ControlSurface` (Boldness/Memory/Precision compound axes)
|
||||||
|
4. Implement `useInputPipeline` (deadzone, zoom, curve, smoothing)
|
||||||
|
5. Build `JoyMap` (zoom minimap with trails, canvas)
|
||||||
|
6. Build `InputHeatmap` (2D color field, canvas)
|
||||||
|
7. Implement `useKeyboard` (shortcuts: 1/2/Z etc.)
|
||||||
|
8. Wire drawer system (dock icons → drawer toggles)
|
||||||
|
|
||||||
|
**Validates**: Layout primitives, compound axis system, input pipeline.
|
||||||
|
|
||||||
|
### Phase 5: Remaining Features
|
||||||
|
|
||||||
|
**Goal**: Feature parity with a-immersive.
|
||||||
|
|
||||||
|
1. **MIDI**: `MIDIInputPanel`, `MIDICCPanel`, `MIDIPresets`, midi-cc output mode
|
||||||
|
2. **Audio Canvas**: Transplant, wire as output mode
|
||||||
|
3. **EOC Chain**: `EOCPanel`, `EOCModule`, EOC joystick, nisps modes (Shared/Linked/Independent)
|
||||||
|
4. **Arpeggiator**: Controls, worker integration
|
||||||
|
5. **Output Pipeline**: `useOutputPipeline` (slew, smoothing, freeze)
|
||||||
|
6. **Weight Health**: `WeightHealth`, `GradientFlow` (canvas)
|
||||||
|
7. **Session Presets**: Save/load full state, URL sharing
|
||||||
|
8. **Persistence**: Auto-save, localStorage round-trip
|
||||||
|
9. **Debug probe**: `window.__nisps` when `?debug=1`
|
||||||
|
|
||||||
|
### Phase 6: ShapeSeq
|
||||||
|
|
||||||
|
**Goal**: Sequencer as optional subsystem.
|
||||||
|
|
||||||
|
1. Transplant sequencer core (already modular)
|
||||||
|
2. Build `ShapeSeqPanel`, `StepVisualizer`, `ChainBuilder`
|
||||||
|
3. Wire to signal bus (`seq.*` topics)
|
||||||
|
4. Lazy-load when enabled via URL param
|
||||||
|
|
||||||
|
### Phase 7: Polish + Tests
|
||||||
|
|
||||||
|
1. Port Playwright e2e tests (update selectors for new DOM)
|
||||||
|
2. Add component-level tests (vitest + solid-testing-library)
|
||||||
|
3. Responsive CSS pass
|
||||||
|
4. Performance profiling (ensure 60fps inference + rendering)
|
||||||
|
5. Accessibility pass on interactive elements
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Key Risks & Mitigations
|
||||||
|
|
||||||
|
| Risk | Mitigation |
|
||||||
|
|------|------------|
|
||||||
|
| WASM + SolidJS reactivity overhead | Typed arrays stay as signals (not store properties). Batch per frame. Profile early in Phase 1. |
|
||||||
|
| 126 params × 20fps = 2520 synth param sends/sec | Keep existing throttle (50ms interval, 0.002 dead zone) in synth subscriber, not in routing. |
|
||||||
|
| Store proxy overhead on large arrays | Use `createSignal` for `Float32Array` outputs, not `createStore`. Stores only for structured config. |
|
||||||
|
| Signal bus wildcard `match()` perf | Memoize prefix scans. Most topics are fixed at startup. Profile if >50 topics. |
|
||||||
|
| Canvas components fighting rAF | Each canvas owns its loop. No shared orchestrator unless profiling shows frame contention. |
|
||||||
|
| MLP resize destroys training data | Same as current: warn user, warm-start weights for joystick IML. Store handles the state transition. |
|
||||||
|
| `equals: false` on bus signals | Required for event semantics but means every emit triggers all subscribers. Keep topic count bounded. |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What Gets Transplanted vs Rewritten
|
||||||
|
|
||||||
|
### Transplanted (minimal changes, mostly just TS types)
|
||||||
|
- `nisps-wasm.js` + worker + WASM binary
|
||||||
|
- `dataset.ts` (FIFO ring buffer)
|
||||||
|
- `FlowFieldVisualizer` class (canvas rendering)
|
||||||
|
- `c15-bridge.ts`, `c15-adapter.ts`
|
||||||
|
- `param-map.ts`, `presets.ts` (pure data)
|
||||||
|
- `additive-engine.ts`, `fm-engine.ts`, `faust-engine-base.ts`
|
||||||
|
- `arpeggiator.ts` + worker
|
||||||
|
- `eoc-chain.ts`, `eoc-module.ts`, all effect modules
|
||||||
|
- ShapeSeq core (`sequencer.ts`, `chain.ts`, `clock.ts`, `pattern.ts`, `primitives.ts`)
|
||||||
|
- `audio-canvas.ts`
|
||||||
|
- `input-pipeline.ts` logic (becomes `useInputPipeline` hook wrapping same math)
|
||||||
|
- `output-pipeline.ts` logic (becomes `useOutputPipeline` hook)
|
||||||
|
- `control-surface.ts` compound axis logic (data tables + interpolation)
|
||||||
|
|
||||||
|
### Rewritten from scratch
|
||||||
|
- All DOM manipulation (→ JSX components)
|
||||||
|
- Event wiring (→ signal bus + reactive effects)
|
||||||
|
- State management (scattered module-scope vars → stores)
|
||||||
|
- Override application (→ `apply-overrides.ts` pure function)
|
||||||
|
- Persistence (→ `SessionProvider` with `createEffect` auto-save)
|
||||||
|
- Layout/CSS (→ new CSS with headless primitives)
|
||||||
|
- Init/boot sequence (→ provider tree + `onMount` hooks)
|
||||||
|
|
||||||
|
### Deleted (not ported)
|
||||||
|
- `b-app.js`, `c-app.js`, `app.js` (consolidated into one app)
|
||||||
|
- `b-workbench.html`, `c-journey.html`, `index.html` (single entry point)
|
||||||
|
- `iml.js`, `mlp.js`, `layer.js`, `node.js` (legacy JS ML engine, WASM only)
|
||||||
|
- `event-bus.js` (replaced by signal bus)
|
||||||
|
- All `wire*()` functions (replaced by component-local event handlers)
|
||||||
|
- DOM-string-building functions (`buildHeatmap`, `buildDrawerHTML`, etc.)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Open Questions
|
||||||
|
|
||||||
|
1. **TypeScript strictness**: Full strict mode from day one, or gradual? (Transplanted JS modules will need type annotations.)
|
||||||
|
2. **CSS approach**: Plain CSS files per component? CSS modules? Vanilla Extract? (Plain CSS is simplest and matches the current approach.)
|
||||||
|
3. **Testing during migration**: Run old Playwright tests against old code in parallel, or wait for Phase 7?
|
||||||
|
4. **WASM loading**: Vite handles `.wasm` imports natively, but the Emscripten glue (`nisps.js`) may need special config. Spike this in Phase 1.
|
||||||
|
5. **Faust DSP loading**: Currently fetched at runtime. Keep as-is or bundle?
|
||||||
Loading…
Reference in a new issue