Add mission infrastructure for SolidJS migration

- Worker skill: solidjs-worker (SKILL.md)
- Services manifest (services.yaml)
- Init script (init.sh)
- Library: architecture.md, environment.md, user-testing.md, solidjs-patterns.md

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
This commit is contained in:
monkey-w1n5t0n 2026-04-07 08:49:52 +02:00
parent 445bc48c24
commit 13112cf447
8 changed files with 519 additions and 0 deletions

18
.factory/init.sh Normal file
View file

@ -0,0 +1,18 @@
#!/usr/bin/env bash
# Idempotent environment setup for SolidJS migration
set -e
REPO_ROOT="/home/w1n5t0n/src/MEMLNaut-NISPS-SOLIDJS"
cd "$REPO_ROOT"
# Install root npm dependencies if needed
if [ ! -d "node_modules" ]; then
npm install
fi
# Ensure Playwright browsers are available
if ! npx playwright install --dry-run chromium 2>/dev/null; then
npx playwright install chromium 2>/dev/null || true
fi
echo "Environment ready. Vite dev server: npx vite --port 5174"

View file

@ -0,0 +1,123 @@
# Architecture
How the MEMLNaut NISPS SolidJS app works.
## High-Level Architecture
```
index.html
└── App.tsx (root component, provider tree)
├── BusProvider (signal bus context)
├── MLProvider (ML store context)
├── InputProvider (input store context)
├── OutputProvider (output store context)
├── SynthProvider (synth store context)
├── EOCProvider (EOC store context)
└── SessionProvider (persistence)
└── ImmersiveLayout.tsx
├── Background canvases (FlowField or SynthVisualizer)
├── HeatmapStrip
├── Joystick + JoyMap
├── RLButtons
├── Dock + DrawerStack
└── Floating controls
```
## Data Flow
```
User Input (drag/gamepad/hands)
→ Input Store (joyX, joyY)
→ Input Pipeline (deadzone, zoom, curve, smoothing)
→ ML Store (setInputs on IML)
→ IML.process() (WASM inference)
→ Outputs Signal (Float32Array, single signal write)
→ Output Routing (apply overrides, route by mode)
├── FlowField (visual mode, own rAF loop reads signal)
├── SynthEngine (synth mode, throttled 50ms)
├── MIDIOutput (midi-cc mode)
└── AudioCanvas (audio-canvas mode)
```
## Key Components
### Signal Bus (`bus/signal-bus.ts`)
Unified event system replacing EventBus + CustomEvents. Topics with `equals: false` for event semantics. Wildcard matching via `match()`.
### ML Store (`stores/ml-store.ts`)
- IML instances (WasmIML) live OUTSIDE the store (opaque handles)
- `outputs` is a `createSignal<Float32Array>` (NOT a store property — proxies break typed arrays)
- Store tracks: loss, lossHistory, exampleCount, isTraining, spreadLevel, noiseLevel, undoStack
### Input Store (`stores/input-store.ts`)
- joyX, joyY as store properties
- Input pipeline config (deadzone, zoom, curve, etc.)
### Output Store (`stores/output-store.ts`)
- Output mode (visual/synth/midi-cc/audio-canvas)
- Per-mode overrides arrays
- Output pipeline config (globalCurve, smoothing, slewRate, freezeGate)
### Dual IML System
- `imlJoy`: 2 inputs (joystick), dynamic outputs
- `imlHand`: 14 inputs (hand tracking), dynamic outputs
- `activeIml`: points to whichever is active
- EOC IML: separate instance for Linked/Independent modes
### WASM Integration
- WASM files in `public/` (no Vite hashing)
- Emscripten glue loaded via dynamic import
- Training worker: lazy-loaded, own WASM instance
- Heap management: persistent buffers for inference
### Canvas Components
Each canvas owns its rAF render loop. The component reads the outputs signal each frame. No shared orchestrator.
### Debug Probe
`window.__nisps` exposed when `?debug=1`. All methods synchronous, bypassing SolidJS batching. `window.__nispsEoc` always exposed.
## Transplanted Modules (from playground/js/)
These are copied with minimal TypeScript adaptation:
- `core/wasm/` — WASM binaries + Emscripten glue (unchanged)
- `core/iml.ts` — WasmIML wrapper (from nisps/nisps-wasm.js)
- `core/dataset.ts` — FIFO ring buffer (from nisps/dataset.js)
- `core/synth/` — All synth engines (c15-bridge, c15-adapter, param-map, presets, additive, fm, faust)
- `core/audio/` — Arpeggiator, AudioCanvas, MIDI I/O
- `core/eoc/` — EOC chain + effect modules
- `core/shapeseq/` — ShapeSeq sequencer
- `core/ui/` — Pure math/logic modules (input-pipeline, output-pipeline, control-surface, visualizer)
## Rewritten Modules
Everything else is rewritten in SolidJS:
- All DOM manipulation → JSX components
- Event wiring → signal bus + reactive effects
- State management → SolidJS stores/signals
- Layout/CSS → new CSS with same design system
- Init/boot → provider tree + onMount hooks
## Key Constants
- N_JOY_INPUTS = 2, N_HAND_INPUTS = 14
- N_VISUAL_OUTPUTS = 20, N_SYNTH_OUTPUTS = 126 (C15)
- STORAGE_KEY = 'nisps-a-immersive'
- MAX_UNDO = 20, PARAM_DEAD_ZONE = 0.002, PARAM_SEND_INTERVAL = 50
- 18 C15 synth sections (SYNTH_SECTIONS)
- 20 visual param names/colors (VISUAL_PARAM_NAMES, VISUAL_PARAM_COLORS)
## COOP/COEP Requirement
C15 synth requires SharedArrayBuffer. Vite dev server must send:
```
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp
```
## MLP Resize Cascade
When output count changes (mode switch, engine switch, EOC mode change):
1. Warn user if training data exists → confirmation dialog
2. Destroy old MLP, create new with different output count
3. Warm-start: copy hidden layer weights, reset output layer
4. Update output signal, rebuild heatmap

View file

@ -0,0 +1,45 @@
# Environment
Environment variables, external dependencies, and setup notes.
**What belongs here:** Required env vars, external dependencies, platform-specific notes.
**What does NOT belong here:** Service ports/commands (use `.factory/services.yaml`).
---
## External Dependencies
- **Node.js**: v25.4.0
- **npm**: v11.7.0
- **Playwright**: v1.59.1 (chromium headless shell v1217 installed)
- **Emscripten**: NOT required — existing WASM binaries are transplanted as-is
## WASM Binaries (transplanted from playground/)
- `public/nisps.wasm` (36KB) — ML engine
- `public/nisps.js` (14KB) — Emscripten glue
- `public/nisps-wasm-worker.js` — Training worker (from playground/js/nisps/)
- `public/c15/c15_engine.wasm` (270KB) — C15 synthesizer
- `public/c15/parameters.json` (163KB) — C15 parameter definitions
- `public/c15/worklet-processor.js` — C15 AudioWorklet
- `public/faust/` — Faust DSP files (fetched at runtime)
## CDN Dependencies
- **MediaPipe** (cdn.jsdelivr.net): Hand tracking only, lazy-loaded on mode switch to Hands
- **Google Fonts**: JetBrains Mono (loaded in index.html)
## Browser Requirements
- SharedArrayBuffer support (requires COOP/COEP headers)
- WebMIDI support (MIDI CC mode)
- AudioWorklet support (C15 synth)
- getUserMedia (hand tracking camera)
- Gamepad API (gamepad input)
## Platform Notes
- Linux x86_64 (kernel 6.14.0)
- No Docker
- 62GB RAM, 12 CPU cores
- ffmpeg 7.1.1 available (not used by SolidJS app)

View file

@ -0,0 +1,124 @@
# SolidJS Patterns
Patterns and conventions for the SolidJS migration.
**What belongs here:** SolidJS-specific patterns, reactivity gotchas, store design patterns.
**What does NOT belong here:** General architecture (use architecture.md), environment details (use environment.md).
---
## Store vs Signal Decision
| Data Type | Use | Why |
|-----------|-----|-----|
| Float32Array outputs | `createSignal` | Store proxies break typed arrays |
| Structured config | `createStore` | Fine-grained reactivity on nested properties |
| Opaque handles (IML, engines) | Module-scope variables | Not reactive — referenced imperatively |
| Event streams | Signal bus topic | `equals: false` for event semantics |
## Reactive Inference Pattern
```typescript
// Inference runs when inputs change, NOT on rAF
createEffect(() => {
const x = inputState.joyX;
const y = inputState.joyY;
const iml = getActiveIml();
if (!iml) return;
iml.setInput(0, x);
iml.setInput(1, y);
iml.process();
setOutputs(new Float32Array(iml.getOutputs()));
});
```
## Canvas Component Pattern
```typescript
const MyCanvas = () => {
let canvasRef: HTMLCanvasElement;
const data = bus.topic<Float32Array>('output.visual');
onMount(() => {
const ctx = canvasRef.getContext('2d')!;
let raf: number;
const loop = () => {
const d = data();
if (d) renderFrame(ctx, d);
raf = requestAnimationFrame(loop);
};
raf = requestAnimationFrame(loop);
onCleanup(() => cancelAnimationFrame(raf));
});
return <canvas ref={canvasRef!} />;
};
```
## Provider Pattern
```typescript
const MLContext = createContext<MLStore>();
export const MLProvider = (props: { children: any }) => {
const [outputs, setOutputs] = createSignal(new Float32Array(126));
const [mlState, setMlState] = createStore({ /* ... */ });
const store = { outputs, setOutputs, mlState, setMlState };
return <MLContext.Provider value={store}>{props.children}</MLContext.Provider>;
};
export const useML = () => useContext(MLContext)!;
```
## Debug Probe (Synchronous)
```typescript
// Must bypass SolidJS batching for sync probe
if (urlParams.debug) {
window.__nisps = {
setInputs: (x, y) => {
batch(() => {
setInputState({ joyX: x, joyY: y });
});
// Imperative: run inference directly
iml.setInput(0, x);
iml.setInput(1, y);
iml.process();
setOutputs(new Float32Array(iml.getOutputs()));
},
getOutputs: () => Array.from(outputs()),
// ... etc
};
}
```
## CSS Pattern
```css
/* Per-component CSS files, no framework */
.component-name {
/* Use CSS custom properties from root */
background: var(--glass-bg);
backdrop-filter: blur(var(--glass-blur));
border: 1px solid var(--glass-border);
}
```
## Worker Cleanup
```typescript
// Always clean up workers
onCleanup(() => {
trainingWorker?.terminate();
arpeggiatorWorker?.terminate();
imlJoy?.destroy();
imlHand?.destroy();
});
```
## Key Gotchas
1. **Store proxies and typed arrays**: NEVER put Float32Array in a store. Use createSignal.
2. **equals: false**: Required for bus topics. Every emit triggers subscribers even with same data.
3. **CSS animations**: Use classList toggling on stable DOM nodes. Don't use `<Show>` for animated elements.
4. **EOC resize cascade**: Cannot be a reactive effect (needs confirmation dialog). Use pending action pattern.
5. **Probe synchrony**: All probe methods must be synchronous. Use batch() + imperative IML calls.
6. **Idle inference waste**: Don't run inference on rAF. Use reactive createEffect on inputs.

View file

@ -0,0 +1,74 @@
# User Testing
Testing surface, required tools, and resource cost classification.
---
## Validation Surface
**Surface**: Browser (web application)
**URL**: http://localhost:5174 (Vite dev server)
**Comparison URL**: http://localhost:7331/a-immersive.html (old playground app)
**Primary testing tool**: agent-browser
**Secondary tool**: Playwright e2e tests via `window.__nisps` debug probe
### Debug Probe Interface
When loaded with `?debug=1`, `window.__nisps` exposes:
| Method | Returns | Sync/Async |
|--------|---------|------------|
| getOutputs() | Float32Array(126) | Sync |
| getLoss() | number \| null | Sync |
| getWeights() | Float32Array | Sync |
| getExampleCount() | number | Sync |
| setInputs(x, y) | void | Sync |
| thumbsUp() | void | Sync |
| thumbsDown() | void | Sync |
| train() | number (loss) | Sync |
| trainAsync() | Promise<number> | Async |
| randomise() | void | Sync |
| clearExamples() | void | Sync |
| saveState() | void | Sync |
| evalLoss() | number \| null | Sync |
| inferBatch(points) | Float32Array[] | Sync |
| getLayerStats() | object[] | Sync |
`window.__nispsEoc` is ALWAYS available (no ?debug=1 required):
- `trainingTarget` getter/setter ('synth' | 'eoc')
- `imlEoc` getter (WasmIML | null)
### DOM Selectors for Validation
Key CSS selectors for UI state assertions:
- `.dock` — right-side dock container
- `.drawer` — drawer panels (`.hidden` when closed)
- `.heatmap-cell` — heatmap parameter bars
- `.heatmap-cell-bar` — bar width element
- `#status-text` — status line text
- `.rl-btn` — RL thumbs up/down buttons
- `.pill-opt[data-mode]` — output mode pills
- `.joystick-container` — joystick wrapper
- `#joy-map` — joystick canvas
- `.noise-ring` — noise indicator
### Skip List
These assertions require physical hardware and cannot be validated in headless browser:
- VAL-JOY-006 (Gamepad) — requires physical gamepad
- VAL-MIDI-001 (MIDI output) — requires physical MIDI device
- VAL-HAND-002 (Camera) — requires camera permission + gesture
## Validation Concurrency
**Machine specs**: 62GB RAM, 12 CPUs, ~23GB available
**agent-browser instances**:
- Each instance: ~300MB RAM (lightweight app)
- Dev server: ~200MB RAM
- Per-validator total: ~500MB
- Usable headroom (70% of 23GB): ~16GB
- Max concurrent: 5 validators (5 × 500MB = 2.5GB, well within budget)
**Playwright tests**: Run serially (single browser, single server)

23
.factory/services.yaml Normal file
View file

@ -0,0 +1,23 @@
commands:
install: npm install
install-solid: npm install && cd /home/w1n5t0n/src/MEMLNaut-NISPS-SOLIDJS && npm install
typecheck: npx tsc --noEmit
build: npx vite build
test-old: npx playwright test --reporter=line
lint: npx eslint src/ --ext .ts,.tsx 2>/dev/null || true
dev: npx vite --port 5174
services:
vite-dev:
start: npx vite --port 5174 --host
stop: lsof -ti :5174 | xargs kill 2>/dev/null || true
healthcheck: curl -sf http://localhost:5174 > /dev/null 2>&1
port: 5174
depends_on: []
old-playground:
start: cd playground && python3 -m http.server 7331
stop: lsof -ti :7331 | xargs kill 2>/dev/null || true
healthcheck: curl -sf http://localhost:7331/a-immersive.html > /dev/null 2>&1
port: 7331
depends_on: []

5
.factory/settings.json Normal file
View file

@ -0,0 +1,5 @@
{
"enabledPlugins": {
"core@factory-plugins": true
}
}

View file

@ -0,0 +1,107 @@
---
name: solidjs-worker
description: General-purpose SolidJS feature implementation worker for the NISPS migration
---
# SolidJS Worker
NOTE: Startup and cleanup are handled by `worker-base`. This skill defines the WORK PROCEDURE.
## When to Use This Skill
All feature implementation for the SolidJS migration. This worker builds components, stores, hooks, and transplants core modules from the old playground app.
## Required Skills
- **agent-browser**: For manual verification of UI behavior in the browser. Use after implementation to verify component rendering and interactions.
- **tuistory**: NOT required for this mission.
## Work Procedure
### Step 1: Understand the Feature
1. Read `mission.md` in the mission directory for overall mission context.
2. Read `.factory/library/architecture.md` for system architecture.
3. Read `.factory/library/environment.md` for environment details.
4. Read the assigned feature from `features.json` carefully — understand every `expectedBehavior` and `fulfills` assertion.
5. Read the relevant assertion descriptions from `validation-contract.md`.
### Step 2: Reference the Old Code
1. The old playground app is at `playground/` (READ-ONLY reference).
2. `playground/js/a-app.js` is the main app (~4300 lines, read in chunks).
3. Identify the exact behavior to replicate by reading the relevant old code sections.
4. For transplanted modules (listed in `architecture.md`), copy from the old code and adapt to TypeScript with minimal changes.
### Step 3: Write Tests First (TDD)
1. Write failing tests BEFORE implementation. Tests go in `tests/` at the repo root.
2. For SolidJS components: use vitest + solid-testing-library (install if needed).
3. For behavioral assertions matching the validation contract: write Playwright e2e tests that use the `window.__nisps` debug probe.
4. For transplanted core modules: write unit tests verifying the same behavior as the old JS code.
5. Run tests to confirm they fail (red).
### Step 4: Implement
1. Follow the architecture in `.factory/library/architecture.md`.
2. Use SolidJS patterns from `.factory/library/solidjs-patterns.md`.
3. For transplanted modules: copy from `playground/js/`, add TypeScript types, adapt imports.
4. For new SolidJS code: use stores (createStore) for structured config, signals (createSignal) for Float32Arrays.
5. Ensure all workers are cleaned up in `onCleanup()`.
6. Ensure CSS uses the glass morphism design system (custom properties from `a-immersive.css`).
### Step 5: Verify
1. Run all tests: `npm test` (or the appropriate test command from services.yaml).
2. Run typecheck: `npx tsc --noEmit`.
3. Start the Vite dev server: `npx vite --port 5174`.
4. Use **agent-browser** to visually verify the feature:
- Navigate to `http://localhost:5174?debug=1`
- Verify the UI renders correctly
- Interact with the component and verify behavior
- Take screenshots as evidence
5. Compare behavior against the old app at `http://localhost:7331/a-immersive.html?debug=1` if needed.
6. Stop the dev server when done.
### Step 6: Commit and Hand Off
1. Stage and commit all changes with a descriptive message.
2. Report all verification evidence in the handoff.
## Example Handoff
```json
{
"salientSummary": "Implemented Joystick component with pointer events and follow mode. Created input-store with joyX/joyY signals. Verified dragging updates position [0,1] and follow mode toggles on double-tap. All 5 tests pass, typecheck clean.",
"whatWasImplemented": "src/components/input/Joystick.tsx (pointer events, position tracking, follow mode toggle), src/stores/input-store.ts (joyX, joyY, isDragging, followMode), src/components/input/JoyMap.tsx (canvas minimap scaffold), tests/e2e/joystick.spec.js (3 tests: drag, follow, clamp)",
"whatWasLeftUndone": "JoyMap trail rendering deferred to layout milestone",
"verification": {
"commandsRun": [
{ "command": "npx tsc --noEmit", "exitCode": 0, "observation": "No type errors" },
{ "command": "npx vitest run tests/e2e/joystick.spec.js", "exitCode": 0, "observation": "3 tests passing" }
],
"interactiveChecks": [
{ "action": "Navigate to localhost:5174?debug=1, drag joystick", "observed": "Joystick dot follows pointer, probe.setInputs matches visual position" },
{ "action": "Double-tap joystick", "observed": "Follow badge appears, pulse animation starts" }
]
},
"tests": {
"added": [
{ "file": "tests/e2e/joystick.spec.js", "cases": [
{ "name": "drag updates position", "verifies": "VAL-JOY-001" },
{ "name": "double-tap toggles follow", "verifies": "VAL-JOY-002" },
{ "name": "input clamping", "verifies": "VAL-JOY-005" }
]}
]
},
"discoveredIssues": []
}
```
## When to Return to Orchestrator
- Feature depends on infrastructure not yet built (e.g., a store that doesn't exist)
- Requirements are ambiguous — cannot determine correct behavior from old code or contract
- Existing bugs in the codebase block this feature
- Need to modify files in `playground/` (off-limits without orchestrator approval)
- Cannot complete within the session — return with `whatWasLeftUndone` detailed