diff --git a/CLAUDE.md b/CLAUDE.md index 2bcfc3b..b0b3b59 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -28,12 +28,32 @@ See `nisps-core/README.md` for complete documentation and examples. The `playground/` directory contains a browser-based interactive demo of the NISPS ML engine. It's a faithful JavaScript port of nisps-core's MLP + IML, with no build step or dependencies. -- **2 inputs** (virtual joystick X/Y) mapped through a `[3, 10, 10, 14, 20]` MLP to **20 outputs** controlling a Canvas2D flow-field particle system +- **2 inputs** (virtual joystick X/Y) mapped through a `[3, 32, 48, 64, 126]` MLP to **126 outputs** +- **Two output modes**: + - **Visual**: first 20 outputs control a Canvas2D flow-field particle system + - **Synth (C15)**: all 126 outputs control the C15 WASM synthesizer — every sonically meaningful continuous parameter across envelopes, oscillators, shapers, filters, feedback/output mixers, cabinet, and effects - **Two learning modes**: Examples (set slider targets, add examples, train) and RL Feedback (thumbs up/down with exploration noise) - **Serve statically**: `cd playground && python3 -m http.server` - **Mobile-first**: designed for touch/foldable phone use -Key files: `js/nisps/` (ML core port), `js/ui/` (visualizer, joystick, controls), `js/app.js` (wiring). +Key files: `js/nisps/` (ML core port), `js/ui/` (visualizer, joystick, controls), `js/synth/` (C15 bridge, param map, arpeggiator), `js/app.js` (wiring). + +### C15 Parameter Map + +The 126 synth parameters in `js/synth/param-map.js` were curated from the C15's 287 total parameters. Excluded categories: + +| Excluded | Count | Reason | +|----------|-------|--------| +| Hardware Amount/Source | 56 | No physical MIDI hardware in browser | +| Macro Controls/Times | 12 | Meta-routing layer conflicts with direct ML control | +| Scale offsets | 13 | Microtuning would break pitch unpredictably | +| Key tracking (`*_KT`) | 11 | Pitch-dependent scaling needs calibrated defaults | +| Velocity (`*_Vel`) | 11 | Velocity-dependent, ML can't observe key velocity | +| Envelope mod depths (`*_Env_A/B/C`) | 19 | Multiplicative interaction with envelope shapes makes space too hard to learn | +| Discrete/structural | 15 | Osc Pitch (full sweep), Master Vol/Tune, Voice Mute/Fade, Unison Voices, Mono modes, Split, Osc Reset | +| Secondary config | 7 | Att Curve, Elevate, Chirp, Decay Gate, Retrigger | +| PM shaper blend | 4 | Secondary routing params | +| FB Mix source selects | 4 | Discrete A/B selectors | ## Build System diff --git a/playground/README.md b/playground/README.md index 8ca2c04..993ef09 100644 --- a/playground/README.md +++ b/playground/README.md @@ -13,16 +13,30 @@ Run the command from the `playground/` directory. ## What it does -- Maps 2D joystick input (X/Y) through an MLP to 20 visualization parameters. -- Renders a flow-field particle visualization on a Canvas2D surface. +- Maps 2D joystick input (X/Y) through an MLP (`[3, 32, 48, 64, 126]`) to 126 outputs. +- **Visual mode**: first 20 outputs drive a flow-field particle visualization on Canvas2D. +- **Synth mode (C15)**: all 126 outputs control the C15 WASM synthesizer — envelopes, oscillators, shapers, filters, feedback/output mixers, cabinet, and effects. - Supports two learning modes: - **Examples**: add explicit input/output pairs and train. - **RL Feedback**: give thumbs up/down while exploring outputs. +## C15 synth parameters + +The 126 synth parameters (`js/synth/param-map.js`) cover all sonically meaningful continuous parameters of the C15 engine. Excluded from the C15's 287 total params: + +- Hardware routing (56) — no physical MIDI in browser +- Macro controls (12) — conflicts with direct ML control +- Scale/tuning (13) — would break pitch +- Key tracking / velocity (22) — depend on note context ML can't observe +- Envelope mod depths (19) — too many multiplicative interactions for 2-input ML +- Discrete/structural/dangerous (19) — pitch sweep, volume, switches, resets +- Secondary config (15) — curves, chirp, shaper blend, source selects + ## UI controls +- **Output mode tabs** (side panel): switch between Visual and Synth modes. - **Expand / Collapse** button on the canvas: makes the visual surface nearly full-screen and compresses lower controls into a minimal view. -- **Presets**: quick demo mappings (`Calm/Chaos`, `Rainbow`, `Vortex`). +- **Presets**: quick demo mappings (`Calm/Chaos`, `Rainbow`, `Vortex`) — visual mode only. - **Help** (`?`) overlay: in-app usage guide. - **Follow mode**: double-click joystick in RL mode to toggle no-hold interaction. - **Keyboard in Follow mode (RL)**: `2` = thumbs up, `1` = thumbs down. @@ -35,3 +49,5 @@ Run the command from the `playground/` directory. - `js/app.js` - app wiring and interaction logic. - `js/ui/` - visualizer, joystick, controls, parameter display. - `js/nisps/` - JavaScript MLP + IML core. +- `js/synth/` - C15 WASM bridge, parameter map, arpeggiator. +- `c15/` - C15 engine WASM binary and parameter definitions.