memlnaut-nisps/docs/specs/dock-spec.md
monkey-w1n5t0n 9ad1f78ddd docs: the specs disposition pass (plan §8)
Roughly 20k lines were deleted from this repo in the last week and much of the
corpus still described the pre-deletion world in the present tense. Executes
the §8 table: archive the retired, reclassify the executed, prune the stale.

  aimmersive-clone-spec      -> _archive/ with a deprecated-by note
  feedback-modes-port-spec   -> plans/, kind: plan, status: executed
  manifold-parity-features   -> plans/, kind: plan, status: active
  playground-2.0-rewrite     -> status: superseded
  engine-architecture        434 -> ~120 lines; seam + spine kept, rewritten
                             present-tense against the shipped engine/
  MAIN.md                    six contradicted claims fixed; registry resynced
  vcv-module.md              pruned to the current 8->16 contract and made the
                             single .nisps format spec
  vcv/NISPS-FORMAT.md        DELETED — documented a v1 format that no longer loads
  vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table
  inputs/backends/dock trio  grounding sections marked historical, dead cites fixed

Two rows of the §8 table were themselves wrong, corrected here: the deleted
full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no
§6.3), and codegen/README.md was already a MAP pointer with no port-solidjs
trigger left to remove.

Beyond the table — found by sweeping every backticked path in the changed docs
against `git ls-files`, which is how these should have been caught before:

  manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current:
  SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a
  shot.spec.ts that does not exist. The whole stage table was keyed on a `focus`
  axis that no longer exists — selection is now sandwich > particles >
  composite. This matters more than the rest: CLAUDE.md tells every agent to
  read ONBOARDING.md first for Manifold work, so it was actively teaching a
  fiction. Rewritten against ConsoleApp.tsx.

  MAP.md claimed the input layer reduces axes to the engine arity with an
  "even/odd blend". input-layer.ts says the opposite in its own header: one
  dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and
  mean-blending was removed deliberately because it diluted every source.

  AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed),
  per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at
  nisps::FixedBuffer (deleted).

Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/
Delta and cachedNovelty behind a live context-menu toggle that nothing reads;
vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's
origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to
9000, so out of the box they do not meet.

Firmware-build docs are deliberately untouched — the PlatformIO migration
lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00

389 lines
32 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

---
kind: spec
stability: evolving
layer: behavioural
---
# Dock Spec — Console Right-Dock Drawers + Per-Output Controls
*Workstream D. Designed 2026-06-27; built 2026-06-28 with operator-directed restructuring.
British spelling in product copy. The built-in synth is the **"Powerful Synth Engine"** — the
string "C15" MUST NEVER appear in the UI.*
> **Grounding note (2026-07-21).** Treat every `file:line` cite in the body as historical
> grounding: `playground/*` died with the retired SolidJS playground, `aimmersive-clone-spec.md`
> is archived at `_archive/aimmersive-clone-spec.md`, and `engine-architecture.md` was trimmed
> 2026-07 (its cited § numbers are gone). What shipped diverges from the prescription in shape
> but not in substance:
>
> - **Drawer roster**: the operator restructured the dock during the build — a top **Mode
> selector** plus five drawers (`learn`, `inputs`, `route`, `settings`, `help` —
> `manifold/src/console/Dock.tsx`/`Drawers.tsx`). The §0 six-icon roster (SYNTH/VISUAL as
> drawers) became per-Mode config inside the Outputs drawer instead.
> - **Depths**: two shipped, not three — `condensed` (360px side panel) and `expanded` (centred
> modal). The separate FULL "advanced backend modal" (§4, `<BackendAdvanced/>`) was built,
> then deleted as a duplicate editor (2026-07 sweep, S18); the sole per-backend editor is
> `manifold/src/dock/OutputsBackendConfig.tsx` in the Outputs drawer.
> - **The per-output control row** (§3, the heart of this spec) shipped essentially as designed:
> `manifold/src/dock/OutputControlRow.tsx` + `output-state.ts` (off/fixed/live + mute +
> solo/arm + min/max/curve).
> - The feedback surface shipped richer than §1 (Explore-and-Place with reroll/nudge/place, plus
> geometric dislike) — see `docs/adr/rl-feedback-design.md` and `nisps/ml/feedback.hpp`.
---
## 0. The 3-depth dock model (frame the design)
The Console's right dock is a 48px rail (`--dock-width`, `aimmersive-clone-spec.md:181`) of icons; each icon owns one drawer with **three depths**:
| Depth | Width / placement | Purpose | Source vocabulary |
|---|---|---|---|
| **peek** | narrow flyout (~72px) hugging the rail | glanceable status + 12 primary toggles; no scrolling | `playground-2026.md` dock+drawer |
| **expand** | glass panel ~300px (cf. a-immersive drawer-stack `width:260px`, `aimmersive-clone-spec.md:312`) | the working surface: the per-output rows, the live sliders, the mode toggles | a-immersive drawers |
| **FULL** | near-fullscreen **modal** (`--z-modal`, glass over dimmed canvas) | the "advanced config page" for one entry — backend CC tables, full param matrix, weight-health lab | mission "advanced" page |
- **Depth is per-drawer state** in `uiStore` (`peek | expand | full`), independent across drawers (a-immersive allows multiple open at once, `aimmersive-clone-spec.md:84`). FULL is mutually exclusive — only one modal at a time (it captures the screen).
- `\` toggles the *focused* drawer peek↔expand; a drawer's header **⤢ button** (or double-click a row) opens FULL.
- Esc closes FULL→expand; closing FULL restores prior depth.
- Drawers slide in via `drawerSlideIn` 0.2s translateX (`aimmersive-clone-spec.md:189`). FULL fades via `helpFadeIn`.
- Dock icons get `.active` tint (`--accent #ff6a00`) when their drawer is open; macOS magnify on hover (`scale(1.35)`, `aimmersive-clone-spec.md:188`).
**Drawer roster** (six icons, top→bottom on the rail):
| Icon | Drawer | One-line role |
|---|---|---|
| 🧠 LEARN | **Learning-Behaviour** | Feedback-mode selector, SOLO/arm chooser, live training params |
| 🎚 IN | **Inputs** | Input source + per-axis pipeline (workstream F territory — referenced, §6) |
| 🔀 OUT | **Outputs / Routing** | The per-output control matrix + backend selector (workstream E + §3, §4) |
| 🔊 SYNTH | **Powerful Synth Engine** | Engine switch + audio/arp controls + group overrides |
| ✦ VISUAL | **Particle / Visual System** | Flow-field visualiser params + presets |
| ? HELP | **Help** | Onboarding overlay (opens modal directly, not a drawer) |
---
## 1. LEARNING-BEHAVIOUR drawer
Owns *how the model learns from your gestures*: the feedback-mode selector, the SOLO/arm variant chooser, and the live training knobs. Binds to `EngineApi.feedback` (`engine-architecture.md` §2.2) and the feedback C API (`findings-engine-surface.md:52`, `nisps_ml_feedback_*`).
### 1.1 FEEDBACK_MODE selector ("Down Action")
The `+` (up) verdict is always "keep this" (`addExample` + train, `findings-feedback-behaviour.md:102`); the `` (down) verdict is **selectable** among the ported feedback modes (`plans/feedback-modes-port-spec.md` §1; `FeedbackController<MLP_T>`). The new 2-mode product surface (per the prompt's "Mode 1 / Mode 2"):
| Selector label (UI copy) | Engine `FeedbackMode` | Behaviour |
|---|---|---|
| **Push away** (Mode 1) | `Avoid` (`feedback.hpp` enum, `findings-feedback-behaviour.md:90`) | down → geometric-dislike: perturbs the mapping away from what you disliked. In the deployed core this routes to `move_weights(speed, spread, pinMask)` (`plans/feedback-modes-port-spec.md` §2.5 — the true k-NN centroid push is firmware-only, out of scope). |
| **Explore & place** (Mode 2) | `RandomiseMlp` (`findings-feedback-behaviour.md:122`) | down → snapshot + `draw_weights(spread)` re-rolls the whole net into a scratchpad you audition by moving the joystick; `+`/drag commits a `+1` example at the chosen input and **restores the real net** (`findings-feedback-behaviour.md:135-146`); down-again **cancels** (restore snapshot). |
- A third engine mode `RandomiseOutputs` (bypass MLP, hold a static random vector, `findings-feedback-behaviour.md:111`) exists in the core but is **not** surfaced as a product mode in v1 — expose it only behind `?debug=1` as "Static roll". **(Open choice 1.)**
- **Peek**: a 2-segment pill (`Push away` / `Explore & place`), the current mode highlighted in `--accent`. This pill is *also* mirrored next to the Verdict cluster (`VerdictCluster`, `findings-design-and-manifold.md:50`) so it is reachable during live play without opening the drawer — matching how a-immersive puts the `rl-label` above the RL buttons (`aimmersive-clone-spec.md:283`).
- **Expand**: the pill + a one-line plain-English description of the active mode + an **"exploring…" indicator** that lights when `engine.feedback.exploring()` is true (`findings-feedback-behaviour.md:237`), reusing the `<NoiseRing>` colour ramp (off/active/high, `aimmersive-clone-spec.md:98`). While exploring in Mode 2, training is paused (`learning_paused()`, `findings-feedback-behaviour.md:48`) — show a small "learning paused" badge.
- **FULL**: the feedback lab — a diagram of the active state machine (idle → exploring → commit/cancel), the raw `FeedbackAction` log of the last presses (`findings-feedback-behaviour.md:90` enum), plus the spread/tame "Health lab" sliders (§1.3) for radical exploration tuning.
- **Bind point**: `engine.feedback.setMode(mode)``nisps_ml_feedback_set_mode` (`plans/feedback-modes-port-spec.md` §4). Switching mode while exploring auto-aborts and restores the net (`set_mode` calls `abort_explore`, `findings-feedback-behaviour.md:96`). Persist `feedbackMode` in the session blob.
### 1.2 SOLO / arm variant chooser
"Solo" and "arm" are the SAME concept (the prompt: *solo(=arm)*): **focus training on one output**. It maps to the engine's **focus mask** (`activeDims_` → `nisps_ml_feedback_set_focus`, `findings-feedback-behaviour.md:159-167`) AND to the **output-pin mask** used by `move_weights`/`buildPinMask` (`findings-feedback-behaviour.md:117`, `mode-runtime.ts:548`).
Two armable scopes (the chooser):
| Variant | Effect | Engine wiring |
|---|---|---|
| **Arm output** (per-row solo) | only the armed output(s) learn/re-roll; all others are frozen in the model | `set_focus(mask)` where `mask[i]=1` for armed dims; unmuted-but-unarmed dims set `0` so `move_weights`/`roll_static_outputs` leave them put (`findings-feedback-behaviour.md:71,103-107`) |
| **Arm all** (default) | every live output learns (empty mask ⇒ all active, `findings-feedback-behaviour.md:161`) | `clear_focus_mask()` |
- Arming is a per-output toggle (the **S** button on each control row, §3) — arming any output flips the drawer into "arm-output" scope automatically; clicking the drawer's **"Arm all"** chip clears it.
- Multiple outputs may be armed at once (mask is a vector). The armed set is highlighted on the ReadoutStrip and on the heatmap (border glow, `--glow-focus`).
- **Peek**: shows "Arm: all" or "Arm: 3 outputs"; tapping cycles to "Arm all".
- **Expand**: the scope chips + a compact list of currently-armed output names with quick-unarm.
- **Persisted** as part of routing state (the focus mask is reconstructable from per-row `armed` flags).
### 1.3 Live training params
Six live knobs, lifted from a-immersive's "NISPS" drawer (`aimmersive-clone-spec.md:67` — the deployed app's *actual* tuning surface, six raw sliders) plus the spread/tame regime controls:
| Slider | Range | Default | Binds to |
|---|---|---|---|
| **Noise** | 0noiseCap | 0.05 | `mlStore.noiseLevel`; the `` perturbation magnitude (`findings-feedback-behaviour.md:110`) |
| **Learning rate** | 1e-6 1e-2 (log) | 1e-5 | `iml.learningRate` (`aimmersive-clone-spec.md:226`) |
| **Decay** | 0.81.0 | 0.97 | `rlExplorationDecay` (`findings-feedback-behaviour.md:103`) |
| **Spread** | 01 | 0.6 | `spreadLevel` — master noise/Xavier regime (`findings-feedback-behaviour.md:108`; CLAUDE.md `spread`) |
| **Tame** | 01 | 1 | output-range safety toward `[safeMin,safeMax]` (`param-map.js:261` `applyTame`) |
| **Max iters / Convergence** | — | — | `iml.maxIterations`/`convergenceThreshold` (`aimmersive-clone-spec.md:274`) — FULL-depth only |
- **Peek**: Noise + Spread (the two most live).
- **Expand**: Noise, Spread, Tame, Learning rate, Decay as `<Slider>`s.
- **FULL**: all six + Max iters/Convergence + a real `<LossPlot>` (needs `nisps_ml_loss_history`, `findings-design-and-manifold.md:62`) + `<WeightHealth>` edge-glow + `<LayerStats>` + `<GradientFlow>` (the diagnostics suite, `findings-design-and-manifold.md:63`). This is the "advanced learning" page.
- All writes are eager engine setters (no React-render coupling, `engine-architecture.md` §4); persisted in the session blob.
---
## 2. INPUTS drawer (workstream F territory — referenced)
**Owner: workstream F (modular N×M inputs + input pipeline).** This drawer is specified there; here we only fix its *dock shape* so the dock model is complete and the two specs dovetail.
- **Source**: input source selector — **Joystick / Hands / MIDI / Mic** (a-immersive Input pill, `aimmersive-clone-spec.md:150`). v1 browser is **fixed-2-input** (`findings-engine-surface.md:67`; `MLP<2,…>`) — modes needing >2 inputs show a "single-input in browser" badge (`engine-architecture.md` §6 open Q2).
- **Per-axis pipeline** (deadzone→zoom→curve→smoothing→momentum, `input/pipeline.ts`, `findings-design-and-manifold.md:84`): one `<ControlAxis>` row per input axis with its pipeline knobs.
- **Depth**: peek = source + zoom; expand = per-axis pipeline rows; FULL = the N×M input-routing matrix (workstream F).
- **Bind**: `engine.setInput(x,y)` is the only input door (`engine-architecture.md` §2.2); pipeline config lives in `input-store`.
- **Per-axis controls reuse the per-output row component** (§3) where sensible (an input axis has min/max/curve too), but mute/solo/freeze semantics differ — defer those to workstream F. **(Open choice 2: confirm input-axis rows share the §3 component or get their own.)**
---
## 3. OUTPUTS / ROUTING drawer + the per-output control row
The heart of this spec. **Owner of the routing matrix + per-output baseline is shared; backend-specific advanced layouts are workstream E (VCV/OSC/MIDI LED-ring backends, `findings-design-and-manifold.md:30`).**
### 3.1 The per-output baseline (EVERY output, EVERY backend)
Every output — synth param, MIDI CC, OSC path, VCV channel, visual param — shares one baseline control set. This faithfully ports the deployed vanilla-JS per-param override system (`aimmersive-clone-spec.md` §2.4 heatmap popup + §2.5 group drawer; `param-map.js` `applyGroupOverride`/`applyCurve`/`applyTame`). The baseline:
| Control | Glyph / widget | Semantics |
|---|---|---|
| **Mute** | **M** toggle | silenced *downstream* but still computed + visible (distinct from off) |
| **Solo / Arm** | **S** toggle | focus training on this output (§1.2; focus mask + pin mask) |
| **Tri-state** | **off / fixed / live** segmented | model-control state (table §3.3) |
| **Min / Max** | `<DualRangeSlider>` (min blue `#4488ff`, max orange `#ff6a00`, `aimmersive-clone-spec.md:130`) | output range remap; `applyGroupOverride(v,curve,min,max)` (`param-map.js:301`) |
| **Curve** | `<CurvePad>` 36×36 drag canvas, 0.5=linear (`aimmersive-clone-spec.md:128`; `applyCurve`, `param-map.js:287`) | per-output response curve |
| **Value** | inline read-out / drag bar | live model value, or the held value when fixed |
### 3.2 The control-row component + state model
```ts
// manifold/src/engine/routing/output-control.ts (engine-side, headless)
type OutputState = 'off' | 'fixed' | 'live'; // the tri-state
interface OutputControl {
index: number; // model output index (0..125 for synth)
name: string; // schema label (NEVER "C15"; synth params from SYNTH_PARAM_MAP)
group: string; // section (Env A, Osc B, …) for grouping
state: OutputState; // off | fixed | live
muted: boolean; // downstream silence; still computed
armed: boolean; // solo / focus-training (=arm)
min: number; // [0,1]
max: number; // [0,1], min<=max enforced
curve: number; // [0,1], 0.5 linear
fixedValue: number; // held value when state==='fixed' (captured on freeze)
// backend-specific, populated by the active backend adapter:
backend?: MidiCcSpec | OscSpec | VcvSpec;
}
```
- Stored in `routing-store` as a `createStore` array indexed by output (`engine-architecture.md` §1; `overrideStore` shape, `aimmersive-clone-spec.md:230`). This unifies a-immersive's split `visualOverrides`/`groupOverrides`/`engineParamOverrides`/`midiCCOverrides` (`aimmersive-clone-spec.md:230-233`) behind one `getSectionView` adapter (`aimmersive-clone-spec.md:140`).
- **Mute ↔ off are distinct fields** (the prompt insists): `muted` is a downstream gate; `state==='off'` excludes from model control. They compose (an output can be off AND muted).
- **The deployed app conflates frozen↔muted via one field** (`aimmersive-clone-spec.md:347` — heatmap popup `frozen` ↔ group drawer `muted` map to the same underlying field). The new model **separates them deliberately**: `state` carries off/fixed/live, `muted` is its own boolean, `armed` its own boolean. This is the one deliberate divergence from the deployed semantics — record in `ALIGNMENT.md`. **(Open choice 3.)**
- **Reactive binding**: the row's controls write the store eagerly; `routedOutput` memo (`engine-architecture.md` §2.1) reads `routing-store` to remap each output via `applyGroupOverride`. The single send-effect pushes to the active backend. A row never holds a second data path.
### 3.3 Tri-state semantics table (precise)
This matches the deployed per-param override system: `off`/`fixed` correspond to a-immersive's `frozen` (excluded/pinned, `aimmersive-clone-spec.md:131`), `live` to the default model-driven path.
| State | Computed by model? | Pin mask (`buildPinMask`) | Sent downstream? | Value emitted | UI |
|---|---|---|---|---|---|
| **off** | **No** — removed from model control; final-layer weights pinned so RL/train never touch it (`findings-feedback-behaviour.md:117`) | pinned (excluded) | yes, at last held value | `fixedValue` (held/excluded) | dimmed row, no bar motion |
| **fixed** (freeze) | computed but result discarded; held at a static value | pinned (protect the held dim) | yes | `fixedValue` (captured on freeze, draggable, `aimmersive-clone-spec.md:131`) | hatched bar overlay (`aimmersive-clone-spec.md:190`), value slider shown |
| **live** | **Yes** — model-driven | not pinned | yes | `applyGroupOverride(modelOut, curve, min, max)` | full bar, animates |
Orthogonal modifiers (compose with any state):
| Modifier | Computed? | Visible? | Sent downstream? | Training focus |
|---|---|---|---|---|
| **mute** | yes (still computed) | **yes** (bar visible) | **no** (silenced) | unaffected |
| **off** (state) | no | yes (dimmed) | yes (held) | excluded |
| **solo / arm** | yes | yes (focus glow) | yes | **this output only** (focus + pin mask) |
> Precise difference the prompt demands: **off** = the model no longer drives this output (excluded from learning, held/pinned). **fixed/freeze** = held at a static value (also pinned, but conceptually "I chose this value", with a draggable `fixedValue`). **mute** = silenced downstream but *still computed and visible* (distinct from off — you still see it move, you just don't hear it). **solo/arm** = focus training on this output.
### 3.4 Drawer depths
- **Peek**: backend badge (Synth / MIDI / OSC / VCV) + count of live/fixed/off/muted outputs + "Arm all" status.
- **Expand**: the **routing matrix** — a scrollable `<For>` of `<OutputControlRow>`s grouped by `group` (collapsible section headers carry a **group master curve** + **mute-group** like a-immersive's group drawer, `aimmersive-clone-spec.md:137`). Each row: name · M · S · [off|fixed|live] · dual-range · curve pad · value. 126 rows for synth — virtualise / collapse non-live by default.
- **FULL**: the **Advanced backend modal** (§4) — backend selector at the top, then the backend-specific editor over the full output set.
- **Backend selector** (workstream E): **Web Audio (synth) / Web MIDI / OSC bridge / VCV** (`engine-architecture.md` §1 backends; `aimmersive-clone-spec.md:143` output-mode tabs). Switching backend may change output count → reuse a-immersive's `confirm()` weight-reset guard (`aimmersive-clone-spec.md:144,344`).
---
## 4. ADVANCED modal — backend-specific layouts (FULL depth)
All backends share the §3.1 baseline (M/S/off-fixed-live/min/max/curve). The FULL modal adds the backend-specific fields. **(Workstream E owns the VCV/OSC/MIDI backend internals; this fixes the modal layout + state contract.)**
### 4.1 MIDI backend
Per-output extra fields (ported from a-immersive's MIDI-CC popup rows, `aimmersive-clone-spec.md:128`):
```ts
interface MidiCcSpec {
cc: number; // 0..127, auto-named from CC_NAMES
channel: number; // 1..16
value: number; // last sent, round(v*127)
// min/max/curve from baseline define the 0..127 range mapping
}
```
- **Top of modal**: **number of CCs** (the output count for the MIDI backend = `midiCCMap.length`, `aimmersive-clone-spec.md:144`), output-device `<select>`, MIDI preset `<select>` (`aimmersive-clone-spec.md:32`).
- **Per-CC row**: editable Name (text), **CC#** (0127 spinner, auto-renames from `CC_NAMES`), **Ch** (116), plus the baseline min/max/curve/state. 7-bit value = `round(applyGroupOverride(v,curve,min,max) * 127)` (`findings-engine-surface.md:36`). Persist via `nisps-midi-cc-map:<engineId>` key (`aimmersive-clone-spec.md:239`).
- **Send**: batched per block via `WebMidiBackend.send` (`engine-architecture.md` §1; `midiOutput.sendBatch`, `aimmersive-clone-spec.md:258`).
### 4.2 OSC backend
```ts
interface OscSpec {
path: string; // e.g. "/synth/cutoff"
min: number; // physical range lo (engineering units, not [0,1])
max: number; // physical range hi
// curve/state from baseline
}
```
- **Top of modal**: bridge connection status (WS, deferred contract — `engine-architecture.md` §1 `osc-bridge.ts`), add/remove path rows.
- **Per-output row**: **OSC path** (text), **range min/max** (typed numeric, physical units — distinct from the [0,1] baseline dual-range; the baseline min/max selects the normalised window, the OSC range maps that window to engineering units), plus baseline curve/state.
- v1: OSC bridge is **stubbed behind a locked contract** (`engine-architecture.md` §1) — the modal renders and persists config but emits only when a bridge is connected.
### 4.3 VCV backend
```ts
interface VcvSpec {
// per-output min/max/freeze ONLY — the simplest backend
// baseline min/max ARE the VCV range; baseline state 'fixed' IS freeze
}
```
- VCV adds **nothing beyond the baseline** — per-output min/max + freeze (= baseline `state==='fixed'`). The LED-ring palette derives from theme tokens (`findings-design-and-manifold.md:30`).
- **Top of modal**: per-channel min/max/freeze grid; that is the whole VCV advanced surface.
### 4.4 Web Audio (synth) — the default
The synth backend's advanced modal *is* the **group-override matrix** (§5 / the Powerful Synth Engine drawer's FULL depth): per-group master curve + per-param min/max/curve/mute over the 126 synth params (`aimmersive-clone-spec.md:137`). No extra per-output struct — the synth output *is* the baseline.
---
## 5. POWERFUL SYNTH ENGINE drawer
Owns the audio engine and its parameter shaping. **The string "C15" must never render** — the built-in engine is labelled **"Powerful Synth Engine"** (`findings-engine-surface.md:74`; CLAUDE.md). Engine display names come from `SynthEngine.displayName` (`aimmersive-clone-spec.md:278`).
- **Peek**: play/pause + master volume + the active engine name ("Powerful Synth Engine") + BPM.
- **Expand**:
- **Engine switcher** — Powerful Synth Engine (default) + other engines (PAFSynth, ChannelStrip, etc. from `mode_select`); alternative engines flagged where not-yet-wired (`findings-engine-surface.md:74`).
- **Audio controls**: Vol, BPM (a-immersive play drawer, `aimmersive-clone-spec.md:31`).
- **Arpeggiator** controls (`aimmersive-clone-spec.md:278`).
- **Synth preset `<select>`** — tiered Manual/Beginner/Intermediate/Advanced/Expert (`aimmersive-clone-spec.md:199`); these set which params are active/muted + their min/max/curve (the synth-side override tier — *distinct* from the visual/RL example presets, keep separate, `aimmersive-clone-spec.md:157`).
- **FULL**: the synth **group-override matrix** (§4.4) — 18 collapsible sections (Env A…Mono, `aimmersive-clone-spec.md:298`), each with a group master curve (48×48 drag, applies relative delta to all child curves, `aimmersive-clone-spec.md:138`) and per-param baseline rows (§3.2). This is `<GroupOverrideDrawer>` promoted to FULL depth.
- **Bind**: `engine.audio.start/stop`, `SynthEngine.setParam(i, v)` throttled ≥50ms / dead-zone >0.002 (`aimmersive-clone-spec.md:257,345` — load-bearing, prevents ring-buffer flooding). Audio starts lazily on the play gesture (`aimmersive-clone-spec.md:278`).
- COOP/COEP server-scoped (`findings-design-and-manifold.md:100`); SharedArrayBuffer only for the browser-only Powerful Synth Engine path (`findings-engine-surface.md:74`).
---
## 6. PARTICLE / VISUAL SYSTEM drawer
Owns the flow-field visualiser — a faithful port of `js/ui/visualizer.js` (`findings-design-and-manifold.md:120` — exact look + behaviour; workstream E). 400-particle Canvas2D flow field driven by 20 named outputs (`aimmersive-clone-spec.md:296`).
- **Peek**: visual on/off + the active visual preset name.
- **Expand**: the 20 visual params as per-output rows (§3 baseline — Flow/Scale/Speed/Hue/Spread/… `aimmersive-clone-spec.md:296`), each with its `VISUAL_PARAM_COLORS` swatch; plus **visual/RL preset chips** (Calm/Chaos, Rainbow, Vortex, Spiral, Embers, `aimmersive-clone-spec.md:158`) which teach the *network* by clearing the dataset and adding hardcoded input→output examples (distinct from synth tiers — keep separate).
- **FULL**: the full visual param matrix + the output→visual-param range table (verbatim ranges, `aimmersive-clone-spec.md:296`) as editable advanced mappings + particle-count / lifetime tuning.
- **Bind**: when the visual backend is active, `routedOutput` fans the 20 outputs into `FlowFieldVisualizer.setParams` (`aimmersive-clone-spec.md:260`); the canvas reads `engine.routedOutput()` in one rAF loop, never inference (`engine-architecture.md` §2.1).
---
## 7. React component tree
The Console wraps everything in `<EngineProvider>` (`engine-architecture.md` §2.2). The dock + drawers read `EngineApi` only; no drawer imports engine internals (lint seam, `engine-architecture.md` §6).
```
<ConsoleApp> // focus stage + dock; owns rAF for canvases
├── <Dock> // 48px right rail
│ └── <DockIcon> ×6 // LEARN, IN, OUT, SYNTH, VISUAL, HELP
│ (data-drawer; click → uiStore.setDrawer(depth))
├── <DrawerHost> // renders the open drawer at its depth
│ ├── <Drawer depth="peek|expand"> // generic glass panel; ⤢ → FULL
│ │ ├── <LearningBehaviourDrawer>
│ │ │ ├── <FeedbackModeSelector> // pill → engine.feedback.setMode (§1.1)
│ │ │ │ └── <ExploringIndicator> // <NoiseRing> ramp; learning-paused badge
│ │ │ ├── <ArmScopeChooser> // arm-all / arm-output (§1.2)
│ │ │ └── <TrainingParams> // <Slider> ×5/6 (§1.3)
│ │ ├── <InputsDrawer> // workstream F (§6); <ControlAxis> rows
│ │ ├── <RoutingDrawer> // §3
│ │ │ ├── <BackendSelector> // Web Audio/MIDI/OSC/VCV
│ │ │ └── <For each=group>
│ │ │ └── <OutputGroupSection> // header: group master curve + mute-group
│ │ │ └── <For each=output>
│ │ │ └── <OutputControlRow> // ← the shared baseline component (§3.2)
│ │ │ ├── <MuteToggle/> M
│ │ │ ├── <ArmToggle/> S
│ │ │ ├── <TriStateSegmented/> off|fixed|live
│ │ │ ├── <DualRangeSlider/> min/max
│ │ │ ├── <CurvePad/> 36×36
│ │ │ └── <ValueBar/> live | fixedValue
│ │ ├── <SynthEngineDrawer> // §5 ("Powerful Synth Engine" — never "C15")
│ │ │ ├── <EngineSwitcher/>
│ │ │ ├── <AudioControls/> Vol/BPM/Arp
│ │ │ └── <SynthPresetSelect/> // tiered
│ │ └── <VisualSystemDrawer> // §6
│ │ ├── <For each=visualOutput><OutputControlRow/></For>
│ │ └── <VisualPresetChips/>
│ │
│ └── <AdvancedModal depth="full"> // near-fullscreen, --z-modal, glass over dim
│ ├── <FeedbackLab/> // LEARN FULL (state-machine + diagnostics)
│ ├── <InputMatrix/> // IN FULL (workstream F)
│ ├── <BackendAdvanced/> // OUT FULL — switches on active backend:
│ │ ├── <MidiCcEditor/> // CC#/Ch/Name + count (§4.1)
│ │ ├── <OscPathEditor/> // path + physical range (§4.2)
│ │ ├── <VcvChannelEditor/> // min/max/freeze grid (§4.3)
│ │ └── <SynthGroupMatrix/> // 18 sections × params (§4.4 / §5 FULL)
│ └── <VisualMatrix/> // VISUAL FULL
├── <VerdictCluster> // +//undo/reroll; mirrors <FeedbackModeSelector> pill
└── <HelpModal> // HELP icon opens directly
```
Shared leaf primitives live in `shared/primitives/` (`<Slider>`, `<DualRangeSlider>`, `<CurvePad>`, `<NoiseRing>`, `<LossPlot>`, `<WeightHealth>`, `<LayerStats>`, `<GradientFlow>`, `<PillToggle>`, `<ControlAxis>``findings-design-and-manifold.md:60`). `<OutputControlRow>` is the one new composite component this spec introduces, reused across the Routing, Synth, Visual, and (optionally, §6 open choice 2) Inputs drawers.
---
## 8. Binding to the reactive spine + engine API (summary)
| UI surface | EngineApi call | Engine route |
|---|---|---|
| Feedback-mode pill | `engine.feedback.setMode(m)` | `nisps_ml_feedback_set_mode` (`plans/feedback-modes-port-spec.md` §4) |
| `` verdict | `engine.feedback.thumbsDown()` | dispatch on mode → `on_down` (`findings-feedback-behaviour.md:90`) |
| `+` verdict | `engine.feedback.thumbsUp()` | `addExample` + train, or `CommitStore` when exploring |
| Arm (S) | `engine.feedback.setFocus(mask)` | `nisps_ml_feedback_set_focus` + `buildPinMask` |
| Tri-state off/fixed | `routing-store` write → `weightsRevision`/pin recompute | `routedOutput` memo + pin mask (`engine-architecture.md` §2.1) |
| min/max/curve | `routing-store` write | `routedOutput` memo `applyGroupOverride` (`param-map.js:301`) |
| mute | `routing-store.muted` | send-effect zeroes/holds downstream, value still in `mlOutput` |
| training sliders | `mlStore` setters | live engine params |
| backend select | `engine.setBackend(b)` | swaps `OutputBackend` adapter; output-count guard |
| synth setParam | (internal) send-effect throttle | `SynthEngine.setParam` ≥50ms/>0.002 |
All writes are **eager, synchronous, off-React-render** (`engine-architecture.md` §4) — drawers mutate stores; the single `routedOutput`→send-effect carries it to the backend; canvases read accessors in rAF. No drawer creates a second data path (the live-feedback guarantee, `engine-architecture.md` §2.3).
---
## 9. Open choices for the operator
1. **Surface `RandomiseOutputs` (third feedback mode)?** Spec hides it behind `?debug=1` ("Static roll"). Confirm v1 ships only the two product modes (Push away / Explore & place) or all three.
2. **Do input-axis rows reuse `<OutputControlRow>`?** They share min/max/curve but not mute/off/live semantics. Spec defers to workstream F; confirm whether to unify the component or fork it.
3. **Separate `state`/`muted`/`armed` fields vs the deployed conflated `frozen`↔`muted` single field.** Spec deliberately splits them (cleaner tri-state); this is a divergence from the deployed override system to record in `ALIGNMENT.md`. Confirm.
4. **Default backend output counts on switch** trigger a `confirm()` weight-reset guard (ported from a-immersive). Confirm you want that friction, or prefer silent warm-start (`createWithWarmStart`, `aimmersive-clone-spec.md:250`).
5. **Group master curve relative-delta behaviour** (drag the group curve nudges all child curves preserving offsets, `aimmersive-clone-spec.md:138`) — keep this a-immersive behaviour, or switch to absolute group curve? Spec keeps relative.
6. **Where does the feedback-mode pill live for live play** — only mirrored next to the Verdict cluster, or also a permanent sub-`<StatusLine>` pill (`aimmersive-clone-spec.md:283`)? Spec puts it on the Verdict cluster.
---
## Verification corrections (adversarial pass, 2026-06-27) — verdict: minor-issues
The design reasoning, drawer model, tri-state/mute/arm separation and per-output baseline are sound; most
`aimmersive-clone-spec.md` citations verified line-by-line. Apply these fixes in the build:
- **`engine.feedback.setMode` / `setFocus` / `exploring()` are not yet a JS surface** — only the C ABI
(`nisps_ml_feedback_set_mode` / `set_focus` / `exploring`) exists. The Manifold `EngineApi.feedback` must
ADD these JS wrappers over the C ABI in Phase 3 (this is intended new surface, not a mistake — but it must be
built, not assumed). Likewise `engine.setBackend` and synth `setParam` are internal, not on EngineApi today.
- **`param-map.js` (`applyTame/applyCurve/applyGroupOverride`) lives only in `deployments/meml-aimmersive`**,
not in `playground/src`. The curve/override math must be **ported into `manifold/`** (see backends-spec
`mapping.ts`), not imported from the deployed snapshot.
- **`--dock-width` and min-thumb `#4488ff` are deployed-a-immersive values, absent from the Manifold tokens** —
use Manifold tokens (`manifold-export/tokens/`) or add the missing ones deliberately.
- A few `findings-feedback-behaviour.md:NN` citations are out of range / point at `plans/feedback-modes-port-spec.md`
instead (`learning_paused()` = `feedback.hpp:85` / port-spec:238; `exploring()` = `feedback.hpp:84`).
- The React `<EngineProvider>`/`useEngine` pattern is in `findings-design-and-manifold.md §4`, not
`engine-architecture.md §2.2` (that doc is SolidJS).