| **peek** | narrow flyout (~72px) hugging the rail | glanceable status + 1–2 primary toggles; no scrolling | `playground-2026.md` dock+drawer |
| **expand** | glass panel ~300px (cf. a-immersive drawer-stack `width:260px`, `aimmersive-clone-spec.md:312`) | the working surface: the per-output rows, the live sliders, the mode toggles | a-immersive drawers |
| **FULL** | near-fullscreen **modal** (`--z-modal`, glass over dimmed canvas) | the "advanced config page" for one entry — backend CC tables, full param matrix, weight-health lab | mission "advanced" page |
- **Depth is per-drawer state** in `uiStore` (`peek | expand | full`), independent across drawers (a-immersive allows multiple open at once, `aimmersive-clone-spec.md:84`). FULL is mutually exclusive — only one modal at a time (it captures the screen).
-`\` toggles the *focused* drawer peek↔expand; a drawer's header **⤢ button** (or double-click a row) opens FULL.
- Esc closes FULL→expand; closing FULL restores prior depth.
- Drawers slide in via `drawerSlideIn` 0.2s translateX (`aimmersive-clone-spec.md:189`). FULL fades via `helpFadeIn`.
- Dock icons get `.active` tint (`--accent #ff6a00`) when their drawer is open; macOS magnify on hover (`scale(1.35)`, `aimmersive-clone-spec.md:188`).
**Drawer roster** (six icons, top→bottom on the rail):
| Icon | Drawer | One-line role |
|---|---|---|
| 🧠 LEARN | **Learning-Behaviour** | Feedback-mode selector, SOLO/arm chooser, live training params |
| 🎚 IN | **Inputs** | Input source + per-axis pipeline (workstream F territory — referenced, §6) |
| 🔀 OUT | **Outputs / Routing** | The per-output control matrix + backend selector (workstream E + §3, §4) |
| ? HELP | **Help** | Onboarding overlay (opens modal directly, not a drawer) |
---
## 1. LEARNING-BEHAVIOUR drawer
Owns *how the model learns from your gestures*: the feedback-mode selector, the SOLO/arm variant chooser, and the live training knobs. Binds to `EngineApi.feedback` (`engine-architecture.md` §2.2) and the feedback C API (`findings-engine-surface.md:52`, `nisps_ml_feedback_*`).
The `+` (up) verdict is always "keep this" (`addExample` + train, `findings-feedback-behaviour.md:102`); the `−` (down) verdict is **selectable** among the ported feedback modes (`plans/feedback-modes-port-spec.md` §1; `FeedbackController<MLP_T>`). The new 2-mode product surface (per the prompt's "Mode 1 / Mode 2"):
| **Push away** (Mode 1) | `Avoid` (`nisps/ml/feedback.hpp`) | down → geometric dislike: stores the rejection and immediately pushes the mapping away from what you disliked; controller-owned elapsed-time replay then continues optimising live rejections (`docs/adr/rl-feedback-design.md`, 2026-07-25 amendment). |
| **Explore & place** (Mode 2) | `ExploreAndPlace` (`nisps/ml/feedback.hpp`) | down → snapshot + `draw_weights(spread)` re-rolls the whole net into a scratchpad you audition by moving the joystick; `+`/drag commits a positive anchor at the chosen input and **restores the real net**; re-roll and cancel remain distinct Explore-and-place actions (`docs/adr/rl-feedback-design.md` §2.2). |
- **Verdict pointer timing is mode-specific:** in **Push away**, pointer-down MUST dispatch exactly one geometric dislike immediately; holding, pointer-up, and release MUST NOT dispatch another dislike, re-roll/randomise, or cross-route into Explore-and-place. Controller-owned elapsed-time replay MAY continue after that single UI dispatch and is not a repeated held-pointer dispatch. **Explore & place alone** MAY interpret its existing long press as a re-roll.
- A third engine mode `RandomiseOutputs` (bypass MLP, hold a static random vector, `findings-feedback-behaviour.md:111`) exists in the core but is **not** surfaced as a product mode in v1 — expose it only behind `?debug=1` as "Static roll". **(Open choice 1.)**
- **Peek**: a 2-segment pill (`Push away` / `Explore & place`), the current mode highlighted in `--accent`. This pill is *also* mirrored next to the Verdict cluster (`VerdictCluster`, `findings-design-and-manifold.md:50`) so it is reachable during live play without opening the drawer — matching how a-immersive puts the `rl-label` above the RL buttons (`aimmersive-clone-spec.md:283`).
- **Expand**: the pill + a one-line plain-English description of the active mode + an **"exploring…" indicator** that lights when `engine.feedback.exploring()` is true (`findings-feedback-behaviour.md:237`), reusing the `<NoiseRing>` colour ramp (off/active/high, `aimmersive-clone-spec.md:98`). While exploring in Mode 2, training is paused (`learning_paused()`, `findings-feedback-behaviour.md:48`) — show a small "learning paused" badge.
- **FULL**: the feedback lab — a diagram of the active state machine (idle → exploring → commit/cancel), the raw `FeedbackAction` log of the last presses (`findings-feedback-behaviour.md:90` enum), plus the spread/tame "Health lab" sliders (§1.3) for radical exploration tuning.
- **Bind point**: `engine.feedback.setMode(mode)` → `nisps_ml_feedback_set_mode` (`plans/feedback-modes-port-spec.md` §4). Switching mode while exploring auto-aborts and restores the net (`set_mode` calls `abort_explore`, `findings-feedback-behaviour.md:96`). Persist `feedbackMode` in the session blob.
"Solo" and "arm" are the SAME concept (the prompt: *solo(=arm)*): **focus training on one output**. It maps to the engine's **focus mask** (`activeDims_` → `nisps_ml_feedback_set_focus`, `findings-feedback-behaviour.md:159-167`) AND to the **output-pin mask** used by `move_weights`/`buildPinMask` (`findings-feedback-behaviour.md:117`, `mode-runtime.ts:548`).
Two armable scopes (the chooser):
| Variant | Effect | Engine wiring |
|---|---|---|
| **Arm output** (per-row solo) | only the armed output(s) learn/re-roll; all others are frozen in the model | `set_focus(mask)` where `mask[i]=1` for armed dims; unmuted-but-unarmed dims set `0` so `move_weights`/`roll_static_outputs` leave them put (`findings-feedback-behaviour.md:71,103-107`) |
| **Arm all** (default) | every live output learns (empty mask ⇒ all active, `findings-feedback-behaviour.md:161`) | `clear_focus_mask()` |
- Arming is a per-output toggle (the **S** button on each control row, §3) — arming any output flips the drawer into "arm-output" scope automatically; clicking the drawer's **"Arm all"** chip clears it.
- Multiple outputs may be armed at once (mask is a vector). The armed set is highlighted on the ReadoutStrip and on the heatmap (border glow, `--glow-focus`).
- **Peek**: shows "Arm: all" or "Arm: 3 outputs"; tapping cycles to "Arm all".
- **Expand**: the scope chips + a compact list of currently-armed output names with quick-unarm.
- **Persisted** as part of routing state (the focus mask is reconstructable from per-row `armed` flags).
### 1.3 Live training params
Six live knobs, lifted from a-immersive's "NISPS" drawer (`aimmersive-clone-spec.md:67` — the deployed app's *actual* tuning surface, six raw sliders) plus the spread/tame regime controls:
- **Expand**: Noise, Spread, Tame, Learning rate, Decay as `<Slider>`s.
- **FULL**: all six + Max iters/Convergence + a real `<LossPlot>` (needs `nisps_ml_loss_history`, `findings-design-and-manifold.md:62`) + `<WeightHealth>` edge-glow + `<LayerStats>` + `<GradientFlow>` (the diagnostics suite, `findings-design-and-manifold.md:63`). This is the "advanced learning" page.
- All writes are eager engine setters (no React-render coupling, `engine-architecture.md` §4); persisted in the session blob.
---
## 2. INPUTS drawer (workstream F territory — referenced)
**Owner: workstream F (modular N×M inputs + input pipeline).** This drawer is specified there; here we only fix its *dock shape* so the dock model is complete and the two specs dovetail.
- **Source**: input source selector — **Joystick / Hands / MIDI / Mic** (a-immersive Input pill, `aimmersive-clone-spec.md:150`). v1 browser is **fixed-2-input** (`findings-engine-surface.md:67`; `MLP<2,…>`) — modes needing >2 inputs show a "single-input in browser" badge (`engine-architecture.md` §6 open Q2).
- **Per-axis pipeline** (deadzone→zoom→curve→smoothing→momentum, `input/pipeline.ts`, `findings-design-and-manifold.md:84`): one `<ControlAxis>` row per input axis with its pipeline knobs.
- **Depth**: peek = source + zoom; expand = per-axis pipeline rows; FULL = the N×M input-routing matrix (workstream F).
- **Bind**: `engine.setInput(x,y)` is the only input door (`engine-architecture.md` §2.2); pipeline config lives in `input-store`.
- **Per-axis controls reuse the per-output row component** (§3) where sensible (an input axis has min/max/curve too), but mute/solo/freeze semantics differ — defer those to workstream F. **(Open choice 2: confirm input-axis rows share the §3 component or get their own.)**
---
## 3. OUTPUTS / ROUTING drawer + the per-output control row
The heart of this spec. **Owner of the routing matrix + per-output baseline is shared; backend-specific advanced layouts are workstream E (VCV/OSC/MIDI LED-ring backends, `findings-design-and-manifold.md:30`).**
### 3.1 The per-output baseline (EVERY output, EVERY backend)
Every output — synth param, MIDI CC, OSC path, VCV channel, visual param — shares one baseline control set. This faithfully ports the deployed vanilla-JS per-param override system (`aimmersive-clone-spec.md` §2.4 heatmap popup + §2.5 group drawer; `param-map.js``applyGroupOverride`/`applyCurve`/`applyTame`). The baseline:
| Control | Glyph / widget | Semantics |
|---|---|---|
| **Mute** | **M** toggle | silenced *downstream* but still computed + visible (distinct from off) |
| **Solo / Arm** | **S** toggle | focus training on this output (§1.2; focus mask + pin mask) |
group: string; // section (Env A, Osc B, …) for grouping
state: OutputState; // off | fixed | live
muted: boolean; // downstream silence; still computed
armed: boolean; // solo / focus-training (=arm)
min: number; // [0,1]
max: number; // [0,1], min<=max enforced
curve: number; // [0,1], 0.5 linear
fixedValue: number; // held value when state==='fixed' (captured on freeze)
// backend-specific, populated by the active backend adapter:
backend?: MidiCcSpec | OscSpec | VcvSpec;
}
```
- Stored in `routing-store` as a `createStore` array indexed by output (`engine-architecture.md` §1; `overrideStore` shape, `aimmersive-clone-spec.md:230`). This unifies a-immersive's split `visualOverrides`/`groupOverrides`/`engineParamOverrides`/`midiCCOverrides` (`aimmersive-clone-spec.md:230-233`) behind one `getSectionView` adapter (`aimmersive-clone-spec.md:140`).
- **Mute ↔ off are distinct fields** (the prompt insists): `muted` is a downstream gate; `state==='off'` excludes from model control. They compose (an output can be off AND muted).
- **The deployed app conflates frozen↔muted via one field** (`aimmersive-clone-spec.md:347` — heatmap popup `frozen` ↔ group drawer `muted` map to the same underlying field). The new model **separates them deliberately**: `state` carries off/fixed/live, `muted` is its own boolean, `armed` its own boolean. This is the one deliberate divergence from the deployed semantics — record in `ALIGNMENT.md`. **(Open choice 3.)**
- **Reactive binding**: the row's controls write the store eagerly; `routedOutput` memo (`engine-architecture.md` §2.1) reads `routing-store` to remap each output via `applyGroupOverride`. The single send-effect pushes to the active backend. A row never holds a second data path.
### 3.3 Tri-state semantics table (precise)
This matches the deployed per-param override system: `off`/`fixed` correspond to a-immersive's `frozen` (excluded/pinned, `aimmersive-clone-spec.md:131`), `live` to the default model-driven path.
| State | Computed by model? | Pin mask (`buildPinMask`) | Sent downstream? | Value emitted | UI |
|---|---|---|---|---|---|
| **off** | **No** — removed from model control; final-layer weights pinned so RL/train never touch it (`findings-feedback-behaviour.md:117`) | pinned (excluded) | yes, at last held value | `fixedValue` (held/excluded) | dimmed row, no bar motion |
| **fixed** (freeze) | computed but result discarded; held at a static value | pinned (protect the held dim) | yes | `fixedValue` (captured on freeze, draggable, `aimmersive-clone-spec.md:131`) | hatched bar overlay (`aimmersive-clone-spec.md:190`), value slider shown |
| **live** | **Yes** — model-driven | not pinned | yes | `applyGroupOverride(modelOut, curve, min, max)` | full bar, animates |
Orthogonal modifiers (compose with any state):
| Modifier | Computed? | Visible? | Sent downstream? | Training focus |
> Precise difference the prompt demands: **off** = the model no longer drives this output (excluded from learning, held/pinned). **fixed/freeze** = held at a static value (also pinned, but conceptually "I chose this value", with a draggable `fixedValue`). **mute** = silenced downstream but *still computed and visible* (distinct from off — you still see it move, you just don't hear it). **solo/arm** = focus training on this output.
- **Expand**: the **routing matrix** — a scrollable `<For>` of `<OutputControlRow>`s grouped by `group` (collapsible section headers carry a **group master curve** + **mute-group** like a-immersive's group drawer, `aimmersive-clone-spec.md:137`). Each row: name · M · S · [off|fixed|live] · dual-range · curve pad · value. 126 rows for synth — virtualise / collapse non-live by default.
- **FULL**: the **Advanced backend modal** (§4) — backend selector at the top, then the backend-specific editor over the full output set.
All backends share the §3.1 baseline (M/S/off-fixed-live/min/max/curve). The FULL modal adds the backend-specific fields. **(Workstream E owns the VCV/OSC/MIDI backend internals; this fixes the modal layout + state contract.)**
### 4.1 MIDI backend
Per-output extra fields (ported from a-immersive's MIDI-CC popup rows, `aimmersive-clone-spec.md:128`):
```ts
interface MidiCcSpec {
cc: number; // 0..127, auto-named from CC_NAMES
channel: number; // 1..16
value: number; // last sent, round(v*127)
// min/max/curve from baseline define the 0..127 range mapping
}
```
- **Top of modal**: **number of CCs** (the output count for the MIDI backend = `midiCCMap.length`, `aimmersive-clone-spec.md:144`), output-device `<select>`, MIDI preset `<select>` (`aimmersive-clone-spec.md:32`).
- **Per-CC row**: editable Name (text), **CC#** (0–127 spinner, auto-renames from `CC_NAMES`), **Ch** (1–16), plus the baseline min/max/curve/state. 7-bit value = `round(applyGroupOverride(v,curve,min,max) * 127)` (`findings-engine-surface.md:36`). Persist via `nisps-midi-cc-map:<engineId>` key (`aimmersive-clone-spec.md:239`).
- **Send**: batched per block via `WebMidiBackend.send` (`engine-architecture.md` §1; `midiOutput.sendBatch`, `aimmersive-clone-spec.md:258`).
### 4.2 OSC backend
```ts
interface OscSpec {
path: string; // e.g. "/synth/cutoff"
min: number; // physical range lo (engineering units, not [0,1])
max: number; // physical range hi
// curve/state from baseline
}
```
- **Top of modal**: bridge connection status (WS, deferred contract — `engine-architecture.md` §1 `osc-bridge.ts`), add/remove path rows.
- **Per-output row**: **OSC path** (text), **range min/max** (typed numeric, physical units — distinct from the [0,1] baseline dual-range; the baseline min/max selects the normalised window, the OSC range maps that window to engineering units), plus baseline curve/state.
- v1: OSC bridge is **stubbed behind a locked contract** (`engine-architecture.md` §1) — the modal renders and persists config but emits only when a bridge is connected.
### 4.3 VCV backend
```ts
interface VcvSpec {
// per-output min/max/freeze ONLY — the simplest backend
// baseline min/max ARE the VCV range; baseline state 'fixed' IS freeze
}
```
- VCV adds **nothing beyond the baseline** — per-output min/max + freeze (= baseline `state==='fixed'`). The LED-ring palette derives from theme tokens (`findings-design-and-manifold.md:30`).
- **Top of modal**: per-channel min/max/freeze grid; that is the whole VCV advanced surface.
### 4.4 Web Audio (synth) — the default
The synth backend's advanced modal *is* the **group-override matrix** (§5 / the Powerful Synth Engine drawer's FULL depth): per-group master curve + per-param min/max/curve/mute over the 126 synth params (`aimmersive-clone-spec.md:137`). No extra per-output struct — the synth output *is* the baseline.
---
## 5. POWERFUL SYNTH ENGINE drawer
Owns the audio engine and its parameter shaping. **The string "C15" must never render** — the built-in engine is labelled **"Powerful Synth Engine"** (`findings-engine-surface.md:74`; CLAUDE.md). Engine display names come from `SynthEngine.displayName` (`aimmersive-clone-spec.md:278`).
- **Peek**: play/pause + master volume + the active engine name ("Powerful Synth Engine") + BPM.
- **Expand**:
- **Engine switcher** — Powerful Synth Engine (default) + other engines (PAFSynth, ChannelStrip, etc. from `mode_select`); alternative engines flagged where not-yet-wired (`findings-engine-surface.md:74`).
- **Audio controls**: Vol, BPM (a-immersive play drawer, `aimmersive-clone-spec.md:31`).
- **Synth preset `<select>`** — tiered Manual/Beginner/Intermediate/Advanced/Expert (`aimmersive-clone-spec.md:199`); these set which params are active/muted + their min/max/curve (the synth-side override tier — *distinct* from the visual/RL example presets, keep separate, `aimmersive-clone-spec.md:157`).
- **FULL**: the synth **group-override matrix** (§4.4) — 18 collapsible sections (Env A…Mono, `aimmersive-clone-spec.md:298`), each with a group master curve (48×48 drag, applies relative delta to all child curves, `aimmersive-clone-spec.md:138`) and per-param baseline rows (§3.2). This is `<GroupOverrideDrawer>` promoted to FULL depth.
- **Bind**: `engine.audio.start/stop`, `SynthEngine.setParam(i, v)` throttled ≥50ms / dead-zone >0.002 (`aimmersive-clone-spec.md:257,345` — load-bearing, prevents ring-buffer flooding). Audio starts lazily on the play gesture (`aimmersive-clone-spec.md:278`).
- COOP/COEP server-scoped (`findings-design-and-manifold.md:100`); SharedArrayBuffer only for the browser-only Powerful Synth Engine path (`findings-engine-surface.md:74`).
---
## 6. PARTICLE / VISUAL SYSTEM drawer
Owns the flow-field visualiser — a faithful port of `js/ui/visualizer.js` (`findings-design-and-manifold.md:120` — exact look + behaviour; workstream E). 400-particle Canvas2D flow field driven by 20 named outputs (`aimmersive-clone-spec.md:296`).
- **Peek**: visual on/off + the active visual preset name.
- **Expand**: the 20 visual params as per-output rows (§3 baseline — Flow/Scale/Speed/Hue/Spread/… `aimmersive-clone-spec.md:296`), each with its `VISUAL_PARAM_COLORS` swatch; plus **visual/RL preset chips** (Calm/Chaos, Rainbow, Vortex, Spiral, Embers, `aimmersive-clone-spec.md:158`) which teach the *network* by clearing the dataset and adding hardcoded input→output examples (distinct from synth tiers — keep separate).
- **FULL**: the full visual param matrix + the output→visual-param range table (verbatim ranges, `aimmersive-clone-spec.md:296`) as editable advanced mappings + particle-count / lifetime tuning.
- **Bind**: when the visual backend is active, `routedOutput` fans the 20 outputs into `FlowFieldVisualizer.setParams` (`aimmersive-clone-spec.md:260`); the canvas reads `engine.routedOutput()` in one rAF loop, never inference (`engine-architecture.md` §2.1).
---
## 7. React component tree
The Console wraps everything in `<EngineProvider>` (`engine-architecture.md` §2.2). The dock + drawers read `EngineApi` only; no drawer imports engine internals (lint seam, `engine-architecture.md` §6).
```
<ConsoleApp> // focus stage + dock; owns rAF for canvases
├── <Dock> // 48px right rail
│ └── <DockIcon>×6 // LEARN, IN, OUT, SYNTH, VISUAL, HELP
│ (data-drawer; click → uiStore.setDrawer(depth))
│
├── <DrawerHost> // renders the open drawer at its depth
│ ├── <Drawerdepth="peek|expand"> // generic glass panel; ⤢ → FULL
Shared leaf primitives live in `shared/primitives/` (`<Slider>`, `<DualRangeSlider>`, `<CurvePad>`, `<NoiseRing>`, `<LossPlot>`, `<WeightHealth>`, `<LayerStats>`, `<GradientFlow>`, `<PillToggle>`, `<ControlAxis>` — `findings-design-and-manifold.md:60`). `<OutputControlRow>` is the one new composite component this spec introduces, reused across the Routing, Synth, Visual, and (optionally, §6 open choice 2) Inputs drawers.
---
## 8. Binding to the reactive spine + engine API (summary)
All writes are **eager, synchronous, off-React-render** (`engine-architecture.md` §4) — drawers mutate stores; the single `routedOutput`→send-effect carries it to the backend; canvases read accessors in rAF. No drawer creates a second data path (the live-feedback guarantee, `engine-architecture.md` §2.3).
---
## 9. Open choices for the operator
1.**Surface `RandomiseOutputs` (third feedback mode)?** Spec hides it behind `?debug=1` ("Static roll"). Confirm v1 ships only the two product modes (Push away / Explore & place) or all three.
2.**Do input-axis rows reuse `<OutputControlRow>`?** They share min/max/curve but not mute/off/live semantics. Spec defers to workstream F; confirm whether to unify the component or fork it.
3.**Separate `state`/`muted`/`armed` fields vs the deployed conflated `frozen`↔`muted` single field.** Spec deliberately splits them (cleaner tri-state); this is a divergence from the deployed override system to record in `ALIGNMENT.md`. Confirm.
4.**Default backend output counts on switch** trigger a `confirm()` weight-reset guard (ported from a-immersive). Confirm you want that friction, or prefer silent warm-start (`createWithWarmStart`, `aimmersive-clone-spec.md:250`).
5.**Group master curve relative-delta behaviour** (drag the group curve nudges all child curves preserving offsets, `aimmersive-clone-spec.md:138`) — keep this a-immersive behaviour, or switch to absolute group curve? Spec keeps relative.
6.**Where does the feedback-mode pill live for live play** — only mirrored next to the Verdict cluster, or also a permanent sub-`<StatusLine>` pill (`aimmersive-clone-spec.md:283`)? Spec puts it on the Verdict cluster.
