Plan §6 says each P5 item is spec-first with its own session. These are the
specs; no implementation is authorised by them and none was written.
plans/mode-layer-reunification.md 5a — ALIGNMENT defect 1, the largest
architectural gap. Storage-policies the
ModeBase orchestration the way P2 did
MLPCore, rather than binding monolithic
mode objects into WASM (which would
contradict the locked two-instance RT
architecture).
plans/browser-mode-coverage-spec.md 5b — an audio-topology notion so Manifold
stops cataloguing 4 modes that
structurally cannot run in a browser.
plans/curated-presets-spec.md 5c — ALIGNMENT defect 2, built on the
operator's §7.6 definition: a curated
preset is configuration only, network
untrained.
plans/hardware-editor-spec.md 5d — ALIGNMENT defect 3, applying
useq-celium's existing discipline (C
header as wire truth + TS mirror + parity
test) to a MEMLNaut serial protocol.
2434 lines. I spot-checked their path citations mechanically against
git ls-files: of 170 backticked paths, every unresolved one is either a file
the spec proposes to create or a cross-reference to a sibling spec in this same
commit. None describes deleted code as live — which is the failure mode that
made half the existing corpus untrustworthy, and the reason the §8 pass earlier
today had so much to do.
The preset spec is the most valuable byproduct: en route it found five places
where existing docs still describe a deleted world, including
manifold-parity-features-spec.md §1.1 specifying a PipelineLayer over
engine/input-pipeline.ts and output-pipeline.ts, both deleted at one-core P4.
Also carries the doc sync for the telemetry and benchmark work in a77770f:
AGENT-REFERENCE gains the throughput and loss-history entries, and dock-spec
§1.3 records that its long-deferred diagnostics suite shipped PARTLY — the loss
curve and weight-health table are real, GradientFlow is not built and is not
planned as drawn (the core records no per-layer gradient magnitudes, and the
fabricated version was deleted in Phase 1).
33 KiB
| kind | stability | layer |
|---|---|---|
| spec | evolving | behavioural |
Dock Spec — Console Right-Dock Drawers + Per-Output Controls
Workstream D. Designed 2026-06-27; built 2026-06-28 with operator-directed restructuring. British spelling in product copy. The built-in synth is the "Powerful Synth Engine" — the string "C15" MUST NEVER appear in the UI.
Grounding note (2026-07-21). Treat every
file:linecite in the body as historical grounding:playground/*died with the retired SolidJS playground,aimmersive-clone-spec.mdis archived at_archive/aimmersive-clone-spec.md, andengine-architecture.mdwas trimmed 2026-07 (its cited § numbers are gone). What shipped diverges from the prescription in shape but not in substance:
- Drawer roster: the operator restructured the dock during the build — a top Mode selector plus five drawers (
learn,inputs,route,settings,help—manifold/src/console/Dock.tsx/Drawers.tsx). The §0 six-icon roster (SYNTH/VISUAL as drawers) became per-Mode config inside the Outputs drawer instead.- Depths: two shipped, not three —
condensed(360px side panel) andexpanded(centred modal). The separate FULL "advanced backend modal" (§4,<BackendAdvanced/>) was built, then deleted as a duplicate editor (2026-07 sweep, S18); the sole per-backend editor ismanifold/src/dock/OutputsBackendConfig.tsxin the Outputs drawer.- The per-output control row (§3, the heart of this spec) shipped essentially as designed:
manifold/src/dock/OutputControlRow.tsx+output-state.ts(off/fixed/live + mute + solo/arm + min/max/curve).- The feedback surface shipped richer than §1 (Explore-and-Place with reroll/nudge/place, plus geometric dislike) — see
docs/adr/rl-feedback-design.mdandnisps/ml/feedback.hpp.- §1.3's diagnostics suite shipped PARTLY, 2026-07-21 (simplification-plan §6.5e). The
nisps_ml_loss_historyC API the section waited on now exists, andmanifold/src/console/TrainingHealth.tsxrenders the real per-iteration loss curve plus the per-layer weight-health table (nisps_ml_get_layer_stats) atexpandeddepth — the advanced surface, since only two depths shipped.<GradientFlow>is NOT built and is not planned as drawn: the core records no per-layer gradient magnitudes, and the fabricated version of it was deleted in the Phase-1 sweep.<WeightHealth>as an edge-glow is likewise unbuilt; the same numbers are in the table. §1.3's six live sliders were deleted as decorative (S16) —noise capand the Xavier switch are the survivors, because they drive real engine setters.
0. The 3-depth dock model (frame the design)
The Console's right dock is a 48px rail (--dock-width, aimmersive-clone-spec.md:181) of icons; each icon owns one drawer with three depths:
| Depth | Width / placement | Purpose | Source vocabulary |
|---|---|---|---|
| peek | narrow flyout (~72px) hugging the rail | glanceable status + 1–2 primary toggles; no scrolling | playground-2026.md dock+drawer |
| expand | glass panel ~300px (cf. a-immersive drawer-stack width:260px, aimmersive-clone-spec.md:312) |
the working surface: the per-output rows, the live sliders, the mode toggles | a-immersive drawers |
| FULL | near-fullscreen modal (--z-modal, glass over dimmed canvas) |
the "advanced config page" for one entry — backend CC tables, full param matrix, weight-health lab | mission "advanced" page |
- Depth is per-drawer state in
uiStore(peek | expand | full), independent across drawers (a-immersive allows multiple open at once,aimmersive-clone-spec.md:84). FULL is mutually exclusive — only one modal at a time (it captures the screen). \toggles the focused drawer peek↔expand; a drawer's header ⤢ button (or double-click a row) opens FULL.- Esc closes FULL→expand; closing FULL restores prior depth.
- Drawers slide in via
drawerSlideIn0.2s translateX (aimmersive-clone-spec.md:189). FULL fades viahelpFadeIn. - Dock icons get
.activetint (--accent #ff6a00) when their drawer is open; macOS magnify on hover (scale(1.35),aimmersive-clone-spec.md:188).
Drawer roster (six icons, top→bottom on the rail):
| Icon | Drawer | One-line role |
|---|---|---|
| 🧠 LEARN | Learning-Behaviour | Feedback-mode selector, SOLO/arm chooser, live training params |
| 🎚 IN | Inputs | Input source + per-axis pipeline (workstream F territory — referenced, §6) |
| 🔀 OUT | Outputs / Routing | The per-output control matrix + backend selector (workstream E + §3, §4) |
| 🔊 SYNTH | Powerful Synth Engine | Engine switch + audio/arp controls + group overrides |
| ✦ VISUAL | Particle / Visual System | Flow-field visualiser params + presets |
| ? HELP | Help | Onboarding overlay (opens modal directly, not a drawer) |
1. LEARNING-BEHAVIOUR drawer
Owns how the model learns from your gestures: the feedback-mode selector, the SOLO/arm variant chooser, and the live training knobs. Binds to EngineApi.feedback (engine-architecture.md §2.2) and the feedback C API (findings-engine-surface.md:52, nisps_ml_feedback_*).
1.1 FEEDBACK_MODE selector ("Down Action")
The + (up) verdict is always "keep this" (addExample + train, findings-feedback-behaviour.md:102); the − (down) verdict is selectable among the ported feedback modes (plans/feedback-modes-port-spec.md §1; FeedbackController<MLP_T>). The new 2-mode product surface (per the prompt's "Mode 1 / Mode 2"):
| Selector label (UI copy) | Engine FeedbackMode |
Behaviour |
|---|---|---|
| Push away (Mode 1) | Avoid (feedback.hpp enum, findings-feedback-behaviour.md:90) |
down → geometric-dislike: perturbs the mapping away from what you disliked. In the deployed core this routes to move_weights(speed, spread, pinMask) (plans/feedback-modes-port-spec.md §2.5 — the true k-NN centroid push is firmware-only, out of scope). |
| Explore & place (Mode 2) | RandomiseMlp (findings-feedback-behaviour.md:122) |
down → snapshot + draw_weights(spread) re-rolls the whole net into a scratchpad you audition by moving the joystick; +/drag commits a +1 example at the chosen input and restores the real net (findings-feedback-behaviour.md:135-146); down-again cancels (restore snapshot). |
- A third engine mode
RandomiseOutputs(bypass MLP, hold a static random vector,findings-feedback-behaviour.md:111) exists in the core but is not surfaced as a product mode in v1 — expose it only behind?debug=1as "Static roll". (Open choice 1.) - Peek: a 2-segment pill (
Push away/Explore & place), the current mode highlighted in--accent. This pill is also mirrored next to the Verdict cluster (VerdictCluster,findings-design-and-manifold.md:50) so it is reachable during live play without opening the drawer — matching how a-immersive puts therl-labelabove the RL buttons (aimmersive-clone-spec.md:283). - Expand: the pill + a one-line plain-English description of the active mode + an "exploring…" indicator that lights when
engine.feedback.exploring()is true (findings-feedback-behaviour.md:237), reusing the<NoiseRing>colour ramp (off/active/high,aimmersive-clone-spec.md:98). While exploring in Mode 2, training is paused (learning_paused(),findings-feedback-behaviour.md:48) — show a small "learning paused" badge. - FULL: the feedback lab — a diagram of the active state machine (idle → exploring → commit/cancel), the raw
FeedbackActionlog of the last presses (findings-feedback-behaviour.md:90enum), plus the spread/tame "Health lab" sliders (§1.3) for radical exploration tuning. - Bind point:
engine.feedback.setMode(mode)→nisps_ml_feedback_set_mode(plans/feedback-modes-port-spec.md§4). Switching mode while exploring auto-aborts and restores the net (set_modecallsabort_explore,findings-feedback-behaviour.md:96). PersistfeedbackModein the session blob.
1.2 SOLO / arm variant chooser
"Solo" and "arm" are the SAME concept (the prompt: solo(=arm)): focus training on one output. It maps to the engine's focus mask (activeDims_ → nisps_ml_feedback_set_focus, findings-feedback-behaviour.md:159-167) AND to the output-pin mask used by move_weights/buildPinMask (findings-feedback-behaviour.md:117, mode-runtime.ts:548).
Two armable scopes (the chooser):
| Variant | Effect | Engine wiring |
|---|---|---|
| Arm output (per-row solo) | only the armed output(s) learn/re-roll; all others are frozen in the model | set_focus(mask) where mask[i]=1 for armed dims; unmuted-but-unarmed dims set 0 so move_weights/roll_static_outputs leave them put (findings-feedback-behaviour.md:71,103-107) |
| Arm all (default) | every live output learns (empty mask ⇒ all active, findings-feedback-behaviour.md:161) |
clear_focus_mask() |
- Arming is a per-output toggle (the S button on each control row, §3) — arming any output flips the drawer into "arm-output" scope automatically; clicking the drawer's "Arm all" chip clears it.
- Multiple outputs may be armed at once (mask is a vector). The armed set is highlighted on the ReadoutStrip and on the heatmap (border glow,
--glow-focus). - Peek: shows "Arm: all" or "Arm: 3 outputs"; tapping cycles to "Arm all".
- Expand: the scope chips + a compact list of currently-armed output names with quick-unarm.
- Persisted as part of routing state (the focus mask is reconstructable from per-row
armedflags).
1.3 Live training params
Six live knobs, lifted from a-immersive's "NISPS" drawer (aimmersive-clone-spec.md:67 — the deployed app's actual tuning surface, six raw sliders) plus the spread/tame regime controls:
| Slider | Range | Default | Binds to |
|---|---|---|---|
| Noise | 0–noiseCap | 0.05 | mlStore.noiseLevel; the − perturbation magnitude (findings-feedback-behaviour.md:110) |
| Learning rate | 1e-6 – 1e-2 (log) | 1e-5 | iml.learningRate (aimmersive-clone-spec.md:226) |
| Decay | 0.8–1.0 | 0.97 | rlExplorationDecay (findings-feedback-behaviour.md:103) |
| Spread | 0–1 | 0.6 | spreadLevel — master noise/Xavier regime (findings-feedback-behaviour.md:108; CLAUDE.md spread) |
| Tame | 0–1 | 1 | output-range safety toward [safeMin,safeMax] (param-map.js:261 applyTame) |
| Max iters / Convergence | — | — | iml.maxIterations/convergenceThreshold (aimmersive-clone-spec.md:274) — FULL-depth only |
- Peek: Noise + Spread (the two most live).
- Expand: Noise, Spread, Tame, Learning rate, Decay as
<Slider>s. - FULL: all six + Max iters/Convergence + a real
<LossPlot>(needsnisps_ml_loss_history,findings-design-and-manifold.md:62) +<WeightHealth>edge-glow +<LayerStats>+<GradientFlow>(the diagnostics suite,findings-design-and-manifold.md:63). This is the "advanced learning" page. - All writes are eager engine setters (no React-render coupling,
engine-architecture.md§4); persisted in the session blob.
2. INPUTS drawer (workstream F territory — referenced)
Owner: workstream F (modular N×M inputs + input pipeline). This drawer is specified there; here we only fix its dock shape so the dock model is complete and the two specs dovetail.
- Source: input source selector — Joystick / Hands / MIDI / Mic (a-immersive Input pill,
aimmersive-clone-spec.md:150). v1 browser is fixed-2-input (findings-engine-surface.md:67;MLP<2,…>) — modes needing >2 inputs show a "single-input in browser" badge (engine-architecture.md§6 open Q2). - Per-axis pipeline (deadzone→zoom→curve→smoothing→momentum,
input/pipeline.ts,findings-design-and-manifold.md:84): one<ControlAxis>row per input axis with its pipeline knobs. - Depth: peek = source + zoom; expand = per-axis pipeline rows; FULL = the N×M input-routing matrix (workstream F).
- Bind:
engine.setInput(x,y)is the only input door (engine-architecture.md§2.2); pipeline config lives ininput-store. - Per-axis controls reuse the per-output row component (§3) where sensible (an input axis has min/max/curve too), but mute/solo/freeze semantics differ — defer those to workstream F. (Open choice 2: confirm input-axis rows share the §3 component or get their own.)
3. OUTPUTS / ROUTING drawer + the per-output control row
The heart of this spec. Owner of the routing matrix + per-output baseline is shared; backend-specific advanced layouts are workstream E (VCV/OSC/MIDI LED-ring backends, findings-design-and-manifold.md:30).
3.1 The per-output baseline (EVERY output, EVERY backend)
Every output — synth param, MIDI CC, OSC path, VCV channel, visual param — shares one baseline control set. This faithfully ports the deployed vanilla-JS per-param override system (aimmersive-clone-spec.md §2.4 heatmap popup + §2.5 group drawer; param-map.js applyGroupOverride/applyCurve/applyTame). The baseline:
| Control | Glyph / widget | Semantics |
|---|---|---|
| Mute | M toggle | silenced downstream but still computed + visible (distinct from off) |
| Solo / Arm | S toggle | focus training on this output (§1.2; focus mask + pin mask) |
| Tri-state | off / fixed / live segmented | model-control state (table §3.3) |
| Min / Max | <DualRangeSlider> (min blue #4488ff, max orange #ff6a00, aimmersive-clone-spec.md:130) |
output range remap; applyGroupOverride(v,curve,min,max) (param-map.js:301) |
| Curve | <CurvePad> 36×36 drag canvas, 0.5=linear (aimmersive-clone-spec.md:128; applyCurve, param-map.js:287) |
per-output response curve |
| Value | inline read-out / drag bar | live model value, or the held value when fixed |
3.2 The control-row component + state model
// manifold/src/engine/routing/output-control.ts (engine-side, headless)
type OutputState = 'off' | 'fixed' | 'live'; // the tri-state
interface OutputControl {
index: number; // model output index (0..125 for synth)
name: string; // schema label (NEVER "C15"; synth params from SYNTH_PARAM_MAP)
group: string; // section (Env A, Osc B, …) for grouping
state: OutputState; // off | fixed | live
muted: boolean; // downstream silence; still computed
armed: boolean; // solo / focus-training (=arm)
min: number; // [0,1]
max: number; // [0,1], min<=max enforced
curve: number; // [0,1], 0.5 linear
fixedValue: number; // held value when state==='fixed' (captured on freeze)
// backend-specific, populated by the active backend adapter:
backend?: MidiCcSpec | OscSpec | VcvSpec;
}
- Stored in
routing-storeas acreateStorearray indexed by output (engine-architecture.md§1;overrideStoreshape,aimmersive-clone-spec.md:230). This unifies a-immersive's splitvisualOverrides/groupOverrides/engineParamOverrides/midiCCOverrides(aimmersive-clone-spec.md:230-233) behind onegetSectionViewadapter (aimmersive-clone-spec.md:140). - Mute ↔ off are distinct fields (the prompt insists):
mutedis a downstream gate;state==='off'excludes from model control. They compose (an output can be off AND muted). - The deployed app conflates frozen↔muted via one field (
aimmersive-clone-spec.md:347— heatmap popupfrozen↔ group drawermutedmap to the same underlying field). The new model separates them deliberately:statecarries off/fixed/live,mutedis its own boolean,armedits own boolean. This is the one deliberate divergence from the deployed semantics — record inALIGNMENT.md. (Open choice 3.) - Reactive binding: the row's controls write the store eagerly;
routedOutputmemo (engine-architecture.md§2.1) readsrouting-storeto remap each output viaapplyGroupOverride. The single send-effect pushes to the active backend. A row never holds a second data path.
3.3 Tri-state semantics table (precise)
This matches the deployed per-param override system: off/fixed correspond to a-immersive's frozen (excluded/pinned, aimmersive-clone-spec.md:131), live to the default model-driven path.
| State | Computed by model? | Pin mask (buildPinMask) |
Sent downstream? | Value emitted | UI |
|---|---|---|---|---|---|
| off | No — removed from model control; final-layer weights pinned so RL/train never touch it (findings-feedback-behaviour.md:117) |
pinned (excluded) | yes, at last held value | fixedValue (held/excluded) |
dimmed row, no bar motion |
| fixed (freeze) | computed but result discarded; held at a static value | pinned (protect the held dim) | yes | fixedValue (captured on freeze, draggable, aimmersive-clone-spec.md:131) |
hatched bar overlay (aimmersive-clone-spec.md:190), value slider shown |
| live | Yes — model-driven | not pinned | yes | applyGroupOverride(modelOut, curve, min, max) |
full bar, animates |
Orthogonal modifiers (compose with any state):
| Modifier | Computed? | Visible? | Sent downstream? | Training focus |
|---|---|---|---|---|
| mute | yes (still computed) | yes (bar visible) | no (silenced) | unaffected |
| off (state) | no | yes (dimmed) | yes (held) | excluded |
| solo / arm | yes | yes (focus glow) | yes | this output only (focus + pin mask) |
Precise difference the prompt demands: off = the model no longer drives this output (excluded from learning, held/pinned). fixed/freeze = held at a static value (also pinned, but conceptually "I chose this value", with a draggable
fixedValue). mute = silenced downstream but still computed and visible (distinct from off — you still see it move, you just don't hear it). solo/arm = focus training on this output.
3.4 Drawer depths
- Peek: backend badge (Synth / MIDI / OSC / VCV) + count of live/fixed/off/muted outputs + "Arm all" status.
- Expand: the routing matrix — a scrollable
<For>of<OutputControlRow>s grouped bygroup(collapsible section headers carry a group master curve + mute-group like a-immersive's group drawer,aimmersive-clone-spec.md:137). Each row: name · M · S · [off|fixed|live] · dual-range · curve pad · value. 126 rows for synth — virtualise / collapse non-live by default. - FULL: the Advanced backend modal (§4) — backend selector at the top, then the backend-specific editor over the full output set.
- Backend selector (workstream E): Web Audio (synth) / Web MIDI / OSC bridge / VCV (
engine-architecture.md§1 backends;aimmersive-clone-spec.md:143output-mode tabs). Switching backend may change output count → reuse a-immersive'sconfirm()weight-reset guard (aimmersive-clone-spec.md:144,344).
4. ADVANCED modal — backend-specific layouts (FULL depth)
All backends share the §3.1 baseline (M/S/off-fixed-live/min/max/curve). The FULL modal adds the backend-specific fields. (Workstream E owns the VCV/OSC/MIDI backend internals; this fixes the modal layout + state contract.)
4.1 MIDI backend
Per-output extra fields (ported from a-immersive's MIDI-CC popup rows, aimmersive-clone-spec.md:128):
interface MidiCcSpec {
cc: number; // 0..127, auto-named from CC_NAMES
channel: number; // 1..16
value: number; // last sent, round(v*127)
// min/max/curve from baseline define the 0..127 range mapping
}
- Top of modal: number of CCs (the output count for the MIDI backend =
midiCCMap.length,aimmersive-clone-spec.md:144), output-device<select>, MIDI preset<select>(aimmersive-clone-spec.md:32). - Per-CC row: editable Name (text), CC# (0–127 spinner, auto-renames from
CC_NAMES), Ch (1–16), plus the baseline min/max/curve/state. 7-bit value =round(applyGroupOverride(v,curve,min,max) * 127)(findings-engine-surface.md:36). Persist vianisps-midi-cc-map:<engineId>key (aimmersive-clone-spec.md:239). - Send: batched per block via
WebMidiBackend.send(engine-architecture.md§1;midiOutput.sendBatch,aimmersive-clone-spec.md:258).
4.2 OSC backend
interface OscSpec {
path: string; // e.g. "/synth/cutoff"
min: number; // physical range lo (engineering units, not [0,1])
max: number; // physical range hi
// curve/state from baseline
}
- Top of modal: bridge connection status (WS, deferred contract —
engine-architecture.md§1osc-bridge.ts), add/remove path rows. - Per-output row: OSC path (text), range min/max (typed numeric, physical units — distinct from the [0,1] baseline dual-range; the baseline min/max selects the normalised window, the OSC range maps that window to engineering units), plus baseline curve/state.
- v1: OSC bridge is stubbed behind a locked contract (
engine-architecture.md§1) — the modal renders and persists config but emits only when a bridge is connected.
4.3 VCV backend
interface VcvSpec {
// per-output min/max/freeze ONLY — the simplest backend
// baseline min/max ARE the VCV range; baseline state 'fixed' IS freeze
}
- VCV adds nothing beyond the baseline — per-output min/max + freeze (= baseline
state==='fixed'). The LED-ring palette derives from theme tokens (findings-design-and-manifold.md:30). - Top of modal: per-channel min/max/freeze grid; that is the whole VCV advanced surface.
4.4 Web Audio (synth) — the default
The synth backend's advanced modal is the group-override matrix (§5 / the Powerful Synth Engine drawer's FULL depth): per-group master curve + per-param min/max/curve/mute over the 126 synth params (aimmersive-clone-spec.md:137). No extra per-output struct — the synth output is the baseline.
5. POWERFUL SYNTH ENGINE drawer
Owns the audio engine and its parameter shaping. The string "C15" must never render — the built-in engine is labelled "Powerful Synth Engine" (findings-engine-surface.md:74; CLAUDE.md). Engine display names come from SynthEngine.displayName (aimmersive-clone-spec.md:278).
- Peek: play/pause + master volume + the active engine name ("Powerful Synth Engine") + BPM.
- Expand:
- Engine switcher — Powerful Synth Engine (default) + other engines (PAFSynth, ChannelStrip, etc. from
mode_select); alternative engines flagged where not-yet-wired (findings-engine-surface.md:74). - Audio controls: Vol, BPM (a-immersive play drawer,
aimmersive-clone-spec.md:31). - Arpeggiator controls (
aimmersive-clone-spec.md:278). - Synth preset
<select>— tiered Manual/Beginner/Intermediate/Advanced/Expert (aimmersive-clone-spec.md:199); these set which params are active/muted + their min/max/curve (the synth-side override tier — distinct from the visual/RL example presets, keep separate,aimmersive-clone-spec.md:157).
- Engine switcher — Powerful Synth Engine (default) + other engines (PAFSynth, ChannelStrip, etc. from
- FULL: the synth group-override matrix (§4.4) — 18 collapsible sections (Env A…Mono,
aimmersive-clone-spec.md:298), each with a group master curve (48×48 drag, applies relative delta to all child curves,aimmersive-clone-spec.md:138) and per-param baseline rows (§3.2). This is<GroupOverrideDrawer>promoted to FULL depth. - Bind:
engine.audio.start/stop,SynthEngine.setParam(i, v)throttled ≥50ms / dead-zone >0.002 (aimmersive-clone-spec.md:257,345— load-bearing, prevents ring-buffer flooding). Audio starts lazily on the play gesture (aimmersive-clone-spec.md:278). - COOP/COEP server-scoped (
findings-design-and-manifold.md:100); SharedArrayBuffer only for the browser-only Powerful Synth Engine path (findings-engine-surface.md:74).
6. PARTICLE / VISUAL SYSTEM drawer
Owns the flow-field visualiser — a faithful port of js/ui/visualizer.js (findings-design-and-manifold.md:120 — exact look + behaviour; workstream E). 400-particle Canvas2D flow field driven by 20 named outputs (aimmersive-clone-spec.md:296).
- Peek: visual on/off + the active visual preset name.
- Expand: the 20 visual params as per-output rows (§3 baseline — Flow/Scale/Speed/Hue/Spread/…
aimmersive-clone-spec.md:296), each with itsVISUAL_PARAM_COLORSswatch; plus visual/RL preset chips (Calm/Chaos, Rainbow, Vortex, Spiral, Embers,aimmersive-clone-spec.md:158) which teach the network by clearing the dataset and adding hardcoded input→output examples (distinct from synth tiers — keep separate). - FULL: the full visual param matrix + the output→visual-param range table (verbatim ranges,
aimmersive-clone-spec.md:296) as editable advanced mappings + particle-count / lifetime tuning. - Bind: when the visual backend is active,
routedOutputfans the 20 outputs intoFlowFieldVisualizer.setParams(aimmersive-clone-spec.md:260); the canvas readsengine.routedOutput()in one rAF loop, never inference (engine-architecture.md§2.1).
7. React component tree
The Console wraps everything in <EngineProvider> (engine-architecture.md §2.2). The dock + drawers read EngineApi only; no drawer imports engine internals (lint seam, engine-architecture.md §6).
<ConsoleApp> // focus stage + dock; owns rAF for canvases
├── <Dock> // 48px right rail
│ └── <DockIcon> ×6 // LEARN, IN, OUT, SYNTH, VISUAL, HELP
│ (data-drawer; click → uiStore.setDrawer(depth))
│
├── <DrawerHost> // renders the open drawer at its depth
│ ├── <Drawer depth="peek|expand"> // generic glass panel; ⤢ → FULL
│ │ ├── <LearningBehaviourDrawer>
│ │ │ ├── <FeedbackModeSelector> // pill → engine.feedback.setMode (§1.1)
│ │ │ │ └── <ExploringIndicator> // <NoiseRing> ramp; learning-paused badge
│ │ │ ├── <ArmScopeChooser> // arm-all / arm-output (§1.2)
│ │ │ └── <TrainingParams> // <Slider> ×5/6 (§1.3)
│ │ ├── <InputsDrawer> // workstream F (§6); <ControlAxis> rows
│ │ ├── <RoutingDrawer> // §3
│ │ │ ├── <BackendSelector> // Web Audio/MIDI/OSC/VCV
│ │ │ └── <For each=group>
│ │ │ └── <OutputGroupSection> // header: group master curve + mute-group
│ │ │ └── <For each=output>
│ │ │ └── <OutputControlRow> // ← the shared baseline component (§3.2)
│ │ │ ├── <MuteToggle/> M
│ │ │ ├── <ArmToggle/> S
│ │ │ ├── <TriStateSegmented/> off|fixed|live
│ │ │ ├── <DualRangeSlider/> min/max
│ │ │ ├── <CurvePad/> 36×36
│ │ │ └── <ValueBar/> live | fixedValue
│ │ ├── <SynthEngineDrawer> // §5 ("Powerful Synth Engine" — never "C15")
│ │ │ ├── <EngineSwitcher/>
│ │ │ ├── <AudioControls/> Vol/BPM/Arp
│ │ │ └── <SynthPresetSelect/> // tiered
│ │ └── <VisualSystemDrawer> // §6
│ │ ├── <For each=visualOutput><OutputControlRow/></For>
│ │ └── <VisualPresetChips/>
│ │
│ └── <AdvancedModal depth="full"> // near-fullscreen, --z-modal, glass over dim
│ ├── <FeedbackLab/> // LEARN FULL (state-machine + diagnostics)
│ ├── <InputMatrix/> // IN FULL (workstream F)
│ ├── <BackendAdvanced/> // OUT FULL — switches on active backend:
│ │ ├── <MidiCcEditor/> // CC#/Ch/Name + count (§4.1)
│ │ ├── <OscPathEditor/> // path + physical range (§4.2)
│ │ ├── <VcvChannelEditor/> // min/max/freeze grid (§4.3)
│ │ └── <SynthGroupMatrix/> // 18 sections × params (§4.4 / §5 FULL)
│ └── <VisualMatrix/> // VISUAL FULL
│
├── <VerdictCluster> // +/−/undo/reroll; mirrors <FeedbackModeSelector> pill
└── <HelpModal> // HELP icon opens directly
Shared leaf primitives live in shared/primitives/ (<Slider>, <DualRangeSlider>, <CurvePad>, <NoiseRing>, <LossPlot>, <WeightHealth>, <LayerStats>, <GradientFlow>, <PillToggle>, <ControlAxis> — findings-design-and-manifold.md:60). <OutputControlRow> is the one new composite component this spec introduces, reused across the Routing, Synth, Visual, and (optionally, §6 open choice 2) Inputs drawers.
8. Binding to the reactive spine + engine API (summary)
| UI surface | EngineApi call | Engine route |
|---|---|---|
| Feedback-mode pill | engine.feedback.setMode(m) |
nisps_ml_feedback_set_mode (plans/feedback-modes-port-spec.md §4) |
− verdict |
engine.feedback.thumbsDown() |
dispatch on mode → on_down (findings-feedback-behaviour.md:90) |
+ verdict |
engine.feedback.thumbsUp() |
addExample + train, or CommitStore when exploring |
| Arm (S) | engine.feedback.setFocus(mask) |
nisps_ml_feedback_set_focus + buildPinMask |
| Tri-state off/fixed | routing-store write → weightsRevision/pin recompute |
routedOutput memo + pin mask (engine-architecture.md §2.1) |
| min/max/curve | routing-store write |
routedOutput memo applyGroupOverride (param-map.js:301) |
| mute | routing-store.muted |
send-effect zeroes/holds downstream, value still in mlOutput |
| training sliders | mlStore setters |
live engine params |
| backend select | engine.setBackend(b) |
swaps OutputBackend adapter; output-count guard |
| synth setParam | (internal) send-effect throttle | SynthEngine.setParam ≥50ms/>0.002 |
All writes are eager, synchronous, off-React-render (engine-architecture.md §4) — drawers mutate stores; the single routedOutput→send-effect carries it to the backend; canvases read accessors in rAF. No drawer creates a second data path (the live-feedback guarantee, engine-architecture.md §2.3).
9. Open choices for the operator
- Surface
RandomiseOutputs(third feedback mode)? Spec hides it behind?debug=1("Static roll"). Confirm v1 ships only the two product modes (Push away / Explore & place) or all three. - Do input-axis rows reuse
<OutputControlRow>? They share min/max/curve but not mute/off/live semantics. Spec defers to workstream F; confirm whether to unify the component or fork it. - Separate
state/muted/armedfields vs the deployed conflatedfrozen↔mutedsingle field. Spec deliberately splits them (cleaner tri-state); this is a divergence from the deployed override system to record inALIGNMENT.md. Confirm. - Default backend output counts on switch trigger a
confirm()weight-reset guard (ported from a-immersive). Confirm you want that friction, or prefer silent warm-start (createWithWarmStart,aimmersive-clone-spec.md:250). - Group master curve relative-delta behaviour (drag the group curve nudges all child curves preserving offsets,
aimmersive-clone-spec.md:138) — keep this a-immersive behaviour, or switch to absolute group curve? Spec keeps relative. - Where does the feedback-mode pill live for live play — only mirrored next to the Verdict cluster, or also a permanent sub-
<StatusLine>pill (aimmersive-clone-spec.md:283)? Spec puts it on the Verdict cluster.
Verification corrections (adversarial pass, 2026-06-27) — verdict: minor-issues
The design reasoning, drawer model, tri-state/mute/arm separation and per-output baseline are sound; most
aimmersive-clone-spec.md citations verified line-by-line. Apply these fixes in the build:
engine.feedback.setMode/setFocus/exploring()are not yet a JS surface — only the C ABI (nisps_ml_feedback_set_mode/set_focus/exploring) exists. The ManifoldEngineApi.feedbackmust ADD these JS wrappers over the C ABI in Phase 3 (this is intended new surface, not a mistake — but it must be built, not assumed). Likewiseengine.setBackendand synthsetParamare internal, not on EngineApi today.param-map.js(applyTame/applyCurve/applyGroupOverride) lives only indeployments/meml-aimmersive, not inplayground/src. The curve/override math must be ported intomanifold/(see backends-specmapping.ts), not imported from the deployed snapshot.--dock-widthand min-thumb#4488ffare deployed-a-immersive values, absent from the Manifold tokens — use Manifold tokens (manifold-export/tokens/) or add the missing ones deliberately.- A few
findings-feedback-behaviour.md:NNcitations are out of range / point atplans/feedback-modes-port-spec.mdinstead (learning_paused()=feedback.hpp:85/ port-spec:238;exploring()=feedback.hpp:84). - The React
<EngineProvider>/useEnginepattern is infindings-design-and-manifold.md §4, notengine-architecture.md §2.2(that doc is SolidJS).