2026-06-28 22:20:58 +02:00
# Manifold UI — Agent Onboarding
> **Purpose of this doc:** everything an agent needs to make a tweak or fix to the Manifold
> front-end without re-grepping the tree. Read this + the relevant section's source files; you
> should not need to read the 40KB design specs for routine work (they're pointers at the end).
>
> **Voice:** ground-truth map. If something here contradicts the code, the code wins — fix this doc
> in the same commit.
---
## 1. What Manifold is
A **convertible-mode React front-end** on the real, parity-tested NISPS ML+audio engine (same
`nisps/` C++ core that the firmware and the `playground/` use, compiled to WASM). It lets you drive
a neural net's mapping from a small input space (XY pad / gamepad / MIDI) to many outputs, shape
that mapping with interactive ML feedback ("explore & place" / dislike), and route the outputs to
several backends (built-in synth, MIDI, OSC, particles, VCV, MEMLNaut hardware).
- **Stack:** Vite + React 18 + TypeScript (strict). No CSS-in-JS — design-token CSS variables.
- **Lives at:** `manifold/` (beside `playground/` ). Deploys to **https://meml.lnfinitemonkeys.org/next/** .
- **Synth is labelled "Powerful Synth Engine" / "Built-in Synth" — NEVER "C15".** British spelling.
---
## 2. Run / build / deploy / test
```bash
cd manifold
bun install
bun run dev # Vite dev server, port 5273 (COOP/COEP headers set)
bun run typecheck # tsc --noEmit ← run before every commit
bun run build # tsc --noEmit & & vite build → dist/
bun run preview # serve dist/ on :4273 (honours COOP/COEP — needed by WASM/worklet)
bun run test:e2e # Playwright smoke (needs `bun run build` first; runs against preview)
```
- **Typecheck is the cheap gate.** It catches most regressions; run it after any edit.
- **E2E on the VPS** needs a non-snap node runner (bun is snap-confined and hides libs from
Chromium): `PLAYWRIGHT_BROWSERS_PATH=/home/w1n5t0n/snap/bun-js/87/.cache/ms-playwright node node_modules/.bin/playwright test` . Preview via bun is fine. The smoke spec (`tests/e2e/smoke.spec.ts`)
asserts: engine WASM loads, spine invariant (setInputs → outputs change), feedback runs, console
docs: the specs disposition pass (plan §8)
Roughly 20k lines were deleted from this repo in the last week and much of the
corpus still described the pre-deletion world in the present tense. Executes
the §8 table: archive the retired, reclassify the executed, prune the stale.
aimmersive-clone-spec -> _archive/ with a deprecated-by note
feedback-modes-port-spec -> plans/, kind: plan, status: executed
manifold-parity-features -> plans/, kind: plan, status: active
playground-2.0-rewrite -> status: superseded
engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten
present-tense against the shipped engine/
MAIN.md six contradicted claims fixed; registry resynced
vcv-module.md pruned to the current 8->16 contract and made the
single .nisps format spec
vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads
vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table
inputs/backends/dock trio grounding sections marked historical, dead cites fixed
Two rows of the §8 table were themselves wrong, corrected here: the deleted
full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no
§6.3), and codegen/README.md was already a MAP pointer with no port-solidjs
trigger left to remove.
Beyond the table — found by sweeping every backticked path in the changed docs
against `git ls-files`, which is how these should have been caught before:
manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current:
SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a
shot.spec.ts that does not exist. The whole stage table was keyed on a `focus`
axis that no longer exists — selection is now sandwich > particles >
composite. This matters more than the rest: CLAUDE.md tells every agent to
read ONBOARDING.md first for Manifold work, so it was actively teaching a
fiction. Rewritten against ConsoleApp.tsx.
MAP.md claimed the input layer reduces axes to the engine arity with an
"even/odd blend". input-layer.ts says the opposite in its own header: one
dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and
mean-blending was removed deliberately because it diluted every source.
AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed),
per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at
nisps::FixedBuffer (deleted).
Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/
Delta and cachedNovelty behind a live context-menu toggle that nothing reads;
vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's
origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to
9000, so out of the box they do not meet.
Firmware-build docs are deliberately untouched — the PlatformIO migration
lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00
renders, **no "C15" in the bundle** , no console errors.
ci: restore verification — reachable submodule pin, codegen + WASM freshness gates
Phase 0 of the 2026-07 simplification audit (plan §1). CI has been 100% red on
main since 2026-07-13 and every "gates green" claim since rested on local runs.
- S7 / critic gap 2: push memllib `feat/nisps-core-swap` (3 commits incl. the
pin b37fc53) to monkey-w1n5t0n/memllib and repoint .gitmodules at the fork.
Those commits existed on exactly one disk; `git ls-remote` now resolves the
pin, so `submodules: recursive` checkout and fresh clones work again. Drops
the compensating unreachable-pin error paragraph in build-firmware-arch.sh.
- S24 / S31: the manifold-tests job regenerates from schemas/, runs the codegen
golden test, and fails on a dirty diff — the "schema changes ship with both
generated outputs" rule is now enforced rather than assumed.
- S32: a WASM freshness gate runs the parity harness against the *committed*
manifold/public/nisps.{js,wasm} before the CI rebuild overwrites it. That
artifact is what the webhook ships to production, so a stale commit now fails
loudly instead of shipping.
- critic gap 3 / operator decision §7.4: the VPS webhook
(~/.config/webhooks/meml-deploy.sh, not in this repo) waits for the `CI`
workflow to conclude success on the pushed SHA before building. Fail-closed;
MEML_SKIP_CI_GATE=1 for an emergency hand-deploy. Verified the gate query
returns `failure` for fa37047, i.e. it would have blocked that deploy.
- S31: corrected run-all-tests.sh's false "single command CI invokes" header.
Docs moved with the code: ALIGNMENT defect 1 deleted (resolved) and the rest
renumbered; MAP.md's unreachable-pin warning replaced with the fork pin and a
pointer to the §7.5 vendoring decision; ONBOARDING documents the deploy gate
and the tracked-WASM-ships-to-prod hazard; plan §1 marked burned down.
Gates: scripts/run-all-tests.sh ALL GREEN (ctest 4/4, parity 1273 floats within
1e-5, lint, typecheck, 33 Playwright specs).
2026-07-21 11:57:32 +02:00
- **Deploy is automatic on push to GitHub `main` , but gated on CI** → webhook → waits for the
`CI` workflow to conclude `success` on that exact SHA → builds `manifold/` → rsyncs to the live
`/next/` subdir. A red or missing CI run aborts the deploy (fail-closed, 20 min timeout);
`MEML_SKIP_CI_GATE=1` bypasses it for an emergency hand-deploy. The gate lives VPS-side in
`~/.config/webhooks/meml-deploy.sh` (not in this repo) — added 2026-07-21 per the simplification
audit. See the `manifold-deploy-pipeline` memory for the full chain and gotchas (the
`cp index.html a-immersive.html` 403 workaround; git-ignored `bun.lock` ).
- **`manifold/public/nisps.{js,wasm}` are tracked artifacts that ship to production** — the webhook
builds only `manifold/` , so vite copies whatever is committed. CI's *WASM freshness gate* runs
the parity harness against the committed artifact before rebuilding it, so a stale commit fails
loudly. Rebuild with `scripts/build-wasm.sh` and commit it whenever `nisps/` changes.
2026-06-28 22:20:58 +02:00
- **`?debug=1`** installs `window.__nisps` (synchronous probe for Playwright/console — see
`src/debug/probe.ts` ). ** `base: './'` ** in `vite.config.ts` keeps asset URLs relative so one
`dist/` mounts at both `/` and `/next/` . **WASM URLs must resolve via `document.baseURI`** , never
hardcoded (see the assetUrl gotcha in §6).
---
## 3. The big picture — three layers
```
┌─────────────────────────────────────────────────────────────────┐
│ UI layer (React) src/console/ src/dock/ src/primitives/
│ ConsoleApp = spine of the UI: holds state, picks a Stage, │
│ renders the Dock. "Convertible" = swappable Stages. │
└───────────────┬─────────────────────────────────────────────────-┘
│ reads engine.version (useSyncExternalStore), reads buffers imperatively
│ writes via engine.setInput / setParam / feedback.*
┌───────────────▼─────────────────────────────────────────────────┐
│ Engine layer (NO React) src/engine/ src/inputs/ src/feedback/ src/backends/
│ Spine = reactive store below React. Per-frame ML inference is │
│ eager + synchronous, OFF the render cycle. React only watches │
│ a monotonic version counter. │
└───────────────┬─────────────────────────────────────────────────┘
│ C ABI
┌───────────────▼─────────────────────────────────────────────────┐
│ WASM (nisps.wasm) — built from the C++ nisps/ core │
│ Loaded TWICE: main thread (ML inference + training + feedback) │
│ and AudioWorklet (audio DSP, separate instance). │
└──────────────────────────────────────────────────────────────────┘
```
**Golden rule of the reactivity model:** components subscribe to `engine.version()` via
`useEngineVersion()` and then read live `Float32Array` s **imperatively** (`engine.getOutputs()`,
`engine.routedOutput()` ). They do **not** get outputs as React state — that was a deliberate perf
decision. Buffers are reused frame-to-frame; never assume a fresh array.
---
## 4. UI layer (where most tweaks happen)
### Entry & composition
- `src/App.tsx` → mounts `<EngineProvider>` (async WASM load, shows `Loading` ) → `<ConsoleApp focus="composite" />` . Installs the `?debug=1` probe once the engine is live.
- `src/console/ConsoleApp.tsx` (**~1040 lines — the UI spine**). Holds nearly all UI state:
`focus` , `modeId` , `params` (`MFParam[]`), `pos` (2D input), feedback state
(`feedbackMode`/`soloMode`/`exploring`/`picking`/`anchorCount`), output config
(`outputMode`/`midiOutputId`/`oscUrl`/`vcvUrl`), and UI flags (`sandwich`, `split` , `stripPinned` ,
`snapshots` , `markers` , `health` , `rev` ). Builds the flat `ConsoleCtx` passed to the Dock + drawers.
docs: the specs disposition pass (plan §8)
Roughly 20k lines were deleted from this repo in the last week and much of the
corpus still described the pre-deletion world in the present tense. Executes
the §8 table: archive the retired, reclassify the executed, prune the stale.
aimmersive-clone-spec -> _archive/ with a deprecated-by note
feedback-modes-port-spec -> plans/, kind: plan, status: executed
manifold-parity-features -> plans/, kind: plan, status: active
playground-2.0-rewrite -> status: superseded
engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten
present-tense against the shipped engine/
MAIN.md six contradicted claims fixed; registry resynced
vcv-module.md pruned to the current 8->16 contract and made the
single .nisps format spec
vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads
vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table
inputs/backends/dock trio grounding sections marked historical, dead cites fixed
Two rows of the §8 table were themselves wrong, corrected here: the deleted
full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no
§6.3), and codegen/README.md was already a MAP pointer with no port-solidjs
trigger left to remove.
Beyond the table — found by sweeping every backticked path in the changed docs
against `git ls-files`, which is how these should have been caught before:
manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current:
SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a
shot.spec.ts that does not exist. The whole stage table was keyed on a `focus`
axis that no longer exists — selection is now sandwich > particles >
composite. This matters more than the rest: CLAUDE.md tells every agent to
read ONBOARDING.md first for Manifold work, so it was actively teaching a
fiction. Rewritten against ConsoleApp.tsx.
MAP.md claimed the input layer reduces axes to the engine arity with an
"even/odd blend". input-layer.ts says the opposite in its own header: one
dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and
mean-blending was removed deliberately because it diluted every source.
AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed),
per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at
nisps::FixedBuffer (deleted).
Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/
Delta and cachedNovelty behind a live context-menu toggle that nothing reads;
vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's
origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to
9000, so out of the box they do not meet.
Firmware-build docs are deliberately untouched — the PlatformIO migration
lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00
### The Stages (one renders at a time)
**There is no `focus` axis any more.** The focus/altitude system (`AltitudeNav`, `SplitStage` ,
`ReadoutStrip` , `InputMini` , `CompactAxis` ) was deleted in the 2026-07 simplification audit —
Manifold ships a single "composite" altitude. Selection is now a plain three-way in `ConsoleApp` :
`sandwich` wins, else `outputMode==='particles'` , else CompositeStage.
2026-06-28 22:20:58 +02:00
| Stage | File | Renders when | What it is |
|---|---|---|---|
docs: the specs disposition pass (plan §8)
Roughly 20k lines were deleted from this repo in the last week and much of the
corpus still described the pre-deletion world in the present tense. Executes
the §8 table: archive the retired, reclassify the executed, prune the stale.
aimmersive-clone-spec -> _archive/ with a deprecated-by note
feedback-modes-port-spec -> plans/, kind: plan, status: executed
manifold-parity-features -> plans/, kind: plan, status: active
playground-2.0-rewrite -> status: superseded
engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten
present-tense against the shipped engine/
MAIN.md six contradicted claims fixed; registry resynced
vcv-module.md pruned to the current 8->16 contract and made the
single .nisps format spec
vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads
vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table
inputs/backends/dock trio grounding sections marked historical, dead cites fixed
Two rows of the §8 table were themselves wrong, corrected here: the deleted
full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no
§6.3), and codegen/README.md was already a MAP pointer with no port-solidjs
trigger left to remove.
Beyond the table — found by sweeping every backticked path in the changed docs
against `git ls-files`, which is how these should have been caught before:
manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current:
SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a
shot.spec.ts that does not exist. The whole stage table was keyed on a `focus`
axis that no longer exists — selection is now sandwich > particles >
composite. This matters more than the rest: CLAUDE.md tells every agent to
read ONBOARDING.md first for Manifold work, so it was actively teaching a
fiction. Rewritten against ConsoleApp.tsx.
MAP.md claimed the input layer reduces axes to the engine arity with an
"even/odd blend". input-layer.ts says the opposite in its own header: one
dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and
mean-blending was removed deliberately because it diluted every source.
AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed),
per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at
nisps::FixedBuffer (deleted).
Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/
Delta and cachedNovelty behind a live context-menu toggle that nothing reads;
vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's
origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to
9000, so out of the box they do not meet.
Firmware-build docs are deliberately untouched — the PlatformIO migration
lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00
| CompositeStage | `CompositeStage.tsx` | **default / hero** | Draggable split-ratio; magnet-snaps to 0.14/0.33/0.5/0.66/0.86; collapses a side to a corner minimap at extremes. |
| SandwichStage | `SandwichStage.tsx` | `sandwich===true` (wins over the others) | Three-pane layout: `Manifold` input surface left, 3D parameter-landscape centre (input → MLP heatmap grid → outputs, drag to orbit), compact `OutputStage` right. |
2026-07-25 15:30:35 +02:00
| ParticleStage | `ParticleStage.tsx` | `outputMode==='particles'` | Flow-field visualiser (`flow-field.ts`, 400-particle Canvas2D port) + interactive output heatmap sliders with cursor tooltips + corner joystick. |
2026-06-28 22:20:58 +02:00
docs: the specs disposition pass (plan §8)
Roughly 20k lines were deleted from this repo in the last week and much of the
corpus still described the pre-deletion world in the present tense. Executes
the §8 table: archive the retired, reclassify the executed, prune the stale.
aimmersive-clone-spec -> _archive/ with a deprecated-by note
feedback-modes-port-spec -> plans/, kind: plan, status: executed
manifold-parity-features -> plans/, kind: plan, status: active
playground-2.0-rewrite -> status: superseded
engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten
present-tense against the shipped engine/
MAIN.md six contradicted claims fixed; registry resynced
vcv-module.md pruned to the current 8->16 contract and made the
single .nisps format spec
vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads
vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table
inputs/backends/dock trio grounding sections marked historical, dead cites fixed
Two rows of the §8 table were themselves wrong, corrected here: the deleted
full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no
§6.3), and codegen/README.md was already a MAP pointer with no port-solidjs
trigger left to remove.
Beyond the table — found by sweeping every backticked path in the changed docs
against `git ls-files`, which is how these should have been caught before:
manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current:
SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a
shot.spec.ts that does not exist. The whole stage table was keyed on a `focus`
axis that no longer exists — selection is now sandwich > particles >
composite. This matters more than the rest: CLAUDE.md tells every agent to
read ONBOARDING.md first for Manifold work, so it was actively teaching a
fiction. Rewritten against ConsoleApp.tsx.
MAP.md claimed the input layer reduces axes to the engine arity with an
"even/odd blend". input-layer.ts says the opposite in its own header: one
dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and
mean-blending was removed deliberately because it diluted every source.
AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed),
per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at
nisps::FixedBuffer (deleted).
Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/
Delta and cachedNovelty behind a live context-menu toggle that nothing reads;
vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's
origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to
9000, so out of the box they do not meet.
Firmware-build docs are deliberately untouched — the PlatformIO migration
lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00
`Manifold.tsx` and `OutputStage.tsx` are no longer top-level stages — they are panes composed by
CompositeStage/SandwichStage. `Manifold.tsx` is the full-bleed 2D input surface (canvas trail + pins
+ feedback markers; pointer → `onMove` ; **double-click the input mark → follow-mouse mode** , a window
`pointermove` listener mapping the whole viewport onto this surface's space, Esc or a second
2026-07-25 15:16:01 +02:00
double-click exits). `OutputStage.tsx` is the output columns; click or drag a bar to temporarily
2026-07-25 15:20:11 +02:00
audition a clamped value (the next actual input change restores the live MLP), and it takes a `compact` prop
2026-07-25 15:16:01 +02:00
for the narrow pane.
docs: the specs disposition pass (plan §8)
Roughly 20k lines were deleted from this repo in the last week and much of the
corpus still described the pre-deletion world in the present tense. Executes
the §8 table: archive the retired, reclassify the executed, prune the stale.
aimmersive-clone-spec -> _archive/ with a deprecated-by note
feedback-modes-port-spec -> plans/, kind: plan, status: executed
manifold-parity-features -> plans/, kind: plan, status: active
playground-2.0-rewrite -> status: superseded
engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten
present-tense against the shipped engine/
MAIN.md six contradicted claims fixed; registry resynced
vcv-module.md pruned to the current 8->16 contract and made the
single .nisps format spec
vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads
vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table
inputs/backends/dock trio grounding sections marked historical, dead cites fixed
Two rows of the §8 table were themselves wrong, corrected here: the deleted
full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no
§6.3), and codegen/README.md was already a MAP pointer with no port-solidjs
trigger left to remove.
Beyond the table — found by sweeping every backticked path in the changed docs
against `git ls-files`, which is how these should have been caught before:
manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current:
SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a
shot.spec.ts that does not exist. The whole stage table was keyed on a `focus`
axis that no longer exists — selection is now sandwich > particles >
composite. This matters more than the rest: CLAUDE.md tells every agent to
read ONBOARDING.md first for Manifold work, so it was actively teaching a
fiction. Rewritten against ConsoleApp.tsx.
MAP.md claimed the input layer reduces axes to the engine arity with an
"even/odd blend". input-layer.ts says the opposite in its own header: one
dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and
mean-blending was removed deliberately because it diluted every source.
AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed),
per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at
nisps::FixedBuffer (deleted).
Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/
Delta and cachedNovelty behind a live context-menu toggle that nothing reads;
vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's
origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to
9000, so out of the box they do not meet.
Firmware-build docs are deliberately untouched — the PlatformIO migration
lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00
2026-06-28 22:20:58 +02:00
- **Output modes** (the TOP dock selector, NOT the same axis as `focus` ): `src/console/output-mode.ts`
2026-07-25 13:55:19 +02:00
defines `OUTPUT_MODES` = **particles** (default) / midi / osc / cv / synth / editor, each mapping to a
2026-07-25 15:16:37 +02:00
`BackendId` . `DEFAULT_OUTPUT_MODE='particles'` . `outputDisplayCount()` is the shared active-card
boundary for the stage, backend context, and routing rows. MIDI starts with eight cards; every
backend can add a card or delete any individual card in condensed and expanded Outputs drawers.
The `N outputs` chip always reports that same set. `MFParam.id` is semantic identity; array position
2026-07-25 15:28:26 +02:00
is not; backend-specific display names (including Particle System names) resolve from that identity,
so deleting a middle card cannot transfer its name or settings to the next card. Settings decides
whether edits retain spare network capacity or keep exact arity.
2026-06-28 22:20:58 +02:00
- `src/console/output-mode.ts` , `types.ts` , `model.ts` are the shared vocabulary — read these first
when touching anything cross-cutting:
- `types.ts` : `Focus` , `OutputMode` , `DrawerKey` , `DrawerDepth` , `FeedbackModeUI` , `SoloMode` ,
`Pin` , `FeedbackMarker` , `Snapshot` , and ** `ConsoleCtx` ** (the flat context handed to the dock).
feat(manifold): P5.2/P5.3 — derive MF_MODES from schema truth + per-mode engine dims
Schema-backed modes in console/model.ts are now DERIVED from the codegen
schemas in src/modes/generated/ (source of truth): real param names/groups/
count, plus each mode's ml net shape (MFMode.ml) and schema engine_id. A thin
manifold OVERLAY supplies only label/glyph/ModeClass/input/ordering. New
browser-viable modes xiasri + slp_workshop get derived entries; schema-less
visualizer + c15 stay hand-written on DEFAULT_MODE_ML. Schema min/max/default/
label/curve surface as engine-unit metadata (schemaMin/... on MFParam) without
touching the 0..1 routing semantics.
Switching instrument mode reshapes the runtime-shaped WASM net to the mode's
schema ml config (ConsoleApp effect keyed on [engine, modeId]; no confirm
modal). Boot lands paf_synth dims (4->[10,10,14]->33) once WASM is ready. The
P2.3 axis-count reshape offer still reads the engine's live inputSize and does
not spuriously prompt on a mode switch.
Adds schema-modes.spec.ts (P5 gate): drives switches via a new window.__mf
debug seam and asserts describe() dims, getWeights count, output length/bounds,
and UI param count FROM the imported schemas; spot-checks trainAsync after a
switch. Updates reshape/probe-api/geo-dislike specs to assert from the boot
mode schema instead of the retired fixed 32/126 shape.
All gates green: typecheck, unit (9), build, e2e (33).
2026-07-18 12:45:06 +02:00
- `model.ts` : the instrument catalogue `MF_MODES` , `MFParam` , `ParamStatus`
2026-06-28 22:20:58 +02:00
(`off|fixed|live`), `ParamGroup` , plus `shapeValues()` (applies min/max/curve to raw engine
feat: curve truth, DriverConfig, real telemetry, engine benchmark
Four items from one workflow, committed together because their build and CI
wiring genuinely interleaves — nisps/CMakeLists.txt, run-all-tests.sh and
ci.yml each carry hunks from two of them, and the stage renumbering (1/5 ->
1/6) touches every line. Splitting would produce commits that do not build,
which is worse than a commit that does four things and says so.
S26 part 2 — the curve declaration now matches reality. params[].curve stays
the mode-wide DEFAULT; a voice_spaces entry may now be {name, curve_overrides}
declaring only the slots where THAT voice space deviates. The 6 modes with one
voice space are byte-identical. The values were derived MECHANICALLY by a new
codegen/curve-audit.ts that models the four idioms a p[N]*p[N] regex misses
(alias form, memlcelium's implicit-counter sq() lambda, loop-generated indices,
smooth_params_), inlines helpers, and RAISES rather than guessing when it
cannot reduce an expression. A drift gate cross-checks 1179 (voice space x
param) slots against engine source on every run and was proved to fail loudly
on three drift classes. Application stays in the engine: nisps/engines,
nisps/pipeline and nisps/core are untouched, generated output is pure insertion
(755 insertions, 0 deletions), and the rebuilt nisps.wasm was byte-identical.
S4 / 7.2 — firmware reads the active mode's driver config at mode start, and
mic/line is real. My brief assumed the engine owns this; the code disagreed and
the code was right. sound_analysis_midi's EngineT is NoOpEngine — the mic lives
on a separately-composed AnalysisEngine member — so engine-level wiring would
have compiled, passed every gate, and left the one mic mode on line input.
Hence a mode-level seam defaulting to engine().driver_config(). Separately,
DriverConfig's defaults (line_level 0, output_volume 1.0) had drifted from
memllib's actual 3/0.8 because nothing had ever read them; wiring them as-is
would have made every silent mode louder and its line input maximally
insensitive — a behaviour change disguised as plumbing. Now pinned by a test.
Also: GetSysClockSpeed() panic()s on unsupported sample rates and runs on the
first line of setup(), so sample_rate needed a fallback ahead of clock setup.
CI's firmware env list gains soundanalysismidi — it is the only mic variant and
nothing else compiles that path.
Plan 5e — telemetry is real. A loss_history C-API entry across the full 5-layer
chain lets the browser read the per-iteration loss the core already records.
The audit named one fabrication site; there were two — wasm-iml.ts's
synchronous train() published lossHistory: [loss] as well. A third, ctx.loss,
was not merely dead but actively synthetic (fallbacks of prev * 0.82 and a
literal 0.5, rendered by nothing) and is deleted. The firmware buffer stays
untouched, per the L25 call. EngineApi.lossHistory() reads spine state rather
than the MLP handle, because trainAsync() fits on the worker's mirror net and
the handle would give a subtly-wrong second answer.
Plan 5f — engine throughput is measurable. One source compiled twice (CMake
natively, emcc for WASM) so the targets compare directly and no WASM export is
added. Sequencers are driven into a working state, and every row prints its own
working-state evidence so a number produced by an idle engine is visible rather
than plausible. Reports, never asserts: a wall-clock threshold on shared
hardware is meaningless or flaky, same call as the firmware size job.
ALIGNMENT: the telemetry defect is deleted (built, not deferred); the
performance defect is rewritten to what is actually left — these are HOST
numbers, and nothing measures the RP2350 at 150 MHz, which is the target the
mission's constraint is about. Q4 (memllib ownership) and Q5 (legacy feedback
modes) are closed.
Corrections to my own earlier claims, both found by agents contradicting the
brief: manifold/ONBOARDING.md was NOT "now accurate" — its primitives list
still named five deleted primitives and cited a seededGradient() that does not
exist. And the parity harness misses the sequencer engines because it runs 128
frames while their sequencers evaluate every 400-500 samples, NOT because
all-params-0.5 fails to trigger them (it does trigger: 0.5 maps to ratio 2,
firing three times per bar). The fix is a longer window, not different params.
Gates: run-all-tests.sh ALL GREEN — 4/4 ctest, parity PASS, lint clean, curve
drift 1179 slots ok, 39 e2e (was 33). Firmware: 5 envs built including the mic
variant.
2026-07-21 22:02:23 +02:00
outputs) and `modeEngineId()` . (`seededGradient()` is GONE — it was the fabricated
gradient-health source, deleted in the 2026-07 sweep, S16.) **Schema-backed modes are DERIVED from the codegen
feat(manifold): P5.2/P5.3 — derive MF_MODES from schema truth + per-mode engine dims
Schema-backed modes in console/model.ts are now DERIVED from the codegen
schemas in src/modes/generated/ (source of truth): real param names/groups/
count, plus each mode's ml net shape (MFMode.ml) and schema engine_id. A thin
manifold OVERLAY supplies only label/glyph/ModeClass/input/ordering. New
browser-viable modes xiasri + slp_workshop get derived entries; schema-less
visualizer + c15 stay hand-written on DEFAULT_MODE_ML. Schema min/max/default/
label/curve surface as engine-unit metadata (schemaMin/... on MFParam) without
touching the 0..1 routing semantics.
Switching instrument mode reshapes the runtime-shaped WASM net to the mode's
schema ml config (ConsoleApp effect keyed on [engine, modeId]; no confirm
modal). Boot lands paf_synth dims (4->[10,10,14]->33) once WASM is ready. The
P2.3 axis-count reshape offer still reads the engine's live inputSize and does
not spuriously prompt on a mode switch.
Adds schema-modes.spec.ts (P5 gate): drives switches via a new window.__mf
debug seam and asserts describe() dims, getWeights count, output length/bounds,
and UI param count FROM the imported schemas; spot-checks trainAsync after a
switch. Updates reshape/probe-api/geo-dislike specs to assert from the boot
mode schema instead of the retired fixed 32/126 shape.
All gates green: typecheck, unit (9), build, e2e (33).
2026-07-18 12:45:06 +02:00
schemas in `src/modes/generated/` ** (one-core-engine P5.2) — real param names/groups/count,
plus each mode's `ml` net shape (`MFMode.ml`) and schema `engine_id` (`MFMode.engineId`)
come from schema truth. A thin manifold OVERLAY (`SCHEMA_MODES` in model.ts) supplies only
label/glyph/ModeClass/input/ordering. Two schema-less manifold-only modes (`visualizer`,
`c15` placeholder) stay hand-written on `DEFAULT_MODE_ML` . Do NOT hand-edit
`src/modes/generated/` — it is codegen output (`bun run codegen/generate.ts`).
2026-06-28 22:20:58 +02:00
### The Dock (right-edge rail) — `src/console/Dock.tsx` + `Drawers.tsx`
- 48px right rail: **TOP** = mode selector (the 5 output modes, popover); **MIDDLE** = 5 drawer
icons, vertically centred macOS-dock style; **BOTTOM** = sandwich toggle.
- Five drawers (`DRAWERS` in `Drawers.tsx` , each has `.render(ctx, depth)` — condensed 360px panel
vs expanded 80vw×
2026-07-25 15:24:50 +02:00
- **learn** — feedback mode (explore-and-place / geometric-dislike) + solo mode + per-output arm,
plus a live model-architecture strip (inputs/outputs highlighted; hidden layers expand on click).
feat: curve truth, DriverConfig, real telemetry, engine benchmark
Four items from one workflow, committed together because their build and CI
wiring genuinely interleaves — nisps/CMakeLists.txt, run-all-tests.sh and
ci.yml each carry hunks from two of them, and the stage renumbering (1/5 ->
1/6) touches every line. Splitting would produce commits that do not build,
which is worse than a commit that does four things and says so.
S26 part 2 — the curve declaration now matches reality. params[].curve stays
the mode-wide DEFAULT; a voice_spaces entry may now be {name, curve_overrides}
declaring only the slots where THAT voice space deviates. The 6 modes with one
voice space are byte-identical. The values were derived MECHANICALLY by a new
codegen/curve-audit.ts that models the four idioms a p[N]*p[N] regex misses
(alias form, memlcelium's implicit-counter sq() lambda, loop-generated indices,
smooth_params_), inlines helpers, and RAISES rather than guessing when it
cannot reduce an expression. A drift gate cross-checks 1179 (voice space x
param) slots against engine source on every run and was proved to fail loudly
on three drift classes. Application stays in the engine: nisps/engines,
nisps/pipeline and nisps/core are untouched, generated output is pure insertion
(755 insertions, 0 deletions), and the rebuilt nisps.wasm was byte-identical.
S4 / 7.2 — firmware reads the active mode's driver config at mode start, and
mic/line is real. My brief assumed the engine owns this; the code disagreed and
the code was right. sound_analysis_midi's EngineT is NoOpEngine — the mic lives
on a separately-composed AnalysisEngine member — so engine-level wiring would
have compiled, passed every gate, and left the one mic mode on line input.
Hence a mode-level seam defaulting to engine().driver_config(). Separately,
DriverConfig's defaults (line_level 0, output_volume 1.0) had drifted from
memllib's actual 3/0.8 because nothing had ever read them; wiring them as-is
would have made every silent mode louder and its line input maximally
insensitive — a behaviour change disguised as plumbing. Now pinned by a test.
Also: GetSysClockSpeed() panic()s on unsupported sample rates and runs on the
first line of setup(), so sample_rate needed a fallback ahead of clock setup.
CI's firmware env list gains soundanalysismidi — it is the only mic variant and
nothing else compiles that path.
Plan 5e — telemetry is real. A loss_history C-API entry across the full 5-layer
chain lets the browser read the per-iteration loss the core already records.
The audit named one fabrication site; there were two — wasm-iml.ts's
synchronous train() published lossHistory: [loss] as well. A third, ctx.loss,
was not merely dead but actively synthetic (fallbacks of prev * 0.82 and a
literal 0.5, rendered by nothing) and is deleted. The firmware buffer stays
untouched, per the L25 call. EngineApi.lossHistory() reads spine state rather
than the MLP handle, because trainAsync() fits on the worker's mirror net and
the handle would give a subtly-wrong second answer.
Plan 5f — engine throughput is measurable. One source compiled twice (CMake
natively, emcc for WASM) so the targets compare directly and no WASM export is
added. Sequencers are driven into a working state, and every row prints its own
working-state evidence so a number produced by an idle engine is visible rather
than plausible. Reports, never asserts: a wall-clock threshold on shared
hardware is meaningless or flaky, same call as the firmware size job.
ALIGNMENT: the telemetry defect is deleted (built, not deferred); the
performance defect is rewritten to what is actually left — these are HOST
numbers, and nothing measures the RP2350 at 150 MHz, which is the target the
mission's constraint is about. Q4 (memllib ownership) and Q5 (legacy feedback
modes) are closed.
Corrections to my own earlier claims, both found by agents contradicting the
brief: manifold/ONBOARDING.md was NOT "now accurate" — its primitives list
still named five deleted primitives and cited a seededGradient() that does not
exist. And the parity harness misses the sequencer engines because it runs 128
frames while their sequencers evaluate every 400-500 samples, NOT because
all-params-0.5 fails to trigger them (it does trigger: 0.5 maps to ratio 2,
firing three times per bar). The fix is a longer window, not different params.
Gates: run-all-tests.sh ALL GREEN — 4/4 ctest, parity PASS, lint clean, curve
drift 1179 slots ok, 39 e2e (was 33). Firmware: 5 envs built including the mic
variant.
2026-07-21 22:02:23 +02:00
At `expanded` depth ONLY it also renders `TrainingHealth.tsx` : the real per-iteration loss
curve (`EngineApi.lossHistory()` ← `nisps_ml_loss_history` ← `MLPCore::loss_history` ) plus
the per-layer weight-health table (`getLayerStats`). ** `depth === 'expanded'` is Manifold's
advanced-surface flag** — there is no separate feature-flag mechanism, so put advanced
surface there rather than inventing one. The panel renders "no training run yet" when the
core has no history; it never synthesises a curve.
2026-06-28 22:20:58 +02:00
- **inputs** — enable/configure input sources (XY pad / MIDI / gamepad).
2026-07-25 15:24:50 +02:00
- **route** (label "Outputs") — per-output control matrix + per-backend config; at expanded depth
the output rows fill the remaining drawer height and scroll independently.
2026-06-28 22:20:58 +02:00
- **settings** — icon style (monochrome/colour), input-map shape (xy/joystick/rect/circular), corner radius.
- **help** — keymap pills + loop explanation.
- `src/dock/` holds the output-routing internals used by the `route` drawer:
- `output-state.ts` — the per-output control model: `OutputControl` (state/muted/armed/min/max/
curve/fixedValue + backend specs `MidiCcSpec` /`OscSpec`/`VcvSpec`), `toOutputControl()` ,
`buildArmMask()` (solo focus).
- `OutputControlRow.tsx` — one output row: name · M(mute) · S(solo/arm) · off|fixed|live · dual-range
· curve pad · live value. **Writes eagerly to the shared `MFParam` store via `onChange`.**
- `OutputsBackendConfig.tsx` — preset bar (save/restore/rename/delete) + per-backend config (MIDI
CC#/channel, OSC path/range, VCV polarity).
### Primitives — `src/primitives/` (barrel: `index.ts`)
feat: curve truth, DriverConfig, real telemetry, engine benchmark
Four items from one workflow, committed together because their build and CI
wiring genuinely interleaves — nisps/CMakeLists.txt, run-all-tests.sh and
ci.yml each carry hunks from two of them, and the stage renumbering (1/5 ->
1/6) touches every line. Splitting would produce commits that do not build,
which is worse than a commit that does four things and says so.
S26 part 2 — the curve declaration now matches reality. params[].curve stays
the mode-wide DEFAULT; a voice_spaces entry may now be {name, curve_overrides}
declaring only the slots where THAT voice space deviates. The 6 modes with one
voice space are byte-identical. The values were derived MECHANICALLY by a new
codegen/curve-audit.ts that models the four idioms a p[N]*p[N] regex misses
(alias form, memlcelium's implicit-counter sq() lambda, loop-generated indices,
smooth_params_), inlines helpers, and RAISES rather than guessing when it
cannot reduce an expression. A drift gate cross-checks 1179 (voice space x
param) slots against engine source on every run and was proved to fail loudly
on three drift classes. Application stays in the engine: nisps/engines,
nisps/pipeline and nisps/core are untouched, generated output is pure insertion
(755 insertions, 0 deletions), and the rebuilt nisps.wasm was byte-identical.
S4 / 7.2 — firmware reads the active mode's driver config at mode start, and
mic/line is real. My brief assumed the engine owns this; the code disagreed and
the code was right. sound_analysis_midi's EngineT is NoOpEngine — the mic lives
on a separately-composed AnalysisEngine member — so engine-level wiring would
have compiled, passed every gate, and left the one mic mode on line input.
Hence a mode-level seam defaulting to engine().driver_config(). Separately,
DriverConfig's defaults (line_level 0, output_volume 1.0) had drifted from
memllib's actual 3/0.8 because nothing had ever read them; wiring them as-is
would have made every silent mode louder and its line input maximally
insensitive — a behaviour change disguised as plumbing. Now pinned by a test.
Also: GetSysClockSpeed() panic()s on unsupported sample rates and runs on the
first line of setup(), so sample_rate needed a fallback ahead of clock setup.
CI's firmware env list gains soundanalysismidi — it is the only mic variant and
nothing else compiles that path.
Plan 5e — telemetry is real. A loss_history C-API entry across the full 5-layer
chain lets the browser read the per-iteration loss the core already records.
The audit named one fabrication site; there were two — wasm-iml.ts's
synchronous train() published lossHistory: [loss] as well. A third, ctx.loss,
was not merely dead but actively synthetic (fallbacks of prev * 0.82 and a
literal 0.5, rendered by nothing) and is deleted. The firmware buffer stays
untouched, per the L25 call. EngineApi.lossHistory() reads spine state rather
than the MLP handle, because trainAsync() fits on the worker's mirror net and
the handle would give a subtly-wrong second answer.
Plan 5f — engine throughput is measurable. One source compiled twice (CMake
natively, emcc for WASM) so the targets compare directly and no WASM export is
added. Sequencers are driven into a working state, and every row prints its own
working-state evidence so a number produced by an idle engine is visible rather
than plausible. Reports, never asserts: a wall-clock threshold on shared
hardware is meaningless or flaky, same call as the firmware size job.
ALIGNMENT: the telemetry defect is deleted (built, not deferred); the
performance defect is rewritten to what is actually left — these are HOST
numbers, and nothing measures the RP2350 at 150 MHz, which is the target the
mission's constraint is about. Q4 (memllib ownership) and Q5 (legacy feedback
modes) are closed.
Corrections to my own earlier claims, both found by agents contradicting the
brief: manifold/ONBOARDING.md was NOT "now accurate" — its primitives list
still named five deleted primitives and cited a seededGradient() that does not
exist. And the parity harness misses the sequencer engines because it runs 128
frames while their sequencers evaluate every 400-500 samples, NOT because
all-params-0.5 fails to trigger them (it does trigger: 0.5 maps to ratio 2,
firing three times per bar). The fix is a longer window, not different params.
Gates: run-all-tests.sh ALL GREEN — 4/4 ctest, parity PASS, lint clean, curve
drift 1179 slots ok, 39 e2e (was 33). Firmware: 5 envs built including the mic
variant.
2026-07-21 22:02:23 +02:00
Seven: `Button` , `Slider` , `PillToggle` , `Badge` , `Switch` , `XYPad` , `VirtualJoystick` . Dumb,
reusable, no engine knowledge. Side-effect import of `styles/primitives.css` styles the range
inputs. `Panel` /`StatusLine`/`ControlAxis`/`CurvePlot`/`Sparkline` were **deleted** in the 2026-07
sweep (L22, zero consumers) — don't cite them.
2026-06-28 22:20:58 +02:00
### Other shared UI files
docs: the specs disposition pass (plan §8)
Roughly 20k lines were deleted from this repo in the last week and much of the
corpus still described the pre-deletion world in the present tense. Executes
the §8 table: archive the retired, reclassify the executed, prune the stale.
aimmersive-clone-spec -> _archive/ with a deprecated-by note
feedback-modes-port-spec -> plans/, kind: plan, status: executed
manifold-parity-features -> plans/, kind: plan, status: active
playground-2.0-rewrite -> status: superseded
engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten
present-tense against the shipped engine/
MAIN.md six contradicted claims fixed; registry resynced
vcv-module.md pruned to the current 8->16 contract and made the
single .nisps format spec
vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads
vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table
inputs/backends/dock trio grounding sections marked historical, dead cites fixed
Two rows of the §8 table were themselves wrong, corrected here: the deleted
full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no
§6.3), and codegen/README.md was already a MAP pointer with no port-solidjs
trigger left to remove.
Beyond the table — found by sweeping every backticked path in the changed docs
against `git ls-files`, which is how these should have been caught before:
manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current:
SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a
shot.spec.ts that does not exist. The whole stage table was keyed on a `focus`
axis that no longer exists — selection is now sandwich > particles >
composite. This matters more than the rest: CLAUDE.md tells every agent to
read ONBOARDING.md first for Manifold work, so it was actively teaching a
fiction. Rewritten against ConsoleApp.tsx.
MAP.md claimed the input layer reduces axes to the engine arity with an
"even/odd blend". input-layer.ts says the opposite in its own header: one
dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and
mean-blending was removed deliberately because it diluted every source.
AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed),
per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at
nisps::FixedBuffer (deleted).
Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/
Delta and cachedNovelty behind a live context-menu toggle that nothing reads;
vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's
origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to
9000, so out of the box they do not meet.
Firmware-build docs are deliberately untouched — the PlatformIO migration
lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00
- `shared-ui.tsx` — `MiniMeters` (read-only output bars). `AltitudeNav` /`CompactAxis` were deleted with the focus system.
2026-06-28 22:20:58 +02:00
- `icons.tsx` — monochrome inline-SVG icons (mode icons + drawer icons + `GLYPH_FALLBACK` for when monochrome is off).
- `VerdictCluster.tsx` — floating bottom-centre feedback UI (perturb ▽ / undo ↺ / commit △ + A/B); labels adapt to feedback mode.
- `CurvePad.tsx` — square canvas curve editor (vertical drag reshapes [0,1]; ~0.43 ≈ linear). Used in OutputEditor + OutputControlRow.
2026-07-25 15:05:32 +02:00
- `OutputEditor.tsx` — inline range/curve popup for a single output (hover/click on a bar), using
`DualRange.tsx` for the shared dual-thumb min/max control.
2026-06-28 22:20:58 +02:00
### Styling — `src/styles/`
CSS-variable design tokens, no CSS-in-JS. `tokens.css` `@import` s `tokens/{base,colors,fonts,
typography,spacing,effects}.css`. All vars on `:root` , consumed via `var(--name)` . Dark theme: deep
black bg, warm-orange `--accent` (#ff6a00), cyan `--accent-2` (#00ccff). Corner radius is driven by
a setting → `--r-*` tokens.
---
## 5. Engine / inputs / feedback / backends (touch when behaviour, not chrome, changes)
### Engine — `src/engine/` (no React except the two binding files)
- `spine.ts` — **the reactive store.** `setInput(x,y)` /`setInputs(arr)` drive raw input synchronously
feat(manifold)!: route input/output pipelines + curves through the WASM core (P4.3/P4.4)
One-core-engine P4.3/P4.4: the input/output pipeline processing and the curve
catalog now live in the C++/WASM core (nisps/pipeline/*, nisps/core/math.hpp).
The TS ports are deleted and the browser drives the WASM chains.
Engine:
- WasmIML owns a nisps_pipeline_create handle + bridge buffers and exposes
setInputConfig (TS InputConfig → 15-float wire), processInput, resetInput,
setOutputConfig (Infinity slew → 0), setOutputFreezeMask, processOutput
(in place), resetOutput, curveApply, curveApplyBatch (chunked). Handle +
buffers created in init_, freed in dispose, output-sized buffers realloc'd
on reshape.
- Spine routes setInputs through iml.processInput/processOutput (state lives
C++-side); config source-of-truth stays TS-side and is pushed on attach /
setInputConfig / setOutputConfig. Preserves ?debug=1 fixed-dt determinism
(same dt fed to the WASM calls). EngineApi gains setInputConfig/
setOutputConfig/curveApply/curveApplyBatch.
- New types-only modules: pipeline-types.ts (InputConfig/OutputConfig +
defaults + wire int mappers) and curve-catalog.ts (CurveName + name→id).
types.ts declares the pipeline/curve C ABI. engine barrel updated.
- DELETED src/engine/{input-pipeline,output-pipeline,curves}.ts.
Tests (P4.4 gate — recorded-gesture regression):
- pipeline-golden.test.ts now loads the built WASM (indirect-eval shim,
tests/wasm-load.ts) and drives the frozen gesture/output fixtures through the
C++ chains, honouring the per-event dt clock contract. Tolerance 1e-5
(non-momentum drift <5e-7). The 3 momentum configs carry 1e-2: proven-inherent
f32 drift (a byte-faithful f32 port of the exact original algorithm matches
the WASM to <6e-8 while both diverge from the f64 capture by ~7-9e-3), NOT a
core bug.
- curves-golden.json: linear/square/sqrt/centered_power kept as the original
f64 captures (C++ matches within <3e-8); exp/log/sigmoid/cubic RE-BASELINED
from the WASM (deliberate switch to firmware-exact k=1 exp/log, slope-6
sigmoid, true cubic x^3). Provenance recorded in-file.
- _generate.ts rebuilt as the WASM curve re-baseline tool; pipeline-golden-lib
trimmed to pure data builders.
Docs: fixtures/README.md + manifold/ONBOARDING.md updated.
Gates: typecheck, bun test (9), vite build, playwright e2e (27) all green.
2026-07-18 12:21:35 +02:00
through `WasmIML.processInput` (WASM input chain) → `processInto()` → `WasmIML.processOutput` (WASM
output chain) → backend, all off the render cycle, reusing buffers. Bumps a monotonic `version` .
`reprocess()` re-ticks the last input after a weight change (stores the full N-D vector so extra axes
survive). Pipeline config lives in `inputConfig_` /`outputConfig_` and is pushed C-side via
`setInputConfig` /`setOutputConfig` (state itself lives in the WASM pipeline handle since P4).
2026-06-28 22:20:58 +02:00
- `engine-api.ts` — ** `EngineApi` , the framework-neutral facade** everything in the UI talks to:
`setInput/setInputs` , `getOutputs/routedOutput` , training (`addExample/train/trainAsync/evalLoss`),
feat: curve truth, DriverConfig, real telemetry, engine benchmark
Four items from one workflow, committed together because their build and CI
wiring genuinely interleaves — nisps/CMakeLists.txt, run-all-tests.sh and
ci.yml each carry hunks from two of them, and the stage renumbering (1/5 ->
1/6) touches every line. Splitting would produce commits that do not build,
which is worse than a commit that does four things and says so.
S26 part 2 — the curve declaration now matches reality. params[].curve stays
the mode-wide DEFAULT; a voice_spaces entry may now be {name, curve_overrides}
declaring only the slots where THAT voice space deviates. The 6 modes with one
voice space are byte-identical. The values were derived MECHANICALLY by a new
codegen/curve-audit.ts that models the four idioms a p[N]*p[N] regex misses
(alias form, memlcelium's implicit-counter sq() lambda, loop-generated indices,
smooth_params_), inlines helpers, and RAISES rather than guessing when it
cannot reduce an expression. A drift gate cross-checks 1179 (voice space x
param) slots against engine source on every run and was proved to fail loudly
on three drift classes. Application stays in the engine: nisps/engines,
nisps/pipeline and nisps/core are untouched, generated output is pure insertion
(755 insertions, 0 deletions), and the rebuilt nisps.wasm was byte-identical.
S4 / 7.2 — firmware reads the active mode's driver config at mode start, and
mic/line is real. My brief assumed the engine owns this; the code disagreed and
the code was right. sound_analysis_midi's EngineT is NoOpEngine — the mic lives
on a separately-composed AnalysisEngine member — so engine-level wiring would
have compiled, passed every gate, and left the one mic mode on line input.
Hence a mode-level seam defaulting to engine().driver_config(). Separately,
DriverConfig's defaults (line_level 0, output_volume 1.0) had drifted from
memllib's actual 3/0.8 because nothing had ever read them; wiring them as-is
would have made every silent mode louder and its line input maximally
insensitive — a behaviour change disguised as plumbing. Now pinned by a test.
Also: GetSysClockSpeed() panic()s on unsupported sample rates and runs on the
first line of setup(), so sample_rate needed a fallback ahead of clock setup.
CI's firmware env list gains soundanalysismidi — it is the only mic variant and
nothing else compiles that path.
Plan 5e — telemetry is real. A loss_history C-API entry across the full 5-layer
chain lets the browser read the per-iteration loss the core already records.
The audit named one fabrication site; there were two — wasm-iml.ts's
synchronous train() published lossHistory: [loss] as well. A third, ctx.loss,
was not merely dead but actively synthetic (fallbacks of prev * 0.82 and a
literal 0.5, rendered by nothing) and is deleted. The firmware buffer stays
untouched, per the L25 call. EngineApi.lossHistory() reads spine state rather
than the MLP handle, because trainAsync() fits on the worker's mirror net and
the handle would give a subtly-wrong second answer.
Plan 5f — engine throughput is measurable. One source compiled twice (CMake
natively, emcc for WASM) so the targets compare directly and no WASM export is
added. Sequencers are driven into a working state, and every row prints its own
working-state evidence so a number produced by an idle engine is visible rather
than plausible. Reports, never asserts: a wall-clock threshold on shared
hardware is meaningless or flaky, same call as the firmware size job.
ALIGNMENT: the telemetry defect is deleted (built, not deferred); the
performance defect is rewritten to what is actually left — these are HOST
numbers, and nothing measures the RP2350 at 150 MHz, which is the target the
mission's constraint is about. Q4 (memllib ownership) and Q5 (legacy feedback
modes) are closed.
Corrections to my own earlier claims, both found by agents contradicting the
brief: manifold/ONBOARDING.md was NOT "now accurate" — its primitives list
still named five deleted primitives and cited a seededGradient() that does not
exist. And the parity harness misses the sequencer engines because it runs 128
frames while their sequencers evaluate every 400-500 samples, NOT because
all-params-0.5 fails to trigger them (it does trigger: 0.5 maps to ratio 2,
firing three times per bar). The fix is a longer window, not different params.
Gates: run-all-tests.sh ALL GREEN — 4/4 ctest, parity PASS, lint clean, curve
drift 1179 slots ok, 39 e2e (was 33). Firmware: 5 envs built including the mic
variant.
2026-07-21 22:02:23 +02:00
weights (`getWeights/setWeights/process/randomise`), telemetry
2026-07-25 15:24:50 +02:00
(`architecture/weightCount/exampleCount/lossHistory/getLayerStats`), `subscribe/version/on` , plus nested `.feedback` and `.audio`
feat: curve truth, DriverConfig, real telemetry, engine benchmark
Four items from one workflow, committed together because their build and CI
wiring genuinely interleaves — nisps/CMakeLists.txt, run-all-tests.sh and
ci.yml each carry hunks from two of them, and the stage renumbering (1/5 ->
1/6) touches every line. Splitting would produce commits that do not build,
which is worse than a commit that does four things and says so.
S26 part 2 — the curve declaration now matches reality. params[].curve stays
the mode-wide DEFAULT; a voice_spaces entry may now be {name, curve_overrides}
declaring only the slots where THAT voice space deviates. The 6 modes with one
voice space are byte-identical. The values were derived MECHANICALLY by a new
codegen/curve-audit.ts that models the four idioms a p[N]*p[N] regex misses
(alias form, memlcelium's implicit-counter sq() lambda, loop-generated indices,
smooth_params_), inlines helpers, and RAISES rather than guessing when it
cannot reduce an expression. A drift gate cross-checks 1179 (voice space x
param) slots against engine source on every run and was proved to fail loudly
on three drift classes. Application stays in the engine: nisps/engines,
nisps/pipeline and nisps/core are untouched, generated output is pure insertion
(755 insertions, 0 deletions), and the rebuilt nisps.wasm was byte-identical.
S4 / 7.2 — firmware reads the active mode's driver config at mode start, and
mic/line is real. My brief assumed the engine owns this; the code disagreed and
the code was right. sound_analysis_midi's EngineT is NoOpEngine — the mic lives
on a separately-composed AnalysisEngine member — so engine-level wiring would
have compiled, passed every gate, and left the one mic mode on line input.
Hence a mode-level seam defaulting to engine().driver_config(). Separately,
DriverConfig's defaults (line_level 0, output_volume 1.0) had drifted from
memllib's actual 3/0.8 because nothing had ever read them; wiring them as-is
would have made every silent mode louder and its line input maximally
insensitive — a behaviour change disguised as plumbing. Now pinned by a test.
Also: GetSysClockSpeed() panic()s on unsupported sample rates and runs on the
first line of setup(), so sample_rate needed a fallback ahead of clock setup.
CI's firmware env list gains soundanalysismidi — it is the only mic variant and
nothing else compiles that path.
Plan 5e — telemetry is real. A loss_history C-API entry across the full 5-layer
chain lets the browser read the per-iteration loss the core already records.
The audit named one fabrication site; there were two — wasm-iml.ts's
synchronous train() published lossHistory: [loss] as well. A third, ctx.loss,
was not merely dead but actively synthetic (fallbacks of prev * 0.82 and a
literal 0.5, rendered by nothing) and is deleted. The firmware buffer stays
untouched, per the L25 call. EngineApi.lossHistory() reads spine state rather
than the MLP handle, because trainAsync() fits on the worker's mirror net and
the handle would give a subtly-wrong second answer.
Plan 5f — engine throughput is measurable. One source compiled twice (CMake
natively, emcc for WASM) so the targets compare directly and no WASM export is
added. Sequencers are driven into a working state, and every row prints its own
working-state evidence so a number produced by an idle engine is visible rather
than plausible. Reports, never asserts: a wall-clock threshold on shared
hardware is meaningless or flaky, same call as the firmware size job.
ALIGNMENT: the telemetry defect is deleted (built, not deferred); the
performance defect is rewritten to what is actually left — these are HOST
numbers, and nothing measures the RP2350 at 150 MHz, which is the target the
mission's constraint is about. Q4 (memllib ownership) and Q5 (legacy feedback
modes) are closed.
Corrections to my own earlier claims, both found by agents contradicting the
brief: manifold/ONBOARDING.md was NOT "now accurate" — its primitives list
still named five deleted primitives and cited a seededGradient() that does not
exist. And the parity harness misses the sequencer engines because it runs 128
frames while their sequencers evaluate every 400-500 samples, NOT because
all-params-0.5 fails to trigger them (it does trigger: 0.5 maps to ratio 2,
firing three times per bar). The fix is a longer window, not different params.
Gates: run-all-tests.sh ALL GREEN — 4/4 ctest, parity PASS, lint clean, curve
drift 1179 slots ok, 39 e2e (was 33). Firmware: 5 envs built including the mic
variant.
2026-07-21 22:02:23 +02:00
facades. ** `lossHistory()` reads SPINE STATE, not the MLP handle** — an async train runs on the
worker's mirror net, so the main handle's own history is empty for those runs; both paths
publish to the spine.
2026-06-28 22:20:58 +02:00
- `EngineProvider.tsx` / `useEngine.ts` — the **only** React coupling. `useEngine()` returns the API
(null until WASM ready); `useEngineVersion()` = `useSyncExternalStore(subscribe, version)` .
- `engine-host.ts` — main-thread audio wiring: AudioContext (user-gesture gated), fetch `nisps.wasm` ,
register + feed the worklet.
- `wasm-iml.ts` (**~750 lines**) — the ML interface to `nisps.wasm` : one MLP handle, dataset, heap
buffers, feedback C-ABI bindings (`nisps_ml_feedback_*`), lazy training worker.
feat: curve truth, DriverConfig, real telemetry, engine benchmark
Four items from one workflow, committed together because their build and CI
wiring genuinely interleaves — nisps/CMakeLists.txt, run-all-tests.sh and
ci.yml each carry hunks from two of them, and the stage renumbering (1/5 ->
1/6) touches every line. Splitting would produce commits that do not build,
which is worse than a commit that does four things and says so.
S26 part 2 — the curve declaration now matches reality. params[].curve stays
the mode-wide DEFAULT; a voice_spaces entry may now be {name, curve_overrides}
declaring only the slots where THAT voice space deviates. The 6 modes with one
voice space are byte-identical. The values were derived MECHANICALLY by a new
codegen/curve-audit.ts that models the four idioms a p[N]*p[N] regex misses
(alias form, memlcelium's implicit-counter sq() lambda, loop-generated indices,
smooth_params_), inlines helpers, and RAISES rather than guessing when it
cannot reduce an expression. A drift gate cross-checks 1179 (voice space x
param) slots against engine source on every run and was proved to fail loudly
on three drift classes. Application stays in the engine: nisps/engines,
nisps/pipeline and nisps/core are untouched, generated output is pure insertion
(755 insertions, 0 deletions), and the rebuilt nisps.wasm was byte-identical.
S4 / 7.2 — firmware reads the active mode's driver config at mode start, and
mic/line is real. My brief assumed the engine owns this; the code disagreed and
the code was right. sound_analysis_midi's EngineT is NoOpEngine — the mic lives
on a separately-composed AnalysisEngine member — so engine-level wiring would
have compiled, passed every gate, and left the one mic mode on line input.
Hence a mode-level seam defaulting to engine().driver_config(). Separately,
DriverConfig's defaults (line_level 0, output_volume 1.0) had drifted from
memllib's actual 3/0.8 because nothing had ever read them; wiring them as-is
would have made every silent mode louder and its line input maximally
insensitive — a behaviour change disguised as plumbing. Now pinned by a test.
Also: GetSysClockSpeed() panic()s on unsupported sample rates and runs on the
first line of setup(), so sample_rate needed a fallback ahead of clock setup.
CI's firmware env list gains soundanalysismidi — it is the only mic variant and
nothing else compiles that path.
Plan 5e — telemetry is real. A loss_history C-API entry across the full 5-layer
chain lets the browser read the per-iteration loss the core already records.
The audit named one fabrication site; there were two — wasm-iml.ts's
synchronous train() published lossHistory: [loss] as well. A third, ctx.loss,
was not merely dead but actively synthetic (fallbacks of prev * 0.82 and a
literal 0.5, rendered by nothing) and is deleted. The firmware buffer stays
untouched, per the L25 call. EngineApi.lossHistory() reads spine state rather
than the MLP handle, because trainAsync() fits on the worker's mirror net and
the handle would give a subtly-wrong second answer.
Plan 5f — engine throughput is measurable. One source compiled twice (CMake
natively, emcc for WASM) so the targets compare directly and no WASM export is
added. Sequencers are driven into a working state, and every row prints its own
working-state evidence so a number produced by an idle engine is visible rather
than plausible. Reports, never asserts: a wall-clock threshold on shared
hardware is meaningless or flaky, same call as the firmware size job.
ALIGNMENT: the telemetry defect is deleted (built, not deferred); the
performance defect is rewritten to what is actually left — these are HOST
numbers, and nothing measures the RP2350 at 150 MHz, which is the target the
mission's constraint is about. Q4 (memllib ownership) and Q5 (legacy feedback
modes) are closed.
Corrections to my own earlier claims, both found by agents contradicting the
brief: manifold/ONBOARDING.md was NOT "now accurate" — its primitives list
still named five deleted primitives and cited a seededGradient() that does not
exist. And the parity harness misses the sequencer engines because it runs 128
frames while their sequencers evaluate every 400-500 samples, NOT because
all-params-0.5 fails to trigger them (it does trigger: 0.5 maps to ratio 2,
firing three times per bar). The fix is a longer window, not different params.
Gates: run-all-tests.sh ALL GREEN — 4/4 ctest, parity PASS, lint clean, curve
drift 1179 slots ok, 39 e2e (was 33). Firmware: 5 envs built including the mic
variant.
2026-07-21 22:02:23 +02:00
- `wasm-worker.ts` — off-thread training worker; returns weights, final loss, and the real
per-iteration loss curve read off its own mirror handle. `worklet/nisps-processor.ts` — the AudioWorklet's
2026-06-28 22:20:58 +02:00
separate WASM instance (raw `WebAssembly.instantiate` , no Emscripten glue; 128-sample blocks).
feat(manifold)!: route input/output pipelines + curves through the WASM core (P4.3/P4.4)
One-core-engine P4.3/P4.4: the input/output pipeline processing and the curve
catalog now live in the C++/WASM core (nisps/pipeline/*, nisps/core/math.hpp).
The TS ports are deleted and the browser drives the WASM chains.
Engine:
- WasmIML owns a nisps_pipeline_create handle + bridge buffers and exposes
setInputConfig (TS InputConfig → 15-float wire), processInput, resetInput,
setOutputConfig (Infinity slew → 0), setOutputFreezeMask, processOutput
(in place), resetOutput, curveApply, curveApplyBatch (chunked). Handle +
buffers created in init_, freed in dispose, output-sized buffers realloc'd
on reshape.
- Spine routes setInputs through iml.processInput/processOutput (state lives
C++-side); config source-of-truth stays TS-side and is pushed on attach /
setInputConfig / setOutputConfig. Preserves ?debug=1 fixed-dt determinism
(same dt fed to the WASM calls). EngineApi gains setInputConfig/
setOutputConfig/curveApply/curveApplyBatch.
- New types-only modules: pipeline-types.ts (InputConfig/OutputConfig +
defaults + wire int mappers) and curve-catalog.ts (CurveName + name→id).
types.ts declares the pipeline/curve C ABI. engine barrel updated.
- DELETED src/engine/{input-pipeline,output-pipeline,curves}.ts.
Tests (P4.4 gate — recorded-gesture regression):
- pipeline-golden.test.ts now loads the built WASM (indirect-eval shim,
tests/wasm-load.ts) and drives the frozen gesture/output fixtures through the
C++ chains, honouring the per-event dt clock contract. Tolerance 1e-5
(non-momentum drift <5e-7). The 3 momentum configs carry 1e-2: proven-inherent
f32 drift (a byte-faithful f32 port of the exact original algorithm matches
the WASM to <6e-8 while both diverge from the f64 capture by ~7-9e-3), NOT a
core bug.
- curves-golden.json: linear/square/sqrt/centered_power kept as the original
f64 captures (C++ matches within <3e-8); exp/log/sigmoid/cubic RE-BASELINED
from the WASM (deliberate switch to firmware-exact k=1 exp/log, slope-6
sigmoid, true cubic x^3). Provenance recorded in-file.
- _generate.ts rebuilt as the WASM curve re-baseline tool; pipeline-golden-lib
trimmed to pure data builders.
Docs: fixtures/README.md + manifold/ONBOARDING.md updated.
Gates: typecheck, bun test (9), vite build, playwright e2e (27) all green.
2026-07-18 12:21:35 +02:00
- **Input/output pipelines + curves live in the C++/WASM core (one-core-engine P4).** The input chain
(invert → deadzone → circular clamp → momentum-modulated zoom → centred power → EMA → momentum) and
output chain (global curve → per-output EMA → slew → freeze/mask) are `nisps/pipeline/*` , exposed via
`nisps_input_*` / `nisps_output_*` and driven by thin `WasmIML` wrappers (`processInput`,
`processOutput` , `setInputConfig` , `setOutputConfig` , `setOutputFreezeMask` , `reset*` ). State lives
C++-side per pipeline handle. First 2 axes get the full pad pipeline; axes 2+ feed raw to the spine.
The old TS `input-pipeline.ts` / `output-pipeline.ts` / `curves.ts` are **deleted** ; config TYPES are
`pipeline-types.ts` , the curve NAME↔id contract is `curve-catalog.ts` , and the curve MATHS is sampled
from the core via `EngineApi.curveApply` / `curveApplyBatch` .
2026-06-28 22:20:58 +02:00
- `dataset.ts` — JS-side example store + sample-weight modes (uniform/recency/spatial/combined).
feat(manifold)!: route input/output pipelines + curves through the WASM core (P4.3/P4.4)
One-core-engine P4.3/P4.4: the input/output pipeline processing and the curve
catalog now live in the C++/WASM core (nisps/pipeline/*, nisps/core/math.hpp).
The TS ports are deleted and the browser drives the WASM chains.
Engine:
- WasmIML owns a nisps_pipeline_create handle + bridge buffers and exposes
setInputConfig (TS InputConfig → 15-float wire), processInput, resetInput,
setOutputConfig (Infinity slew → 0), setOutputFreezeMask, processOutput
(in place), resetOutput, curveApply, curveApplyBatch (chunked). Handle +
buffers created in init_, freed in dispose, output-sized buffers realloc'd
on reshape.
- Spine routes setInputs through iml.processInput/processOutput (state lives
C++-side); config source-of-truth stays TS-side and is pushed on attach /
setInputConfig / setOutputConfig. Preserves ?debug=1 fixed-dt determinism
(same dt fed to the WASM calls). EngineApi gains setInputConfig/
setOutputConfig/curveApply/curveApplyBatch.
- New types-only modules: pipeline-types.ts (InputConfig/OutputConfig +
defaults + wire int mappers) and curve-catalog.ts (CurveName + name→id).
types.ts declares the pipeline/curve C ABI. engine barrel updated.
- DELETED src/engine/{input-pipeline,output-pipeline,curves}.ts.
Tests (P4.4 gate — recorded-gesture regression):
- pipeline-golden.test.ts now loads the built WASM (indirect-eval shim,
tests/wasm-load.ts) and drives the frozen gesture/output fixtures through the
C++ chains, honouring the per-event dt clock contract. Tolerance 1e-5
(non-momentum drift <5e-7). The 3 momentum configs carry 1e-2: proven-inherent
f32 drift (a byte-faithful f32 port of the exact original algorithm matches
the WASM to <6e-8 while both diverge from the f64 capture by ~7-9e-3), NOT a
core bug.
- curves-golden.json: linear/square/sqrt/centered_power kept as the original
f64 captures (C++ matches within <3e-8); exp/log/sigmoid/cubic RE-BASELINED
from the WASM (deliberate switch to firmware-exact k=1 exp/log, slope-6
sigmoid, true cubic x^3). Provenance recorded in-file.
- _generate.ts rebuilt as the WASM curve re-baseline tool; pipeline-golden-lib
trimmed to pure data builders.
Docs: fixtures/README.md + manifold/ONBOARDING.md updated.
Gates: typecheck, bun test (9), vite build, playwright e2e (27) all green.
2026-07-18 12:21:35 +02:00
`sink.ts` — `EngineSink` framework boundary. `types.ts` — C-ABI surface types.
2026-07-13 23:30:28 +02:00
- `exploration.ts` — `ExplorationController` adapter for the Jolt press + OU explore gestures
feat(manifold): geometric dislike + jolt/OU via shared core (one-core-engine P3)
TS half of P3.1/P3.2/P3.3: wire the new WASM exports and delete the TS
approximations now that geometric-dislike, jolt, OU, and the seeded RNG live
in the C++/WASM core.
- types.ts/wasm-iml.ts: bind + wrap dislike_geometric, store_positive,
positive/negative_count, set_avoid_style, jolt_press/step/release/active/
lr_scale/tick_lr_ramp, explore_intensity/get/apply. Weight-mutating wrappers
republish weights (version bump); explore_apply reuses feedbackBuf.
- engine-api: extend .feedback (dislikeGeometric/storePositive/counts/
setAvoidStyle) + new .explore facade. thumbsDown now passes the HEARD
(routed) vector, not the raw MLP output (raw == net output => inert cold-start).
- feedback/controller.ts: delete dislikes[] + applyDislikeBias + both C++ GAP
blocks + the SeededRng field; dislike() -> core dislikeGeometric (returns
action, 15 => cold-start prompt); like() feeds the centroid via the core
thumbsUp; getState() exposes positive/negative counts. Delete feedback/rng.ts.
- engine/exploration.ts: execute the P3 SWAP POINT -> drive engine.explore.*;
delete engine/jolt.ts + ou-explore.ts. UI surface unchanged.
- ConsoleApp: one-time cold-start banner (British spelling), routed heard vector
at the dislike call site.
- App.tsx/spine.ts: under ?debug=1 pin a fixed seed + fixed per-tick dt so the
probe/e2e are deterministic (production keeps time-seeded, real-time dt).
- tests: new geo-dislike.spec.ts; probe gains dislikeGeometric/storePositive/
feedbackCounts/setAvoidStyle; the thumbsDown probe test now drives a real
distinct-heard-vector dislike (bare thumbsDown on the net's own output is
correctly inert under the geometric core). 27 e2e + 9 unit green.
Docs: manifold/ONBOARDING.md engine+feedback sections synced.
2026-07-14 04:54:27 +02:00
(Learning drawer). **As of one-core-engine P3 the maths lives in the shared C++/WASM core** — this
class is a thin driver that owns only the control-rate timers and calls `engine.explore.*`
(`joltPress`/`joltStep`/`joltRelease`/`joltActive` → `nisps_ml_jolt_*` ; `setExploreIntensity` /
`exploreApply` → `nisps_ml_explore_*` ). The held ~200 Hz driver calls `joltStep()` + `process()` ;
the OU walk is the spine's inert-by-default `setOutputMorph` hook, now `exploreApply(routed)` (copy
into the WASM heap → advance+add in the core → copy back). The interim `jolt.ts` / `ou-explore.ts`
TS ports were deleted with the swap.
2026-06-28 22:20:58 +02:00
### Inputs — `src/inputs/`
- `input-layer.ts` — composition hub. One rAF loop polls sources, pulls all axes into a vector,
forwards N→engine. ** `MAX_AXES = 32` ** (WASM net over-provisioned to 32 inputs). **Dedicated
dimensions, NO mean-blending** — each active axis drives its own engine slot 1:1; unused slots
2026-07-25 15:16:37 +02:00
zero-padded (inert). Changing the layout uses the same persisted identity-aware I/O policy as output
cards.
2026-06-28 22:20:58 +02:00
- `base-source.ts` + sources: `xy-pad-source.ts` (push, 2 axes), `gamepad-source.ts` (single=2 /
double=4 axes, deadzone 0.08), `midi-input-source.ts` (Web MIDI, batch CC-learn, multi-port).
- `useInputLayer.ts` — React binding; manages exclusive input mode + gamepad stick mode + MIDI
device/learn map; exposes `pushPad` , `sources` , `channelLayout` , etc.
2026-07-25 15:16:37 +02:00
- **I/O migration (P2.3, live):** the net is **runtime-shaped** . It boots at the default
feat(manifold): runtime-shaped net reshape with confirm modal (P2.3)
Wire the runtime-shaped WASM MLP (one-core-engine P2) through the manifold:
- WasmIML.reshape(dims, spread): calls nisps_ml_reshape, re-describes the
instance, reallocates every dim-dependent heap buffer, refreshes weightCount,
clears the TS Dataset mirror (C-side resets), drops the stale training worker,
and pushes the new shape through the sink.
- EngineApi.reshape exposes it and re-ticks the spine so outputs/audio reflect
the new net. Spine already tolerates the arity change (buffers resize, version
bumps); documented.
- Training worker protocol carries the current hidden dims; the worker
ensureNet()s its mirror net to match after a reshape.
- ConsoleApp offers the reshape behind ReshapeModal on an active-layout CHANGE
(never on load; default 32-input over-provisioned head + zero-padding
preserved when declined). British copy, reset-on-reshape.
- Drawers: delete the stale even/odd blending note; honest dedicated-dimensions
line + net-arity chip.
- Probe: __nisps.reshape(nIn) / .describe(); e2e reshape.spec (default 32/126,
reshape to 4, describe reports 4, bounded outputs, weight count 3148→2868,
spine still propagates). All 25 e2e pass (20 existing + 5 new).
- ONBOARDING: refresh the reshape status + stale hardwired-arity gotcha.
2026-07-14 03:54:12 +02:00
over-provisioned 32-input head (zero-padding preserved), and `EngineApi.reshape({ inputSize, … })`
2026-07-25 15:16:37 +02:00
→ `WasmIML.reshape` . `engine/io-reshape.ts` owns the identity map: **Keep capacity** (default)
permutes weights/examples in place and reconstructs only when active I/O outgrows the net;
**Exact I/O** reconstructs whenever active arity changes. Existing examples either adapt by
deleting removed dimensions and inserting the saved neutral placeholders (input 0, output 0.5 by
default), or clear, according to the persistent Settings choice. Feedback replay/exploration state
resets because it has no stable-ID contract. The spine tolerates arity changes; the training worker
feat(manifold): runtime-shaped net reshape with confirm modal (P2.3)
Wire the runtime-shaped WASM MLP (one-core-engine P2) through the manifold:
- WasmIML.reshape(dims, spread): calls nisps_ml_reshape, re-describes the
instance, reallocates every dim-dependent heap buffer, refreshes weightCount,
clears the TS Dataset mirror (C-side resets), drops the stale training worker,
and pushes the new shape through the sink.
- EngineApi.reshape exposes it and re-ticks the spine so outputs/audio reflect
the new net. Spine already tolerates the arity change (buffers resize, version
bumps); documented.
- Training worker protocol carries the current hidden dims; the worker
ensureNet()s its mirror net to match after a reshape.
- ConsoleApp offers the reshape behind ReshapeModal on an active-layout CHANGE
(never on load; default 32-input over-provisioned head + zero-padding
preserved when declined). British copy, reset-on-reshape.
- Drawers: delete the stale even/odd blending note; honest dedicated-dimensions
line + net-arity chip.
- Probe: __nisps.reshape(nIn) / .describe(); e2e reshape.spec (default 32/126,
reshape to 4, describe reports 4, bounded outputs, weight count 3148→2868,
spine still propagates). All 25 e2e pass (20 existing + 5 new).
- ONBOARDING: refresh the reshape status + stale hardwired-arity gotcha.
2026-07-14 03:54:12 +02:00
(`wasm-worker.ts`) carries the current dims in its train message and re-creates its mirror net to
2026-07-25 15:16:37 +02:00
match. The raw debug `window.__nisps.reshape(nIn)` remains a low-level reconstruct-and-clear call.
feat(manifold): P5.2/P5.3 — derive MF_MODES from schema truth + per-mode engine dims
Schema-backed modes in console/model.ts are now DERIVED from the codegen
schemas in src/modes/generated/ (source of truth): real param names/groups/
count, plus each mode's ml net shape (MFMode.ml) and schema engine_id. A thin
manifold OVERLAY supplies only label/glyph/ModeClass/input/ordering. New
browser-viable modes xiasri + slp_workshop get derived entries; schema-less
visualizer + c15 stay hand-written on DEFAULT_MODE_ML. Schema min/max/default/
label/curve surface as engine-unit metadata (schemaMin/... on MFParam) without
touching the 0..1 routing semantics.
Switching instrument mode reshapes the runtime-shaped WASM net to the mode's
schema ml config (ConsoleApp effect keyed on [engine, modeId]; no confirm
modal). Boot lands paf_synth dims (4->[10,10,14]->33) once WASM is ready. The
P2.3 axis-count reshape offer still reads the engine's live inputSize and does
not spuriously prompt on a mode switch.
Adds schema-modes.spec.ts (P5 gate): drives switches via a new window.__mf
debug seam and asserts describe() dims, getWeights count, output length/bounds,
and UI param count FROM the imported schemas; spot-checks trainAsync after a
switch. Updates reshape/probe-api/geo-dislike specs to assert from the boot
mode schema instead of the retired fixed 32/126 shape.
All gates green: typecheck, unit (9), build, e2e (33).
2026-07-18 12:45:06 +02:00
- **Per-mode net dims (P5.3):** switching INSTRUMENT mode reshapes the net to that mode's schema
2026-07-25 15:07:35 +02:00
`ml` config (`MFMode.ml` — input/hidden/output + legacy spread) via a `ConsoleApp` effect keyed on
2026-07-25 15:16:37 +02:00
`[engine, modeId]` . No confirm modal (switching instrument is deliberate). The effect depends on `engine` , so on boot
feat(manifold): P5.2/P5.3 — derive MF_MODES from schema truth + per-mode engine dims
Schema-backed modes in console/model.ts are now DERIVED from the codegen
schemas in src/modes/generated/ (source of truth): real param names/groups/
count, plus each mode's ml net shape (MFMode.ml) and schema engine_id. A thin
manifold OVERLAY supplies only label/glyph/ModeClass/input/ordering. New
browser-viable modes xiasri + slp_workshop get derived entries; schema-less
visualizer + c15 stay hand-written on DEFAULT_MODE_ML. Schema min/max/default/
label/curve surface as engine-unit metadata (schemaMin/... on MFParam) without
touching the 0..1 routing semantics.
Switching instrument mode reshapes the runtime-shaped WASM net to the mode's
schema ml config (ConsoleApp effect keyed on [engine, modeId]; no confirm
modal). Boot lands paf_synth dims (4->[10,10,14]->33) once WASM is ready. The
P2.3 axis-count reshape offer still reads the engine's live inputSize and does
not spuriously prompt on a mode switch.
Adds schema-modes.spec.ts (P5 gate): drives switches via a new window.__mf
debug seam and asserts describe() dims, getWeights count, output length/bounds,
and UI param count FROM the imported schemas; spot-checks trainAsync after a
switch. Updates reshape/probe-api/geo-dislike specs to assert from the boot
mode schema instead of the retired fixed 32/126 shape.
All gates green: typecheck, unit (9), build, e2e (33).
2026-07-18 12:45:06 +02:00
it fires once WASM is ready and lands the boot mode's dims (**paf_synth → 4→[10,10,14]→33**, weights
809 — NOT the 32→126 default). The reshape-offer effect reads the engine's CURRENT `inputSize`
live, so a mode switch that changes arity doesn't spuriously prompt (its baseline tracks axis
COUNT, unchanged by a pure dim change). Non-schema modes restore `DEFAULT_MODE_ML` (32→126).
Debug seam for tests: under `?debug=1` ConsoleApp installs `window.__mf`
(`setMode`/`getModeId`/`paramCount`/`modeIds`) — the UI-level analogue of `__nisps` , since no
in-UI instrument picker exists yet (`ctx.modes`/`setModeId` are plumbed but unrendered).
2026-07-25 15:07:35 +02:00
Manifold passes `spread=0` for boot, mode-switch reshapes, direct re-rolls, explore-and-place
scratchpad rolls, and VCV-forwarded randomise gestures by default. Settings → Experimental
features → **Xavier / spread randomisation** restores the schema spread and reveals the centred
regime switch in the expanded Learning drawer.
2026-06-28 22:20:58 +02:00
### Feedback — `src/feedback/`
feat(manifold): geometric dislike + jolt/OU via shared core (one-core-engine P3)
TS half of P3.1/P3.2/P3.3: wire the new WASM exports and delete the TS
approximations now that geometric-dislike, jolt, OU, and the seeded RNG live
in the C++/WASM core.
- types.ts/wasm-iml.ts: bind + wrap dislike_geometric, store_positive,
positive/negative_count, set_avoid_style, jolt_press/step/release/active/
lr_scale/tick_lr_ramp, explore_intensity/get/apply. Weight-mutating wrappers
republish weights (version bump); explore_apply reuses feedbackBuf.
- engine-api: extend .feedback (dislikeGeometric/storePositive/counts/
setAvoidStyle) + new .explore facade. thumbsDown now passes the HEARD
(routed) vector, not the raw MLP output (raw == net output => inert cold-start).
- feedback/controller.ts: delete dislikes[] + applyDislikeBias + both C++ GAP
blocks + the SeededRng field; dislike() -> core dislikeGeometric (returns
action, 15 => cold-start prompt); like() feeds the centroid via the core
thumbsUp; getState() exposes positive/negative counts. Delete feedback/rng.ts.
- engine/exploration.ts: execute the P3 SWAP POINT -> drive engine.explore.*;
delete engine/jolt.ts + ou-explore.ts. UI surface unchanged.
- ConsoleApp: one-time cold-start banner (British spelling), routed heard vector
at the dislike call site.
- App.tsx/spine.ts: under ?debug=1 pin a fixed seed + fixed per-tick dt so the
probe/e2e are deterministic (production keeps time-seeded, real-time dt).
- tests: new geo-dislike.spec.ts; probe gains dislikeGeometric/storePositive/
feedbackCounts/setAvoidStyle; the thumbsDown probe test now drives a real
distinct-heard-vector dislike (bare thumbsDown on the net's own output is
correctly inert under the geometric core). 27 e2e + 9 unit green.
Docs: manifold/ONBOARDING.md engine+feedback sections synced.
2026-07-14 04:54:27 +02:00
- `controller.ts` — `FeedbackController` , framework-neutral, owned by ConsoleApp. **As of one-core-
engine P3 it holds NO algorithm approximation** — a thin driver over the shared C++ core. Two modes:
**geometric-dislike** (default, Mode 1, "Push away") — `dislike()` calls `engine.feedback.
dislikeGeometric(heardVec)` (the k-NN centroid push-away in `nisps/ml/geo_push.hpp` ; returns
FeedbackAction 14=push / 15=cold-start) then `process()` ; `like()` runs the core's `thumbsUp`
(auto-stores the positive centroid in Avoid+Geometric) + `addExample` + `train` . **explore-and-
place** (Mode 2, positive-only) drives the core's snapshot/scratchpad/undo lifecycle; caller
accumulates anchors and trains on finalise with warm-start. Solo/arm via per-output mask.
- **The heard-vector rule:** the geometric dislike trains AWAY from the HEARD (post-pipeline, routed)
output — pass `engine.routedOutput()` , never the raw MLP output, or the cold-start MSE derivative is
zero (inert). The `EngineApi.feedback.thumbsDown` facade + the Mode-1 `dislike()` call site honour this.
- **Cold-start prompt:** a dislike with zero positives returns action 15 → ConsoleApp shows a one-time
"Like a few sounds first…" banner (dismissed on the next like or the dismiss button; rl-feedback §7).
- The interim `rng.ts` (`SeededRng`) and the `C++ GAP` approximation markers are **gone** — the seeded
RNG, geometric push, jolt, and OU all run in the core now.
2026-06-28 22:20:58 +02:00
### Backends — `src/backends/`
- `manager.ts` — `BackendManager` , the **single consumer of the engine spine** for output: subscribes,
reads `routedOutput()` each version bump, calls `active.send(routed)` . Gates audio (mutes synth when
a non-synth backend is active).
- Registered: `midi` (`midi-backend.ts`, CC out, ~20Hz throttle), `osc` (`osc-client.ts` → WebSocket
to the Deno bridge in `osc-bridge/bridge.ts` ), `synth` /`cvgate` (`passthrough-backend.ts`),
`particles` (`particle-backend.ts`, no-op — the visualiser is a separate `flow-field.ts` consumer),
`vcv` (`vcv-backend.ts`, bidirectional). `mapping.ts` , `presets.ts` , `useBackendManager.ts` support
config. `backend.ts` defines the `OutputBackend` interface + `OutputMapping` .
### Misc
- `src/serial/memlnaut-serial.ts` — **STUB** Web Serial scaffold for the MEMLNaut Editor mode (protocol TODO). `EditorPanel.tsx` is its UI.
2026-07-25 15:07:35 +02:00
- `src/settings/settings-store.ts` — localStorage settings (`mf-settings`): icon style, input-map
2026-07-25 15:16:37 +02:00
shape, exact-vs-capacity network resizing, adapt-vs-clear examples + neutral new-dimension values,
corner radius, and the opt-in legacy Xavier/spread feature flag.
2026-06-28 22:20:58 +02:00
- `src/midi-devices/` — codegen'd external-synth device templates.
- `src/debug/probe.ts` — `window.__nisps` synchronous probe (engine/audio/bus). Some playground
feature-store methods are present-but-inert (not ported yet) to keep the surface stable.
---
## 6. Gotchas & non-obvious rules
1. **Outputs are read imperatively, not via React state.** Subscribe to `version` , then call
`getOutputs()` /`routedOutput()`. Buffers are reused — copy if you need to retain.
2. **Asset URLs must resolve against `document.baseURI`, not `location.origin`.** `base: './'` +
the `/next/` subpath mean a hardcoded `/nisps.wasm` 404s. The `assetUrl()` helper (in
`engine-host.ts` , `wasm-iml.ts` , `wasm-worker.ts` ) handles this — use it.
3. ** `nisps.js` is non-module Emscripten glue (no ES exports).** Workers/worklet fetch + indirect-eval
to install the global `createNispsModule` ; the worklet uses raw `WebAssembly.instantiate` .
feat(manifold): runtime-shaped net reshape with confirm modal (P2.3)
Wire the runtime-shaped WASM MLP (one-core-engine P2) through the manifold:
- WasmIML.reshape(dims, spread): calls nisps_ml_reshape, re-describes the
instance, reallocates every dim-dependent heap buffer, refreshes weightCount,
clears the TS Dataset mirror (C-side resets), drops the stale training worker,
and pushes the new shape through the sink.
- EngineApi.reshape exposes it and re-ticks the spine so outputs/audio reflect
the new net. Spine already tolerates the arity change (buffers resize, version
bumps); documented.
- Training worker protocol carries the current hidden dims; the worker
ensureNet()s its mirror net to match after a reshape.
- ConsoleApp offers the reshape behind ReshapeModal on an active-layout CHANGE
(never on load; default 32-input over-provisioned head + zero-padding
preserved when declined). British copy, reset-on-reshape.
- Drawers: delete the stale even/odd blending note; honest dedicated-dimensions
line + net-arity chip.
- Probe: __nisps.reshape(nIn) / .describe(); e2e reshape.spec (default 32/126,
reshape to 4, describe reports 4, bounded outputs, weight count 3148→2868,
spine still propagates). All 25 e2e pass (20 existing + 5 new).
- ONBOARDING: refresh the reshape status + stale hardwired-arity gotcha.
2026-07-14 03:54:12 +02:00
4. **WASM MLP is runtime-shaped (since one-core-engine P2).** `nisps_ml_create(in, out, hidden[])`
honours its dims (non-positive/null → the default `32→[10,14,18]→126` head, so pre-P2 callers are
bit-identical), and `nisps_ml_reshape` swaps in a warm-started net at new dims. Weights = 3148 at
the default shape; reshaping only the input arity shifts the first layer (e.g. →4 inputs = 2868).
The firmware MLP stays compile-time templated — only the WASM/browser build is dynamic.
feat(manifold)!: route input/output pipelines + curves through the WASM core (P4.3/P4.4)
One-core-engine P4.3/P4.4: the input/output pipeline processing and the curve
catalog now live in the C++/WASM core (nisps/pipeline/*, nisps/core/math.hpp).
The TS ports are deleted and the browser drives the WASM chains.
Engine:
- WasmIML owns a nisps_pipeline_create handle + bridge buffers and exposes
setInputConfig (TS InputConfig → 15-float wire), processInput, resetInput,
setOutputConfig (Infinity slew → 0), setOutputFreezeMask, processOutput
(in place), resetOutput, curveApply, curveApplyBatch (chunked). Handle +
buffers created in init_, freed in dispose, output-sized buffers realloc'd
on reshape.
- Spine routes setInputs through iml.processInput/processOutput (state lives
C++-side); config source-of-truth stays TS-side and is pushed on attach /
setInputConfig / setOutputConfig. Preserves ?debug=1 fixed-dt determinism
(same dt fed to the WASM calls). EngineApi gains setInputConfig/
setOutputConfig/curveApply/curveApplyBatch.
- New types-only modules: pipeline-types.ts (InputConfig/OutputConfig +
defaults + wire int mappers) and curve-catalog.ts (CurveName + name→id).
types.ts declares the pipeline/curve C ABI. engine barrel updated.
- DELETED src/engine/{input-pipeline,output-pipeline,curves}.ts.
Tests (P4.4 gate — recorded-gesture regression):
- pipeline-golden.test.ts now loads the built WASM (indirect-eval shim,
tests/wasm-load.ts) and drives the frozen gesture/output fixtures through the
C++ chains, honouring the per-event dt clock contract. Tolerance 1e-5
(non-momentum drift <5e-7). The 3 momentum configs carry 1e-2: proven-inherent
f32 drift (a byte-faithful f32 port of the exact original algorithm matches
the WASM to <6e-8 while both diverge from the f64 capture by ~7-9e-3), NOT a
core bug.
- curves-golden.json: linear/square/sqrt/centered_power kept as the original
f64 captures (C++ matches within <3e-8); exp/log/sigmoid/cubic RE-BASELINED
from the WASM (deliberate switch to firmware-exact k=1 exp/log, slope-6
sigmoid, true cubic x^3). Provenance recorded in-file.
- _generate.ts rebuilt as the WASM curve re-baseline tool; pipeline-golden-lib
trimmed to pure data builders.
Docs: fixtures/README.md + manifold/ONBOARDING.md updated.
Gates: typecheck, bun test (9), vite build, playwright e2e (27) all green.
2026-07-18 12:21:35 +02:00
5. **Curves + input/output pipelines are C++/WASM only (one-core-engine P4).** No TS curve/pipeline
maths remains; the browser samples `nisps/core/math.hpp` + `nisps/pipeline/*` via the WASM. The
golden test (`tests/pipeline-golden.test.ts`) drives the WASM chains against the frozen fixtures.
NOTE: `exp/log/sigmoid/cubic` deliberately changed to the firmware-exact maths at P4 (see
`tests/fixtures/README.md` ); `linear/square/sqrt/centered_power` are unchanged. The 3 momentum input
configs carry a wide (`1e-2`) tolerance — proven-inherent f32 drift, documented in the test header.
2026-06-28 22:20:58 +02:00
6. **COOP/COEP headers are mandatory** for the WASM/worklet path — set in `vite.config.ts` for
dev+preview, and at nginx server scope in prod (inherited by `/next/` ).
7. **Never emit "C15"** in code or bundle — the smoke test fails on it. Synth is "Powerful Synth
Engine"/"Built-in Synth".
8. **Two `MFParam` writers** (stage controls AND dock rows) share one store — edits in either re-render both.
---
## 7. Deeper references (only when this doc isn't enough)
2026-07-13 22:15:46 +02:00
All in `docs/specs/` (at the repo root, not under `manifold/` ), with subdirectories:
- `plans/BUILD-PLAN.md` — locked decisions + the 12-step build sequence + spec pointers (the resume anchor).
2026-06-28 22:20:58 +02:00
- `engine-architecture.md` — full engine/spine/WASM design.
- `dock-spec.md` — dock + drawers spec. `inputs-spec.md` — mixed-input design. `backends-spec.md` — backends.
docs: the specs disposition pass (plan §8)
Roughly 20k lines were deleted from this repo in the last week and much of the
corpus still described the pre-deletion world in the present tense. Executes
the §8 table: archive the retired, reclassify the executed, prune the stale.
aimmersive-clone-spec -> _archive/ with a deprecated-by note
feedback-modes-port-spec -> plans/, kind: plan, status: executed
manifold-parity-features -> plans/, kind: plan, status: active
playground-2.0-rewrite -> status: superseded
engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten
present-tense against the shipped engine/
MAIN.md six contradicted claims fixed; registry resynced
vcv-module.md pruned to the current 8->16 contract and made the
single .nisps format spec
vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads
vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table
inputs/backends/dock trio grounding sections marked historical, dead cites fixed
Two rows of the §8 table were themselves wrong, corrected here: the deleted
full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no
§6.3), and codegen/README.md was already a MAP pointer with no port-solidjs
trigger left to remove.
Beyond the table — found by sweeping every backticked path in the changed docs
against `git ls-files`, which is how these should have been caught before:
manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current:
SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a
shot.spec.ts that does not exist. The whole stage table was keyed on a `focus`
axis that no longer exists — selection is now sandwich > particles >
composite. This matters more than the rest: CLAUDE.md tells every agent to
read ONBOARDING.md first for Manifold work, so it was actively teaching a
fiction. Rewritten against ConsoleApp.tsx.
MAP.md claimed the input layer reduces axes to the engine arity with an
"even/odd blend". input-layer.ts says the opposite in its own header: one
dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and
mean-blending was removed deliberately because it diluted every source.
AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed),
per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at
nisps::FixedBuffer (deleted).
Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/
Delta and cachedNovelty behind a live context-menu toggle that nothing reads;
vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's
origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to
9000, so out of the box they do not meet.
Firmware-build docs are deliberately untouched — the PlatformIO migration
lands next and rewrites all of them.
2026-07-21 20:17:58 +02:00
- `docs/adr/rl-feedback-design.md` + `plans/feedback-modes-port-spec.md` (executed) + `recon/findings-feedback-behaviour.md` — feedback modes (Mode 1/Mode 2).
- `_archive/aimmersive-clone-spec.md` / `recon/playground-2026.md` — the a-immersive feature parity target (archived reference).
2026-06-28 22:20:58 +02:00
- `src/backends/README.md` — backend wiring notes.
**Memories** (auto-loaded): `manifold-build` (status + locked decisions), `manifold-mixed-inputs`
(N-D input design), `manifold-deploy-pipeline` (the auto-deploy chain), `single-double-joystick-toggle` .
---
*Keep this in sync with the code — update it in the same commit as any change to UI structure, the
engine spine, the dock/drawers, or the build/deploy commands.*
