9ad1f78ddd
Roughly 20k lines were deleted from this repo in the last week and much of the
corpus still described the pre-deletion world in the present tense. Executes
the §8 table: archive the retired, reclassify the executed, prune the stale.
aimmersive-clone-spec -> _archive/ with a deprecated-by note
feedback-modes-port-spec -> plans/, kind: plan, status: executed
manifold-parity-features -> plans/, kind: plan, status: active
playground-2.0-rewrite -> status: superseded
engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten
present-tense against the shipped engine/
MAIN.md six contradicted claims fixed; registry resynced
vcv-module.md pruned to the current 8->16 contract and made the
single .nisps format spec
vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads
vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table
inputs/backends/dock trio grounding sections marked historical, dead cites fixed
Two rows of the §8 table were themselves wrong, corrected here: the deleted
full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no
§6.3), and codegen/README.md was already a MAP pointer with no port-solidjs
trigger left to remove.
Beyond the table — found by sweeping every backticked path in the changed docs
against `git ls-files`, which is how these should have been caught before:
manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current:
SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a
shot.spec.ts that does not exist. The whole stage table was keyed on a `focus`
axis that no longer exists — selection is now sandwich > particles >
composite. This matters more than the rest: CLAUDE.md tells every agent to
read ONBOARDING.md first for Manifold work, so it was actively teaching a
fiction. Rewritten against ConsoleApp.tsx.
MAP.md claimed the input layer reduces axes to the engine arity with an
"even/odd blend". input-layer.ts says the opposite in its own header: one
dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and
mean-blending was removed deliberately because it diluted every source.
AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed),
per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at
nisps::FixedBuffer (deleted).
Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/
Delta and cachedNovelty behind a live context-menu toggle that nothing reads;
vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's
origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to
9000, so out of the box they do not meet.
Firmware-build docs are deliberately untouched — the PlatformIO migration
lands next and rewrites all of them.
88 lines
5.2 KiB
Markdown
88 lines
5.2 KiB
Markdown
---
|
||
kind: spec
|
||
stability: stable
|
||
layer: cross-cutting
|
||
---
|
||
|
||
# Browser Engine Architecture — the EngineApi Seam and the Reactive Spine
|
||
|
||
*Supersession note (2026-07): this spec's original "One Engine, Two Skins" framing — a fresh
|
||
`playground2/` tree hosting a faithful a-immersive skin plus a Console skin — is superseded by
|
||
`plans/one-core-engine-refactor.md` (executed), which states: "the seam is built — Manifold is its
|
||
realisation; the axis of unification now runs vertically, core↔targets, not horizontally,
|
||
skin↔skin." The playground2 directory plan, reuse-vs-rebuild table, two-entry Vite/nginx serving
|
||
scheme, S0–S7 build sequence, and operator open questions were all consumed by the Manifold build
|
||
(`plans/BUILD-PLAN.md`, executed) and are deleted here; git history keeps them. What remains below
|
||
is the surviving contract: the headless `EngineApi` boundary and the reactive-spine invariant, as
|
||
realised in `manifold/src/engine/`.*
|
||
|
||
---
|
||
|
||
## 1. The seam: engine vs presentation
|
||
|
||
**The engine is separated from presentation by an explicit, headless boundary — `EngineApi` — and
|
||
the UI is a pure consumer of it.** The litmus test for every file: if it imports React, JSX, or
|
||
touches the DOM, it is presentation; otherwise it is the engine. The engine compiles and is
|
||
testable headlessly (the `window.__nisps` probe under `?debug=1`) with zero presentation.
|
||
|
||
```
|
||
┌──────────────────────────── ONE ENGINE ────────────────────────────┐
|
||
gesture │ input pipeline → ML (WASM) → output pipeline → backends │ sound
|
||
────────► │ (the reactive spine: one derivation path + one side-effect) │ ────────►
|
||
│ exposes a headless EngineApi — NO JSX, NO DOM │
|
||
└────────────────────────────────────────────────────────────────────┘
|
||
▲
|
||
│ reads EngineApi
|
||
┌──────────┴───────────┐
|
||
│ Manifold Console │
|
||
│ (React, manifold/) │
|
||
└──────────────────────┘
|
||
```
|
||
|
||
Realisation (see `MAP.md` §manifold for the per-file inventory):
|
||
|
||
- `manifold/src/engine/engine-api.ts` — the `EngineApi` class: the entire UI-facing surface
|
||
(inputs, training, feedback wrappers over the `nisps_ml_feedback_*` C ABI, audio lifecycle).
|
||
- `manifold/src/engine/spine.ts` — the reactive spine (§2).
|
||
- `manifold/src/engine/EngineProvider.tsx` + `useEngine.ts` — the React binding via
|
||
`useSyncExternalStore`. These two files are the only presentation-side entry to the engine.
|
||
- `manifold/src/debug/probe.ts` — `window.__nisps`, reads `EngineApi` only, gated by `?debug=1`.
|
||
|
||
The UI cannot create a second data path because it has no access to `WasmIML`, the backend, or the
|
||
spine's internals — only to `EngineApi`. That structural property, not vigilance, is what makes
|
||
the historic "MLP output stops updating when I move the joystick" bug class impossible.
|
||
|
||
---
|
||
|
||
## 2. The reactive spine
|
||
|
||
There is exactly one path from gesture to sound, every consumer reads it, and a desync is a
|
||
failing test rather than a recurring prod bug.
|
||
|
||
### 2.1 The derivation path (`manifold/src/engine/spine.ts`)
|
||
|
||
The spine is an external store living BELOW React (deliberately not on the render scheduler): the
|
||
`setInput` action derives processed → ml → routed **eagerly and synchronously** (the input
|
||
pipeline, inference, and output pipeline are all C++/WASM since one-core-engine P4) and fires the
|
||
single `backend.send` at the action's tail, off-render. Contract rules:
|
||
|
||
- **One side-effect.** `backend.send(routed)` at the tail of `setInput` is the only transport
|
||
call in the derivation path.
|
||
- **No per-frame allocation.** Buffers are reused (`routedBuf` threaded through the output
|
||
pipeline and handed to the backend).
|
||
- **Weights mutate only through engine actions** (train / feedback / reshape), each of which bumps
|
||
the spine's version counter — there is no write path that can update audio without notifying
|
||
the UI.
|
||
- **React subscribes via `useSyncExternalStore(subscribe, version)`** — the version counter, not
|
||
the arrays; canvases read the live `Float32Array`s imperatively in rAF and never re-render per
|
||
frame.
|
||
|
||
### 2.2 The live-feedback guarantee and its e2e assertion
|
||
|
||
**Guarantee:** any change to the input (gesture) or the weights (train / feedback) propagates — in
|
||
the same synchronous action — to the routed output, which the single effect sends to the backend
|
||
and which every UI consumer reads.
|
||
|
||
The invariant is asserted in CI by `manifold/tests/e2e/spine.spec.ts` via the synchronous
|
||
`window.__nisps` probe: set inputs, read outputs, expect change (plus probe-survives-mode-switch).
|
||
Because the probe reads `EngineApi` only, it exercises the exact path the UI uses.
|