89 lines
5.4 KiB
Markdown
89 lines
5.4 KiB
Markdown
---
|
||
kind: spec
|
||
stability: stable
|
||
layer: cross-cutting
|
||
---
|
||
|
||
# Browser Engine Architecture — the EngineApi Seam and the Reactive Spine
|
||
|
||
*Supersession note (2026-07): this spec's original "One Engine, Two Skins" framing — a fresh
|
||
`playground2/` tree hosting a faithful a-immersive skin plus a Console skin — is superseded by
|
||
`plans/one-core-engine-refactor.md` (executed), which states: "the seam is built — Manifold is its
|
||
realisation; the axis of unification now runs vertically, core↔targets, not horizontally,
|
||
skin↔skin." The playground2 directory plan, reuse-vs-rebuild table, two-entry Vite/nginx serving
|
||
scheme, S0–S7 build sequence, and operator open questions were all consumed by the Manifold build
|
||
(`plans/BUILD-PLAN.md`, executed) and are deleted here; git history keeps them. What remains below
|
||
is the surviving contract: the headless `EngineApi` boundary and the reactive-spine invariant, as
|
||
realised in `manifold/src/engine/`.*
|
||
|
||
---
|
||
|
||
## 1. The seam: engine vs presentation
|
||
|
||
**The engine is separated from presentation by an explicit, headless boundary — `EngineApi` — and
|
||
the UI is a pure consumer of it.** The litmus test for every file: if it imports React, JSX, or
|
||
touches the DOM, it is presentation; otherwise it is the engine. The engine compiles and is
|
||
testable headlessly (the `window.__nisps` probe under `?debug=1`) with zero presentation.
|
||
|
||
```
|
||
┌──────────────────────────── ONE ENGINE ────────────────────────────┐
|
||
gesture │ input pipeline → ML (WASM) → output pipeline → backends │ sound
|
||
────────► │ (the reactive spine: one derivation path + one side-effect) │ ────────►
|
||
│ exposes a headless EngineApi — NO JSX, NO DOM │
|
||
└────────────────────────────────────────────────────────────────────┘
|
||
▲
|
||
│ reads EngineApi
|
||
┌──────────┴───────────┐
|
||
│ Manifold Console │
|
||
│ (React, manifold/) │
|
||
└──────────────────────┘
|
||
```
|
||
|
||
Realisation (see `MAP.md` §manifold for the per-file inventory):
|
||
|
||
- `manifold/src/engine/engine-api.ts` — the `EngineApi` class: the entire UI-facing surface
|
||
(inputs, training, feedback wrappers over the `nisps_ml_feedback_*` C ABI, audio lifecycle).
|
||
- `manifold/src/engine/spine.ts` — the reactive spine (§2).
|
||
- `manifold/src/engine/EngineProvider.tsx` + `useEngine.ts` — the React binding via
|
||
`useSyncExternalStore`. These two files are the only presentation-side entry to the engine.
|
||
- `manifold/src/debug/probe.ts` — `window.__nisps`, reads `EngineApi` only, gated by `?debug=1`.
|
||
|
||
The UI cannot create a second data path because it has no access to `WasmIML`, the backend, or the
|
||
spine's internals — only to `EngineApi`. That structural property, not vigilance, is what makes
|
||
the historic "MLP output stops updating when I move the joystick" bug class impossible.
|
||
|
||
---
|
||
|
||
## 2. The reactive spine
|
||
|
||
There is exactly one path from gesture to sound, every consumer reads it, and a desync is a
|
||
failing test rather than a recurring prod bug.
|
||
|
||
### 2.1 The derivation path (`manifold/src/engine/spine.ts`)
|
||
|
||
The spine is an external store living BELOW React (deliberately not on the render scheduler): the
|
||
`setInput` action derives processed → ml → routed **eagerly and synchronously** (the input
|
||
pipeline, inference, and output pipeline are all C++/WASM since one-core-engine P4) and fires the
|
||
single `backend.send` at the action's tail, off-render. Contract rules:
|
||
|
||
- **One side-effect.** `backend.send(routed)` at the tail of `setInput` is the only transport
|
||
call in the derivation path.
|
||
- **No per-frame allocation.** Buffers are reused (`routedBuf` threaded through the output
|
||
pipeline and handed to the backend).
|
||
- **Weights mutate only through engine actions** (train / feedback / reshape). Structural/training
|
||
state bumps `version`; every inference advances a separate `outputVersion`. Both expose explicit
|
||
subscriptions, so audio propagation never depends on React scheduling.
|
||
- **React state and live output are separate channels.** Structural/training consumers use
|
||
`useSyncExternalStore(subscribe, version)`. DOM output consumers opt into the throttled 30 Hz
|
||
`subscribeOutputs/outputVersion` channel. Canvases read and shape the live `Float32Array`s
|
||
imperatively in rAF and never re-render per frame.
|
||
|
||
### 2.2 The live-feedback guarantee and its e2e assertion
|
||
|
||
**Guarantee:** any change to the input (gesture) or the weights (train / feedback) propagates — in
|
||
the same synchronous action — to the routed output, which the single effect sends to the backend
|
||
and which every UI consumer reads.
|
||
|
||
The invariant is asserted in CI by `manifold/tests/e2e/spine.spec.ts` via the synchronous
|
||
`window.__nisps` probe: set inputs, read outputs, expect change (plus probe-survives-mode-switch).
|
||
Because the probe reads `EngineApi` only, it exercises the exact path the UI uses.
|