memlnaut-nisps/docs/specs/engine-architecture.md

kind	stability	layer
spec	stable	cross-cutting

Browser Engine Architecture — the EngineApi Seam and the Reactive Spine

Supersession note (2026-07): this spec's original "One Engine, Two Skins" framing — a fresh playground2/ tree hosting a faithful a-immersive skin plus a Console skin — is superseded by plans/one-core-engine-refactor.md (executed), which states: "the seam is built — Manifold is its realisation; the axis of unification now runs vertically, core↔targets, not horizontally, skin↔skin." The playground2 directory plan, reuse-vs-rebuild table, two-entry Vite/nginx serving scheme, S0–S7 build sequence, and operator open questions were all consumed by the Manifold build (plans/BUILD-PLAN.md, executed) and are deleted here; git history keeps them. What remains below is the surviving contract: the headless EngineApi boundary and the reactive-spine invariant, as realised in manifold/src/engine/.

---

1. The seam: engine vs presentation

The engine is separated from presentation by an explicit, headless boundary — EngineApi — and the UI is a pure consumer of it. The litmus test for every file: if it imports React, JSX, or touches the DOM, it is presentation; otherwise it is the engine. The engine compiles and is testable headlessly (the window.__nisps probe under ?debug=1) with zero presentation.

            ┌──────────────────────────── ONE ENGINE ────────────────────────────┐
   gesture  │  input pipeline → ML (WASM) → output pipeline → backends            │  sound
  ────────► │      (the reactive spine: one derivation path + one side-effect)    │ ────────►
            │  exposes a headless EngineApi — NO JSX, NO DOM                      │
            └────────────────────────────────────────────────────────────────────┘
                                          ▲
                                          │ reads EngineApi
                               ┌──────────┴───────────┐
                               │  Manifold Console    │
                               │  (React, manifold/)  │
                               └──────────────────────┘

Realisation (see MAP.md §manifold for the per-file inventory):

• manifold/src/engine/engine-api.ts — the EngineApi class: the entire UI-facing surface (inputs, training, feedback wrappers over the nisps_ml_feedback_* C ABI, audio lifecycle).
• manifold/src/engine/spine.ts — the reactive spine (§2).
• manifold/src/engine/EngineProvider.tsx + useEngine.ts — the React binding via useSyncExternalStore. These two files are the only presentation-side entry to the engine.
• manifold/src/debug/probe.ts — window.__nisps, reads EngineApi only, gated by ?debug=1.

The UI cannot create a second data path because it has no access to WasmIML, the backend, or the spine's internals — only to EngineApi. That structural property, not vigilance, is what makes the historic "MLP output stops updating when I move the joystick" bug class impossible.

---

2. The reactive spine

There is exactly one path from gesture to sound, every consumer reads it, and a desync is a failing test rather than a recurring prod bug.

2.1 The derivation path (manifold/src/engine/spine.ts)

The spine is an external store living BELOW React (deliberately not on the render scheduler): the setInput action derives processed → ml → routed eagerly and synchronously (the input pipeline, inference, and output pipeline are all C++/WASM since one-core-engine P4) and fires the single backend.send at the action's tail, off-render. Contract rules:

• One side-effect. backend.send(routed) at the tail of setInput is the only transport call in the derivation path.
• No per-frame allocation. Buffers are reused (routedBuf threaded through the output pipeline and handed to the backend).
• Weights mutate only through engine actions (train / feedback / reshape). Structural/training state bumps version; every inference advances a separate outputVersion. Both expose explicit subscriptions, so audio propagation never depends on React scheduling.
• React state and live output are separate channels. Structural/training consumers use useSyncExternalStore(subscribe, version). DOM output consumers opt into the throttled 30 Hz subscribeOutputs/outputVersion channel. Canvases read and shape the live Float32Arrays imperatively in rAF and never re-render per frame.

2.2 The live-feedback guarantee and its e2e assertion

Guarantee: any change to the input (gesture) or the weights (train / feedback) propagates — in the same synchronous action — to the routed output, which the single effect sends to the backend and which every UI consumer reads.

The invariant is asserted in CI by manifold/tests/e2e/spine.spec.ts via the synchronous window.__nisps probe: set inputs, read outputs, expect change (plus probe-survives-mode-switch). Because the probe reads EngineApi only, it exercises the exact path the UI uses.