memlnaut-nisps/docs/specs/plans/one-core-engine-refactor.md

---
kind: plan
status: active
---

# One Core Engine — Firmware + Manifold Reunification Plan

*Dated 2026-07-13. Operator-confirmed scope (see §1). Principle: **one core engine (`nisps/`) for both the
resource-constrained RP2350 hardware and the Manifold browser app.** Everything with behaviour — ML, feedback,
exploration, RNG, DSP, input/output processing, curves — lives once, in C++. TypeScript keeps only UI,
reactivity, and transport. This plan supersedes the browser-side "one engine, two skins" framing of
`../engine-architecture.md` (that seam is built — Manifold is its realisation); the axis of unification now
runs vertically, core↔targets, not horizontally, skin↔skin.*

## 0. Where duplication actually lives today (audited 2026-07-13)

| Component | Canonical | Duplicate / divergence | Disposition |
|---|---|---|---|
| MLP, training, init, RL `move_weights` | `nisps/ml/` | `vcv/src/iml.hpp` full runtime reimpl ("delta #5") | Reunify (P2, P6) |
| Feedback explore-and-place | `nisps/ml/feedback.hpp` | none — firmware + manifold both call it ✓ | Keep |
| Geometric dislike (Mode 1) | *(missing from core)* | `manifold/src/feedback/controller.ts` TS approximation, explicit `C++ GAP` markers (lines ~369, ~436) | Port to core (P3) |
| Jolt weight-morph | `nisps/ml/jolt.hpp` | `playground/src/ml/jolt.ts` TS port; absent in manifold | Expose via WASM (P3) |
| OU exploration noise | `nisps/ml/ou_noise.hpp` | `playground/src/output/ou-explore.ts` TS port; absent in manifold | Expose via WASM (P3) |
| Deterministic RNG | `nisps/core/rng.hpp` | `manifold/src/feedback/rng.ts` (`SeededRng`, self-described stand-in) | Replace with core stream (P3) |
| Input pipeline | *(TS-only concept)* | `playground/src/input/pipeline.ts` ≡ `manifold/src/engine/input-pipeline.ts` | Port to core (P4) |
| Output pipeline + curves | curves mirror `nisps/core/math.hpp` | byte-identical TS pairs in playground + manifold | Port to core (P4) |
| Audio DSP engines (×8) | `nisps/engines/` | none — both targets run the WASM ✓ | Keep |
| TS engine spine (wasm-iml, worker, host, worklet) | — | manifold forked from playground; several files byte-identical | Single home in manifold (P1) |
| Compiled `nisps.wasm` | `scripts/build-wasm.sh` → `playground/public/` | `manifold/public/` byte-identical **manual copy** | Retarget build (P0) |
| Mode param schemas | `schemas/modes/*.json` → codegen | manifold bypasses codegen; hardcodes `MF_MODES` in `console/model.ts` | Codegen for manifold (P5) |
| WASM MLP arity | `bindings.cpp` `MLP<32,10,14,18,126>`; `nisps_ml_create` ignores requested dims | forces clamping, blocks per-mode arch, motivated the VCV reimpl | Runtime-shaped browser MLP (P2) |

## 1. Locked decisions (operator, 2026-07-13)

| Decision | Choice |
|---|---|
| `playground/` | **Retired.** Archived to branch `archive/playground-solidjs`, deleted from main. Manifold is the sole browser app; WASM build, parity-check, codegen all retarget manifold. |
| Depth of "one core" | **All algorithms in C++.** Feedback (incl. geometric dislike), Jolt, OU, RNG, input/output pipelines, curves move into `nisps/`; TS keeps UI/reactivity/transport only. |
| MLP arity | **Template the hardware, dynamic browser.** Firmware keeps compile-time `MLP<...>` (zero-heap contract intact); WASM/VCV targets get a runtime-shaped MLP sharing the same algorithm code. Heap allowed at construction time on non-RP2350 targets only. |
| VCV | **In scope, final phase** (P6); may slip without blocking anything else. |

**Supersedes:** `BUILD-PLAN.md` locked decision "Modular N×M MLP: multiple WASM modules + warm-start; NOT
runtime-shaped" (2026-06-27). The runtime-shaped MLP is now the chosen mechanism; warm-start on reshape is
retained as a feature of it.

## 2. Target architecture

```
                    ┌───────────────────────── nisps/ (ONE CORE) ─────────────────────────┐
                    │ core/   rng, math+curves, buffers, perf attrs                        │
                    │ ml/     algorithms templated on StoragePolicy:                       │
                    │           Fixed<NIn,H1,H2,H3,NOut>  (arrays, zero-heap → RP2350)     │
                    │           Dynamic                    (sized at construction → WASM,VCV)│
                    │         mlp / training / init / rl / jolt / ou_noise / feedback /    │
                    │         geo_push / replay  (geometric dislike lands here)            │
                    │ pipeline/  input chain (deadzone→zoom→curve→smooth→momentum)         │
                    │            output chain (curve→smooth→slew→freeze)      ← NEW        │
                    │ dsp/ engines/ modes/   (unchanged)                                   │
                    └──────────────────────────────────────────────────────────────────────┘
                       ▲ direct #include                ▲ nisps/wasm/bindings.cpp (C ABI)
              ┌────────┴─────────┐            ┌─────────┴──────────┐        ┌───────────────┐
              │ firmware/ glue   │            │ manifold/ (React)  │        │ vcv/ (P6)     │
              │ RP2350, Fixed    │            │ UI + reactive spine│        │ Dynamic       │
              │ storage, no heap │            │ + transports only  │        │ storage       │
              └──────────────────┘            └────────────────────┘        └───────────────┘
```

Litmus test per file: **if it computes behaviour that affects sound, weights, or parameter values, it is C++
in `nisps/`; if it schedules, renders, or transports, it may be TS.** The existing browser litmus (JSX/DOM =
skin) still applies inside manifold; this plan adds the vertical one.

## 3. Phases

Each phase ends green on its test gate and is independently landable. File phases as **ergo** tasks
(one per phase, sub-tasks per bullet); do not start a phase before its predecessor's gate is green.

### P0 — Plumbing hygiene (hours, no behaviour change) — ✅ landed 2026-07-13

- ✅ `scripts/build-wasm.sh` emits to `manifold/public/` (keep `playground/public/` copy only until P1 lands).
- ✅ `scripts/parity-check.sh` (and `tests/cpp/parity_wasm.mjs`) read the manifold artifact.
- ✅ Fix stale doc: the `MLP<2,…>` line was in `docs/AGENT-REFERENCE.md` + `nisps/wasm/README.md` (MAP.md was already correct); both now say `MLP<32u,10u,14u,18u,126u>`.
- ✅ `ml-debug.log`/`graphify-out/` were already absent from the tree; `.claude/worktrees/` gitignored.
- **Gate met:** `run-all-tests.sh` green (Playwright leg via the BUILD-PLAN non-snap-node VPS runner); parity PASS reading `manifold/public/` (max delta 2.4e-7); manifold typecheck+build green against the freshly-built artifact.

### P1 — Retire playground, single TS home (≈1 day) — ✅ landed 2026-07-13

- ✅ Branch `archive/playground-solidjs` + tag `playground-solidjs-final`; `playground/` deleted from main.
  Parity fixtures for P4 captured FIRST: `manifold/tests/fixtures/` (288-event gesture trace, curve
  goldens, input/output pipeline goldens under 14+8 configs) + drift-guard `pipeline-golden.test.ts`.
  Note: input pipeline's momentum path reads `performance.now()` — fixtures pin a clock contract (see
  fixtures README). C15 (stub + c15.wasm) now lives only on the archive branch (ALIGNMENT defect #1).
- ✅ Jolt press + OU explore UI shells in the Learning drawer; interim TS math via get/set_weights in
  `manifold/src/engine/{exploration,jolt,ou-explore}.ts`, marked `P3 SWAP POINT`.
- ✅ Keeper Playwright specs ported: `probe-api.spec.ts` (15 tests), `spine.spec.ts` (4). Playground UI/
  persistence specs dropped with the playground.
- ✅ References retargeted: run-all-tests stage 5 → manifold, ci.yml manifold-tests job, osc-bridge.yml
  (was already broken), codegen TS target removed (returns P5), AGENTS/README/MAP/ALIGNMENT/
  AGENT-REFERENCE/specs-MAIN. The VPS deploy webhook already built only manifold — no change needed.
- **Gate met:** full `run-all-tests.sh` green with playground gone (ctest 4/4, parity PASS, lint clean,
  manifold 9 unit + 20 e2e).

### P2 — Storage-policy split: templated hardware, dynamic browser (the structural centre, ≈1 wk) — ✅ landed 2026-07-14 (hardware timing spot-check of chokepoint B pending physical MEMLNaut)

- ✅ (landed 2026-07-13) Refactor `nisps/ml/` so algorithms (forward, backprop/SGD, init, `move_weights`)
  are written once against a storage policy (`mlp.hpp` `MLPCore<Storage>`; jolt/OU/feedback already operate
  on the MLP surface and needed no change). Two models:
  - `FixedStorage<NIn,H1,H2,H3,NOut>` (`nisps/ml/storage.hpp`) — `std::array`, zero heap; `MLP<...>` is an
    alias preserving the full compile-time surface. **RP2350 contract verified: PAFSynth `.text`
    122324→122692 = +0.30% (±1% budget); ctest + golden vectors + WASM parity bit-stable.**
  - `DynamicStorage` (`nisps/ml/dynamic_storage.hpp`) — sizes at construction, single arena allocation,
    nothing per-call. `#error`s under `NISPS_TARGET_EMBEDDED`; sole `lint-cpp.sh` heap-allowlist entry, with
    a lint check that fails if the guard is ever removed.
- ✅ (landed 2026-07-14, operator-approved) `nisps_ml_create(input, output, hidden[])` honours its
  arguments (non-positive/null → the historical 32→[10,14,18]→126 defaults, keeping pre-P2 callers
  bit-identical). `nisps_ml_reshape` = new instance + warm-start copy of overlapping weights
  (`nisps/ml/warm_start.hpp`); feedback controller re-created (state resets — front-end modal).
  `FeedbackController` got the same storage split (`FeedbackControllerCore<FbStorage>`, fixed alias for
  firmware/tests, `DynamicFeedbackStorage` for the browser). `nisps_ml_describe` now takes the handle
  (null → default shape). Verified: reshape ABI smoke (dims honoured, overlap survives, invalid dims
  rejected), warm-start ctest (grow+shrink), parity PASS unchanged, firmware `.text` unchanged.
- ✅ (landed 2026-07-14) Manifold reshape wiring: `WasmIML.reshape` + `EngineApi.reshape` (buffers/worker/
  spine follow the new dims; training-worker protocol carries `hidden`); reset-on-reshape confirm modal
  offered on genuine input-layout changes (default 32-in over-provisioned head untouched on load/decline);
  stale even/odd-blend copy deleted. The mean-blend clamping itself was already gone (dedicated dims);
  the `min(n, inputSize)` guard + zero-padding stay as the legitimate declined-reshape behaviour.
  XIASRI/sound-analysis multi-input modes become browser-viable (schema-real dims arrive at P5).
- **Gate met:** fixed↔dynamic bit-identical ctest (+ warm-start grow/shrink ctest + reshape ABI smoke);
  `parity-check.sh` native↔WASM PASS unchanged (2.4e-7); PAFSynth `.text` 122324→122692 (+0.30%, within
  ±1%); manifold 9 unit + 25 e2e. Hardware audio-callback timing check deferred to the chokepoint-B
  hardware session (operator, physical MEMLNaut).

### P3 — Exploration + feedback fully in core (≈3–4 days) — ✅ software landed 2026-07-14; ⏳ chokepoint A hardware spot-check pending (operator, physical MEMLNaut)

- ✅ **Geometric dislike:** firmware k-NN centroid push (upstream `InterfaceRL` @ `0a541cc`, verbatim
  constants) ported into `nisps/ml/geo_push.hpp` + `replay.hpp` + `mlp.train_targets`; the press+optimise
  halves collapsed into one synchronous `dislike_geometric()` per the ADR. `AvoidStyle::Geometric` is the
  Avoid default; `Diffuse` (legacy move_weights) kept for A/B. Exposed via
  `nisps_ml_feedback_dislike_geometric` (+ store_positive/counts/set_avoid_style). TS approximation +
  both `C++ GAP` blocks deleted; two call-site bugs fixed en route (raw vs HEARD post-pipeline vector —
  raw gives a zero-derivative no-op). Cold-start prompt per ADR §7.
- ✅ **Jolt / OU:** exposed through bindings (`nisps_ml_jolt_press/step/release/lr_scale/...`,
  `nisps_ml_explore_intensity/apply`; browser `OUNoise<4096>` over-provisioned); manifold's P1 UI shells
  switched at the marked swap point; interim TS ports (`jolt.ts`, `ou-explore.ts`) deleted. OU applies
  through the spine's output morph hook.
- ✅ **RNG:** `manifold/src/feedback/rng.ts` deleted; all feedback/exploration randomness is the core's
  seeded per-instance `Rng` streams. (Debug-only fixed seed/dt under `?debug=1` for deterministic e2e.)
- **Gate:** ✅ parity Stage 6 (v4) — scripted seeded feedback session (2 likes → centroid → 2 dislikes)
  produces identical weight trajectories native↔WASM (961 floats, 2.4e-7); ctest geo suite; manifold 9
  unit + 27 e2e. ⏳ Hardware spot-check (chokepoint A) BLOCKED on the physical MEMLNaut — and on ergo bug
  `10c3e55c`: the explore/place wiring is linker-GC'd out of the PAFSynth ELF (pre-existing, discovered
  during P3; the gestures are dead code on that build until fixed).

### P4 — Input/output pipelines + curves into core (≈3 days) — ✅ landed 2026-07-18

- ✅ New `nisps/pipeline/`: `input_chain.hpp` + `output_chain.hpp` (capacity-templated), perf-contract
  structs, caller-supplied dt with an internal clock (no wall time — deterministic), serialisable state.
- ✅ Curves: `nisps/core/math.hpp` is the single catalog (+ parameterised `centered_power`); bindings expose
  `nisps_curve_apply(id, x, param)` + batch. TS `curves.ts` deleted. NOTE: the TS mirror's `exp/log/
  sigmoid/cubic` maths silently diverged from the canonical catalog — the browser now behaves
  firmware-exact (re-baselined in `curves-golden.json` with a provenance note; ALIGNMENT entry).
- ✅ Manifold pipelines are thin WASM calls (one `nisps_pipeline_create` handle per WasmIML; state C++-side;
  config stays TS-side and is pushed over the 15-float wire layout).
- ✅ Golden TS-vs-C++ tests retired → the fixture suite drives the WASM directly.
- **Gate met:** spine e2e unchanged (27 e2e green); recorded-gesture regression against the P1 pre-migration
  fixtures passes — non-momentum configs at ≤5e-7, momentum configs at 1e-2 with the drift PROVEN inherent
  f32-vs-f64 (a byte-faithful f32 reference reproduces the WASM to <6e-8; documented in the test + fixtures
  README). Parity stage 7 (v5): native↔WASM pipeline/curve floats bit-identical.

### P5 — Schema/codegen serves manifold (≈2 days) — ✅ landed 2026-07-18

- ✅ `codegen/generate.ts` emits `manifold/src/modes/generated/` alongside the C++ headers. `MF_MODES` is
  derived from the generated schemas (real param names/groups/counts + per-mode `ml` config + `engine_id`);
  labels/glyphs/class/ordering stay a manifold overlay; `xiasri` + `slp_workshop` join the browser catalog;
  schema-less `visualizer`/`c15` remain hand-written overlays on the default shape.
- ✅ Per-mode `ml.input_size/output_size/hidden` is REAL in the browser: mode switch reshapes the runtime
  MLP to the schema config (boot mode paf_synth runs 4→[10,10,14]→33, not the old fixed 32→126). Codegen
  gained the firmware-fit check (exactly 3 hidden layers; dims ≤4096).
- **Gate met:** codegen idempotence golden covers both languages and runs in `run-all-tests.sh` stage 5;
  `schema-modes.spec.ts` asserts dims/weight-count/param-count FROM the imported schemas for 4 modes; the
  existing shape-assuming specs were re-derived from schema constants. 33 e2e green.

### P6 — VCV reunification (later; may slip)

- `vcv/src/iml.hpp` deleted; module consumes `nisps/ml/` with `DynamicStorage` (its 8→16 runtime shape is
  exactly the P2 case). `DetRng` replaced by `nisps/core/rng.hpp`. Closes vcv-module.md delta #5.
- OSC server stays vcv-local (transport, not behaviour).
- **Gate:** vcv builds; seeded train/infer parity vs native ctest.

## 4. Risks

| Risk | Mitigation |
|---|---|
| P2 template refactor destabilises firmware perf (chokepoint B) | Fixed-storage path stays `std::array` + same attrs; compare `.text`/RAM and audio-callback timing before/after; land behind a ctest parity gate. |
| Dynamic storage leaks heap use into firmware paths | Compile-time guard (`#if NISPS_TARGET_EMBEDDED` excludes `DynamicStorage`); `lint-cpp.sh` already fails heap under `nisps/` — carve an explicit allowlist for the dynamic TU only. |
| Per-event WASM calls for pipelines add latency | Pointer-rate ≈120 Hz, one call each — negligible; batch API as fallback. rAF/canvas never calls WASM. |
| Geometric-dislike port changes feel vs firmware | Port from upstream source verbatim, seeded-trajectory parity test, then hardware A/B (chokepoint A) before deleting the TS fallback. |
| Retiring playground loses a working reference | Archive branch + tag; parity fixtures (recorded gestures, golden outputs) extracted *before* deletion in P1. |
| Manifold worklet + main thread now both depend on retargeted wasm build | P0 gate builds manifold from fresh artifact in CI, killing the manual-copy hazard permanently. |

## 5. Doc-sync obligations

- On each phase landing: update `MAP.md` (it is the shipped-behaviour authority) and burn down this plan's
  phase table; when all phases land, set `status: executed`.
- `MAIN.md` registry: this file added; `engine-architecture.md` remains authoritative for the *browser-side*
  seam only; BUILD-PLAN.md's MLP row annotated as superseded (done 2026-07-13).
- `ALIGNMENT.md` defect #3 (fixed WASM arity) closes at P2; the accepted-divergence log entries for Jolt/OU
  TS ports and vcv delta #5 close at P3/P6.