docs: P2 complete — burn down plan, close ALIGNMENT defect #3 (fixed WASM arity)

- plan P2 marked landed (hardware timing spot-check deferred to the
  chokepoint-B session on physical MEMLNaut)
- ALIGNMENT: defect #3 resolved by the runtime-shaped browser MLP
- MAP/AGENT-REFERENCE/nisps-wasm-README: describe the honoured-dims
  create + reshape ABI; note per-mode dims become schema-real at P5
- fresh wasm artifacts from the gate run
This commit is contained in:
monkey-w1n5t0n 2026-07-14 03:56:04 +02:00
parent 7e4dc26f6e
commit 7e957457cd
5 changed files with 30 additions and 39 deletions

View file

@ -26,14 +26,6 @@ The clean-slate rewrite (2026-04-29) consolidated everything into one C++20 code
**Rough cost.** Half a day. Add `nisps_ml_train_with_history` (or extend the existing call) returning a pointer to the loss array; copy on the JS side. **Rough cost.** Half a day. Add `nisps_ml_train_with_history` (or extend the existing call) returning a pointer to the loss array; copy on the JS side.
### 3. WASM MLP architecture is fixed (2026-04-29)
**What.** WASM compiles `MLP<2, 10, 14, 18, 126>` only. Modes with smaller `output_size` use a slice; modes that want different hidden layer shapes (e.g. `[10, 10, 14]` for smaller modes) can't get them in the browser.
**Why it blocks the mission.** Per-mode ML architecture variation is one of our four "research dimensions". The fix isn't urgent for the current mode set (all schemas are within the universal arch's capacity), but the moment we want to experiment with bigger networks or different shapes, this hits.
**Rough cost.** Medium. Either compile multiple WASM modules (one per arch shape; load on demand) or move to a runtime-shaped MLP (loses some compile-time perf). Templates-vs-runtime is a research-vs-firmware-perf tradeoff worth a separate decision doc.
### 4. `NISPS_AUDIO_FUNC` host fallback is misshapen (2026-04-29) ### 4. `NISPS_AUDIO_FUNC` host fallback is misshapen (2026-04-29)
**What.** `nisps/core/perf.hpp` defines `NISPS_AUDIO_FUNC(decl) decl` for the host but the firmware path `__not_in_flash_func(name)` takes only a function name (it stringifies into a section attribute). The two forms don't match. Stream 6 (firmware glue) avoided the macro to dodge the inconsistency, but it's still a footgun. **What.** `nisps/core/perf.hpp` defines `NISPS_AUDIO_FUNC(decl) decl` for the host but the firmware path `__not_in_flash_func(name)` takes only a function name (it stringifies into a section attribute). The two forms don't match. Stream 6 (firmware glue) avoided the macro to dodge the inconsistency, but it's still a footgun.
@ -84,6 +76,8 @@ The original a-immersive was mobile-first ("designed for touch / foldable phone
## Recently resolved (delete after a few weeks) ## Recently resolved (delete after a few weeks)
- 2026-07-14: Defect "WASM MLP architecture is fixed" resolved by one-core-engine P2: `nisps/ml/` is storage-policied (`MLPCore<Storage>`); the browser MLP is runtime-shaped (`DynamicStorage`), `nisps_ml_create` honours dims, `nisps_ml_reshape` warm-starts. Firmware keeps the zero-heap fixed template (`.text` +0.30%, within contract).
- 2026-04-29: Three-implementation ML duplication (firmware `memlp`, `nisps-core`, JS engine) collapsed to single `nisps/` C++ codebase. - 2026-04-29: Three-implementation ML duplication (firmware `memlp`, `nisps-core`, JS engine) collapsed to single `nisps/` C++ codebase.
- 2026-04-29: Firmware mode forks (~280400 lines duplicated across 8 modes) collapsed via `nisps/modes/base.hpp` CRTP scaffold; concrete modes are now ~50130 lines. - 2026-04-29: Firmware mode forks (~280400 lines duplicated across 8 modes) collapsed via `nisps/modes/base.hpp` CRTP scaffold; concrete modes are now ~50130 lines.
- 2026-04-29: meml-ues double-scaling MSE bug fixed in `nisps/ml/loss.hpp` + `mlp.hpp`. - 2026-04-29: meml-ues double-scaling MSE bug fixed in `nisps/ml/loss.hpp` + `mlp.hpp`.

2
MAP.md
View file

@ -143,7 +143,7 @@ the "BUILD DELTAS" block at the top of `docs/specs/vcv-module.md`). `src/MEMLNau
- C++ identifiers: `PascalCase` types, `snake_case` functions/variables, `kPascalCase` constexpr. JSON keys `snake_case`. TS types `PascalCase`, components `PascalCase.tsx`, modules `kebab-case.ts`. - C++ identifiers: `PascalCase` types, `snake_case` functions/variables, `kPascalCase` constexpr. JSON keys `snake_case`. TS types `PascalCase`, components `PascalCase.tsx`, modules `kebab-case.ts`.
- `Curve` enum lives in `nisps/core/math.hpp` (lowercase: `linear/exp/log/square/sqrt/sigmoid/cubic`); generated mode headers re-export via `using Curve = ::nisps::Curve;`. TS mirror at `manifold/src/engine/curves.ts` with same names (moves into core at P4). - `Curve` enum lives in `nisps/core/math.hpp` (lowercase: `linear/exp/log/square/sqrt/sigmoid/cubic`); generated mode headers re-export via `using Curve = ::nisps::Curve;`. TS mirror at `manifold/src/engine/curves.ts` with same names (moves into core at P4).
- Modes are TSX components composed of primitives; mode parameter contracts are JSON schemas with codegen → C++ types (TS codegen returns at P5). **No declarative JSON UI.** - Modes are TSX components composed of primitives; mode parameter contracts are JSON schemas with codegen → C++ types (TS codegen returns at P5). **No declarative JSON UI.**
- WASM and firmware share the same C++; WASM is fixed at `MLP<32, 10, 14, 18, 126>` (`nisps_ml_create` ignores requested dims — see `plans/one-core-engine-refactor.md` P2) and modes use a slice of outputs based on schema's `output_size`. - WASM and firmware share the same C++; the browser MLP is runtime-shaped (`MLPCore<DynamicStorage>`, since P2): `nisps_ml_create` honours `(input, output, hidden[3])` with non-positive/null args defaulting to `32→[10,14,18]→126`; `nisps_ml_reshape` warm-starts a new shape. Firmware keeps compile-time `MLP<...>` (zero heap). Modes currently still use a slice of the default 126 outputs (per-mode dims become schema-real at P5).
- Cross-platform parity: `scripts/parity-check.sh` enforces native vs WASM agreement within 1e-5. - Cross-platform parity: `scripts/parity-check.sh` enforces native vs WASM agreement within 1e-5.
## Gotchas ## Gotchas

View file

@ -114,12 +114,11 @@ Two WASM instances at runtime:
C API is in `nisps/wasm/bindings.cpp`. Build: `bash scripts/build-wasm.sh` (~94KB output to `manifold/public/`). C API is in `nisps/wasm/bindings.cpp`. Build: `bash scripts/build-wasm.sh` (~94KB output to `manifold/public/`).
The WASM target is fixed at `MLP<32u, 10u, 14u, 18u, 126u>` (`nisps_ml_create` ignores requested dims — see `docs/specs/plans/one-core-engine-refactor.md` P2). Modes with smaller `input_size`/`output_size` use a slice. The browser MLP is runtime-shaped since P2 (`MLPCore<DynamicStorage>`): `nisps_ml_create` honours `(input, output, hidden[3])`; non-positive/null args default to `32→[10,14,18]→126`. `nisps_ml_reshape` swaps in a new shape warm-started from the overlapping weights (examples + feedback state reset). Modes currently still slice the default 126 outputs; per-mode dims become schema-real at P5.
### Known limitations ### Known limitations
- Loss history not yet plumbed through C API; only the final loss of a training run reaches TS. - Loss history not yet plumbed through C API; only the final loss of a training run reaches TS.
- Engine MLP architecture is fixed at compile time — runtime-shaped browser MLP arrives at P2 of `docs/specs/plans/one-core-engine-refactor.md`.
- Mic input through the worklet for XIASRI / SoundAnalysisMIDI is not wired in manifold. - Mic input through the worklet for XIASRI / SoundAnalysisMIDI is not wired in manifold.
- C15 has no home on main (see `ALIGNMENT.md` defect #1). - C15 has no home on main (see `ALIGNMENT.md` defect #1).
- The browser Jolt/OU controls in manifold reimplement the gesture math in TS (interim, ported from the retired playground) rather than calling the C++ `ml::Jolt`/`ml::OUNoise` through WASM. They drive weights via the existing `nisps_ml_get/set_weights` bindings — the P3 phase of the one-core-engine plan replaces them with `nisps_ml_jolt_press/release` + `nisps_ml_explore_intensity` bindings. - The browser Jolt/OU controls in manifold reimplement the gesture math in TS (interim, ported from the retired playground) rather than calling the C++ `ml::Jolt`/`ml::OUNoise` through WASM. They drive weights via the existing `nisps_ml_get/set_weights` bindings — the P3 phase of the one-core-engine plan replaces them with `nisps_ml_jolt_press/release` + `nisps_ml_explore_intensity` bindings.

View file

@ -99,7 +99,7 @@ Each phase ends green on its test gate and is independently landable. File phase
- **Gate met:** full `run-all-tests.sh` green with playground gone (ctest 4/4, parity PASS, lint clean, - **Gate met:** full `run-all-tests.sh` green with playground gone (ctest 4/4, parity PASS, lint clean,
manifold 9 unit + 20 e2e). manifold 9 unit + 20 e2e).
### P2 — Storage-policy split: templated hardware, dynamic browser (the structural centre, ≈1 wk) ### P2 — Storage-policy split: templated hardware, dynamic browser (the structural centre, ≈1 wk) — ✅ landed 2026-07-14 (hardware timing spot-check of chokepoint B pending physical MEMLNaut)
- ✅ (landed 2026-07-13) Refactor `nisps/ml/` so algorithms (forward, backprop/SGD, init, `move_weights`) - ✅ (landed 2026-07-13) Refactor `nisps/ml/` so algorithms (forward, backprop/SGD, init, `move_weights`)
are written once against a storage policy (`mlp.hpp` `MLPCore<Storage>`; jolt/OU/feedback already operate are written once against a storage policy (`mlp.hpp` `MLPCore<Storage>`; jolt/OU/feedback already operate
@ -118,11 +118,16 @@ Each phase ends green on its test gate and is independently landable. File phase
firmware/tests, `DynamicFeedbackStorage` for the browser). `nisps_ml_describe` now takes the handle firmware/tests, `DynamicFeedbackStorage` for the browser). `nisps_ml_describe` now takes the handle
(null → default shape). Verified: reshape ABI smoke (dims honoured, overlap survives, invalid dims (null → default shape). Verified: reshape ABI smoke (dims honoured, overlap survives, invalid dims
rejected), warm-start ctest (grow+shrink), parity PASS unchanged, firmware `.text` unchanged. rejected), warm-start ctest (grow+shrink), parity PASS unchanged, firmware `.text` unchanged.
- Manifold drops input clamping/phantom-channel handling; XIASRI/sound-analysis multi-input modes become - ✅ (landed 2026-07-14) Manifold reshape wiring: `WasmIML.reshape` + `EngineApi.reshape` (buffers/worker/
browser-viable. spine follow the new dims; training-worker protocol carries `hidden`); reset-on-reshape confirm modal
- **Gate:** parity — fixed and dynamic storage produce bit-identical outputs for identical shapes/seeds offered on genuine input-layout changes (default 32-in over-provisioned head untouched on load/decline);
(new ctest); `parity-check.sh` native↔WASM ≤1e-5 unchanged; firmware builds byte-comparable (chokepoint B: stale even/odd-blend copy deleted. The mean-blend clamping itself was already gone (dedicated dims);
compile PAFSynth, compare `.text` size ±1%). the `min(n, inputSize)` guard + zero-padding stay as the legitimate declined-reshape behaviour.
XIASRI/sound-analysis multi-input modes become browser-viable (schema-real dims arrive at P5).
- **Gate met:** fixed↔dynamic bit-identical ctest (+ warm-start grow/shrink ctest + reshape ABI smoke);
`parity-check.sh` native↔WASM PASS unchanged (2.4e-7); PAFSynth `.text` 122324→122692 (+0.30%, within
±1%); manifold 9 unit + 25 e2e. Hardware audio-callback timing check deferred to the chokepoint-B
hardware session (operator, physical MEMLNaut).
### P3 — Exploration + feedback fully in core (≈34 days) ### P3 — Exploration + feedback fully in core (≈34 days)

View file

@ -28,30 +28,23 @@ Both files are committed (so the browser apps work from a fresh clone
without a C++ toolchain). Re-run `build-wasm.sh` after changes to without a C++ toolchain). Re-run `build-wasm.sh` after changes to
`nisps/{core,ml,engines,wasm}`. `nisps/{core,ml,engines,wasm}`.
## Architecture limit (read this) ## Architecture (runtime-shaped since one-core-engine P2)
The MLP class template is parametrised on `(input_size, hidden1, hidden2, The browser MLP is `MLPCore<DynamicStorage>` (`nisps/ml/dynamic_storage.hpp`):
hidden3, output_size)`. WASM cannot recompile templates at runtime, so `nisps_ml_create(input, output, hidden[3], n, seed)` HONOURS its dimensions.
this build instantiates exactly ONE configuration: The 4-layer topology (ReLU×3 + Sigmoid) is fixed; only the dimensions are
runtime, capped at 4096 per dim. Non-positive/null arguments fall back to the
historical defaults:
nisps::ml::MLP<32, 10, 14, 18, 126> 32 inputs → [10, 14, 18] hidden → 126 outputs
That serves the browser use case (up to 32 input axes → up to 126 synth `nisps_ml_reshape(ml, in, out, hidden, n, spread)` constructs a new net at
parameters). `nisps_ml_create()` accepts caller-supplied dimensions for the requested shape, warm-starts it by copying the overlapping weight region
forward compatibility but currently ignores them — see comment at the top (`nisps/ml/warm_start.hpp`), and swaps it in. The C-side dataset and the
of `bindings.cpp`. The schemas in `schemas/modes/*.json` use up to feedback controller state RESET on reshape (front-end shows a confirm modal).
`output_size=126`; modes whose output_size is < 126 simply ignore the Heap is used only at create/reshape time, never per-call; the firmware target
trailing entries. never compiles the dynamic storage at all (`#error` under
`NISPS_TARGET_EMBEDDED`).
To support additional architectures, either:
1. Compile multiple wasm modules (`nisps_small.wasm`,
`nisps_default.wasm`, …) and let the playground load the right one
based on the active mode.
2. Add a runtime-shape MLP variant to `nisps/ml` (heap allocation only at
`create()`; no impact on hot paths).
Both options are deferred to a future stream.
## C API surface ## C API surface
@ -59,7 +52,7 @@ See `bindings.cpp` for the full list. Summary:
| Group | Functions | | Group | Functions |
|-----------|------------------------------------------------------------------------------| |-----------|------------------------------------------------------------------------------|
| ML life | `nisps_ml_create`, `nisps_ml_destroy`, `nisps_ml_reset` | | ML life | `nisps_ml_create`, `nisps_ml_destroy`, `nisps_ml_reset`, `nisps_ml_reshape` |
| ML I/O | `nisps_ml_set_input`, `nisps_ml_process`, `nisps_ml_outputs`, `nisps_ml_infer_batch` | | ML I/O | `nisps_ml_set_input`, `nisps_ml_process`, `nisps_ml_outputs`, `nisps_ml_infer_batch` |
| Training | `nisps_ml_add_example`, `nisps_ml_train`, `nisps_ml_eval_loss`, `nisps_ml_clear_examples`, `nisps_ml_example_count` | | Training | `nisps_ml_add_example`, `nisps_ml_train`, `nisps_ml_eval_loss`, `nisps_ml_clear_examples`, `nisps_ml_example_count` |
| Weights | `nisps_ml_weight_count`, `nisps_ml_get_weights`, `nisps_ml_set_weights`, `nisps_ml_draw_weights`, `nisps_ml_move_weights` | | Weights | `nisps_ml_weight_count`, `nisps_ml_get_weights`, `nisps_ml_set_weights`, `nisps_ml_draw_weights`, `nisps_ml_move_weights` |