docs: P2 complete — burn down plan, close ALIGNMENT defect #3 (fixed WASM arity)

- plan P2 marked landed (hardware timing spot-check deferred to the
  chokepoint-B session on physical MEMLNaut)
- ALIGNMENT: defect #3 resolved by the runtime-shaped browser MLP
- MAP/AGENT-REFERENCE/nisps-wasm-README: describe the honoured-dims
  create + reshape ABI; note per-mode dims become schema-real at P5
- fresh wasm artifacts from the gate run
This commit is contained in:
monkey-w1n5t0n 2026-07-14 03:56:04 +02:00
parent 7e4dc26f6e
commit 7e957457cd
5 changed files with 30 additions and 39 deletions

View file

@ -26,14 +26,6 @@ The clean-slate rewrite (2026-04-29) consolidated everything into one C++20 code
**Rough cost.** Half a day. Add `nisps_ml_train_with_history` (or extend the existing call) returning a pointer to the loss array; copy on the JS side.
### 3. WASM MLP architecture is fixed (2026-04-29)
**What.** WASM compiles `MLP<2, 10, 14, 18, 126>` only. Modes with smaller `output_size` use a slice; modes that want different hidden layer shapes (e.g. `[10, 10, 14]` for smaller modes) can't get them in the browser.
**Why it blocks the mission.** Per-mode ML architecture variation is one of our four "research dimensions". The fix isn't urgent for the current mode set (all schemas are within the universal arch's capacity), but the moment we want to experiment with bigger networks or different shapes, this hits.
**Rough cost.** Medium. Either compile multiple WASM modules (one per arch shape; load on demand) or move to a runtime-shaped MLP (loses some compile-time perf). Templates-vs-runtime is a research-vs-firmware-perf tradeoff worth a separate decision doc.
### 4. `NISPS_AUDIO_FUNC` host fallback is misshapen (2026-04-29)
**What.** `nisps/core/perf.hpp` defines `NISPS_AUDIO_FUNC(decl) decl` for the host but the firmware path `__not_in_flash_func(name)` takes only a function name (it stringifies into a section attribute). The two forms don't match. Stream 6 (firmware glue) avoided the macro to dodge the inconsistency, but it's still a footgun.
@ -84,6 +76,8 @@ The original a-immersive was mobile-first ("designed for touch / foldable phone
## Recently resolved (delete after a few weeks)
- 2026-07-14: Defect "WASM MLP architecture is fixed" resolved by one-core-engine P2: `nisps/ml/` is storage-policied (`MLPCore<Storage>`); the browser MLP is runtime-shaped (`DynamicStorage`), `nisps_ml_create` honours dims, `nisps_ml_reshape` warm-starts. Firmware keeps the zero-heap fixed template (`.text` +0.30%, within contract).
- 2026-04-29: Three-implementation ML duplication (firmware `memlp`, `nisps-core`, JS engine) collapsed to single `nisps/` C++ codebase.
- 2026-04-29: Firmware mode forks (~280400 lines duplicated across 8 modes) collapsed via `nisps/modes/base.hpp` CRTP scaffold; concrete modes are now ~50130 lines.
- 2026-04-29: meml-ues double-scaling MSE bug fixed in `nisps/ml/loss.hpp` + `mlp.hpp`.

2
MAP.md
View file

@ -143,7 +143,7 @@ the "BUILD DELTAS" block at the top of `docs/specs/vcv-module.md`). `src/MEMLNau
- C++ identifiers: `PascalCase` types, `snake_case` functions/variables, `kPascalCase` constexpr. JSON keys `snake_case`. TS types `PascalCase`, components `PascalCase.tsx`, modules `kebab-case.ts`.
- `Curve` enum lives in `nisps/core/math.hpp` (lowercase: `linear/exp/log/square/sqrt/sigmoid/cubic`); generated mode headers re-export via `using Curve = ::nisps::Curve;`. TS mirror at `manifold/src/engine/curves.ts` with same names (moves into core at P4).
- Modes are TSX components composed of primitives; mode parameter contracts are JSON schemas with codegen → C++ types (TS codegen returns at P5). **No declarative JSON UI.**
- WASM and firmware share the same C++; WASM is fixed at `MLP<32, 10, 14, 18, 126>` (`nisps_ml_create` ignores requested dims — see `plans/one-core-engine-refactor.md` P2) and modes use a slice of outputs based on schema's `output_size`.
- WASM and firmware share the same C++; the browser MLP is runtime-shaped (`MLPCore<DynamicStorage>`, since P2): `nisps_ml_create` honours `(input, output, hidden[3])` with non-positive/null args defaulting to `32→[10,14,18]→126`; `nisps_ml_reshape` warm-starts a new shape. Firmware keeps compile-time `MLP<...>` (zero heap). Modes currently still use a slice of the default 126 outputs (per-mode dims become schema-real at P5).
- Cross-platform parity: `scripts/parity-check.sh` enforces native vs WASM agreement within 1e-5.
## Gotchas

View file

@ -114,12 +114,11 @@ Two WASM instances at runtime:
C API is in `nisps/wasm/bindings.cpp`. Build: `bash scripts/build-wasm.sh` (~94KB output to `manifold/public/`).
The WASM target is fixed at `MLP<32u, 10u, 14u, 18u, 126u>` (`nisps_ml_create` ignores requested dims — see `docs/specs/plans/one-core-engine-refactor.md` P2). Modes with smaller `input_size`/`output_size` use a slice.
The browser MLP is runtime-shaped since P2 (`MLPCore<DynamicStorage>`): `nisps_ml_create` honours `(input, output, hidden[3])`; non-positive/null args default to `32→[10,14,18]→126`. `nisps_ml_reshape` swaps in a new shape warm-started from the overlapping weights (examples + feedback state reset). Modes currently still slice the default 126 outputs; per-mode dims become schema-real at P5.
### Known limitations
- Loss history not yet plumbed through C API; only the final loss of a training run reaches TS.
- Engine MLP architecture is fixed at compile time — runtime-shaped browser MLP arrives at P2 of `docs/specs/plans/one-core-engine-refactor.md`.
- Mic input through the worklet for XIASRI / SoundAnalysisMIDI is not wired in manifold.
- C15 has no home on main (see `ALIGNMENT.md` defect #1).
- The browser Jolt/OU controls in manifold reimplement the gesture math in TS (interim, ported from the retired playground) rather than calling the C++ `ml::Jolt`/`ml::OUNoise` through WASM. They drive weights via the existing `nisps_ml_get/set_weights` bindings — the P3 phase of the one-core-engine plan replaces them with `nisps_ml_jolt_press/release` + `nisps_ml_explore_intensity` bindings.

View file

@ -99,7 +99,7 @@ Each phase ends green on its test gate and is independently landable. File phase
- **Gate met:** full `run-all-tests.sh` green with playground gone (ctest 4/4, parity PASS, lint clean,
manifold 9 unit + 20 e2e).
### P2 — Storage-policy split: templated hardware, dynamic browser (the structural centre, ≈1 wk)
### P2 — Storage-policy split: templated hardware, dynamic browser (the structural centre, ≈1 wk) — ✅ landed 2026-07-14 (hardware timing spot-check of chokepoint B pending physical MEMLNaut)
- ✅ (landed 2026-07-13) Refactor `nisps/ml/` so algorithms (forward, backprop/SGD, init, `move_weights`)
are written once against a storage policy (`mlp.hpp` `MLPCore<Storage>`; jolt/OU/feedback already operate
@ -118,11 +118,16 @@ Each phase ends green on its test gate and is independently landable. File phase
firmware/tests, `DynamicFeedbackStorage` for the browser). `nisps_ml_describe` now takes the handle
(null → default shape). Verified: reshape ABI smoke (dims honoured, overlap survives, invalid dims
rejected), warm-start ctest (grow+shrink), parity PASS unchanged, firmware `.text` unchanged.
- Manifold drops input clamping/phantom-channel handling; XIASRI/sound-analysis multi-input modes become
browser-viable.
- **Gate:** parity — fixed and dynamic storage produce bit-identical outputs for identical shapes/seeds
(new ctest); `parity-check.sh` native↔WASM ≤1e-5 unchanged; firmware builds byte-comparable (chokepoint B:
compile PAFSynth, compare `.text` size ±1%).
- ✅ (landed 2026-07-14) Manifold reshape wiring: `WasmIML.reshape` + `EngineApi.reshape` (buffers/worker/
spine follow the new dims; training-worker protocol carries `hidden`); reset-on-reshape confirm modal
offered on genuine input-layout changes (default 32-in over-provisioned head untouched on load/decline);
stale even/odd-blend copy deleted. The mean-blend clamping itself was already gone (dedicated dims);
the `min(n, inputSize)` guard + zero-padding stay as the legitimate declined-reshape behaviour.
XIASRI/sound-analysis multi-input modes become browser-viable (schema-real dims arrive at P5).
- **Gate met:** fixed↔dynamic bit-identical ctest (+ warm-start grow/shrink ctest + reshape ABI smoke);
`parity-check.sh` native↔WASM PASS unchanged (2.4e-7); PAFSynth `.text` 122324→122692 (+0.30%, within
±1%); manifold 9 unit + 25 e2e. Hardware audio-callback timing check deferred to the chokepoint-B
hardware session (operator, physical MEMLNaut).
### P3 — Exploration + feedback fully in core (≈34 days)

View file

@ -28,30 +28,23 @@ Both files are committed (so the browser apps work from a fresh clone
without a C++ toolchain). Re-run `build-wasm.sh` after changes to
`nisps/{core,ml,engines,wasm}`.
## Architecture limit (read this)
## Architecture (runtime-shaped since one-core-engine P2)
The MLP class template is parametrised on `(input_size, hidden1, hidden2,
hidden3, output_size)`. WASM cannot recompile templates at runtime, so
this build instantiates exactly ONE configuration:
The browser MLP is `MLPCore<DynamicStorage>` (`nisps/ml/dynamic_storage.hpp`):
`nisps_ml_create(input, output, hidden[3], n, seed)` HONOURS its dimensions.
The 4-layer topology (ReLU×3 + Sigmoid) is fixed; only the dimensions are
runtime, capped at 4096 per dim. Non-positive/null arguments fall back to the
historical defaults:
nisps::ml::MLP<32, 10, 14, 18, 126>
32 inputs → [10, 14, 18] hidden → 126 outputs
That serves the browser use case (up to 32 input axes → up to 126 synth
parameters). `nisps_ml_create()` accepts caller-supplied dimensions for
forward compatibility but currently ignores them — see comment at the top
of `bindings.cpp`. The schemas in `schemas/modes/*.json` use up to
`output_size=126`; modes whose output_size is < 126 simply ignore the
trailing entries.
To support additional architectures, either:
1. Compile multiple wasm modules (`nisps_small.wasm`,
`nisps_default.wasm`, …) and let the playground load the right one
based on the active mode.
2. Add a runtime-shape MLP variant to `nisps/ml` (heap allocation only at
`create()`; no impact on hot paths).
Both options are deferred to a future stream.
`nisps_ml_reshape(ml, in, out, hidden, n, spread)` constructs a new net at
the requested shape, warm-starts it by copying the overlapping weight region
(`nisps/ml/warm_start.hpp`), and swaps it in. The C-side dataset and the
feedback controller state RESET on reshape (front-end shows a confirm modal).
Heap is used only at create/reshape time, never per-call; the firmware target
never compiles the dynamic storage at all (`#error` under
`NISPS_TARGET_EMBEDDED`).
## C API surface
@ -59,7 +52,7 @@ See `bindings.cpp` for the full list. Summary:
| Group | Functions |
|-----------|------------------------------------------------------------------------------|
| ML life | `nisps_ml_create`, `nisps_ml_destroy`, `nisps_ml_reset` |
| ML life | `nisps_ml_create`, `nisps_ml_destroy`, `nisps_ml_reset`, `nisps_ml_reshape` |
| ML I/O | `nisps_ml_set_input`, `nisps_ml_process`, `nisps_ml_outputs`, `nisps_ml_infer_batch` |
| Training | `nisps_ml_add_example`, `nisps_ml_train`, `nisps_ml_eval_loss`, `nisps_ml_clear_examples`, `nisps_ml_example_count` |
| Weights | `nisps_ml_weight_count`, `nisps_ml_get_weights`, `nisps_ml_set_weights`, `nisps_ml_draw_weights`, `nisps_ml_move_weights` |