diff --git a/ALIGNMENT.md b/ALIGNMENT.md index db1fcdf..5699a96 100644 --- a/ALIGNMENT.md +++ b/ALIGNMENT.md @@ -26,14 +26,6 @@ The clean-slate rewrite (2026-04-29) consolidated everything into one C++20 code **Rough cost.** Half a day. Add `nisps_ml_train_with_history` (or extend the existing call) returning a pointer to the loss array; copy on the JS side. -### 3. WASM MLP architecture is fixed (2026-04-29) - -**What.** WASM compiles `MLP<2, 10, 14, 18, 126>` only. Modes with smaller `output_size` use a slice; modes that want different hidden layer shapes (e.g. `[10, 10, 14]` for smaller modes) can't get them in the browser. - -**Why it blocks the mission.** Per-mode ML architecture variation is one of our four "research dimensions". The fix isn't urgent for the current mode set (all schemas are within the universal arch's capacity), but the moment we want to experiment with bigger networks or different shapes, this hits. - -**Rough cost.** Medium. Either compile multiple WASM modules (one per arch shape; load on demand) or move to a runtime-shaped MLP (loses some compile-time perf). Templates-vs-runtime is a research-vs-firmware-perf tradeoff worth a separate decision doc. - ### 4. `NISPS_AUDIO_FUNC` host fallback is misshapen (2026-04-29) **What.** `nisps/core/perf.hpp` defines `NISPS_AUDIO_FUNC(decl) decl` for the host but the firmware path `__not_in_flash_func(name)` takes only a function name (it stringifies into a section attribute). The two forms don't match. Stream 6 (firmware glue) avoided the macro to dodge the inconsistency, but it's still a footgun. @@ -84,6 +76,8 @@ The original a-immersive was mobile-first ("designed for touch / foldable phone ## Recently resolved (delete after a few weeks) +- 2026-07-14: Defect "WASM MLP architecture is fixed" resolved by one-core-engine P2: `nisps/ml/` is storage-policied (`MLPCore`); the browser MLP is runtime-shaped (`DynamicStorage`), `nisps_ml_create` honours dims, `nisps_ml_reshape` warm-starts. Firmware keeps the zero-heap fixed template (`.text` +0.30%, within contract). + - 2026-04-29: Three-implementation ML duplication (firmware `memlp`, `nisps-core`, JS engine) collapsed to single `nisps/` C++ codebase. - 2026-04-29: Firmware mode forks (~280–400 lines duplicated across 8 modes) collapsed via `nisps/modes/base.hpp` CRTP scaffold; concrete modes are now ~50–130 lines. - 2026-04-29: meml-ues double-scaling MSE bug fixed in `nisps/ml/loss.hpp` + `mlp.hpp`. diff --git a/MAP.md b/MAP.md index cfc081f..2781521 100644 --- a/MAP.md +++ b/MAP.md @@ -143,7 +143,7 @@ the "BUILD DELTAS" block at the top of `docs/specs/vcv-module.md`). `src/MEMLNau - C++ identifiers: `PascalCase` types, `snake_case` functions/variables, `kPascalCase` constexpr. JSON keys `snake_case`. TS types `PascalCase`, components `PascalCase.tsx`, modules `kebab-case.ts`. - `Curve` enum lives in `nisps/core/math.hpp` (lowercase: `linear/exp/log/square/sqrt/sigmoid/cubic`); generated mode headers re-export via `using Curve = ::nisps::Curve;`. TS mirror at `manifold/src/engine/curves.ts` with same names (moves into core at P4). - Modes are TSX components composed of primitives; mode parameter contracts are JSON schemas with codegen → C++ types (TS codegen returns at P5). **No declarative JSON UI.** -- WASM and firmware share the same C++; WASM is fixed at `MLP<32, 10, 14, 18, 126>` (`nisps_ml_create` ignores requested dims — see `plans/one-core-engine-refactor.md` P2) and modes use a slice of outputs based on schema's `output_size`. +- WASM and firmware share the same C++; the browser MLP is runtime-shaped (`MLPCore`, since P2): `nisps_ml_create` honours `(input, output, hidden[3])` with non-positive/null args defaulting to `32→[10,14,18]→126`; `nisps_ml_reshape` warm-starts a new shape. Firmware keeps compile-time `MLP<...>` (zero heap). Modes currently still use a slice of the default 126 outputs (per-mode dims become schema-real at P5). - Cross-platform parity: `scripts/parity-check.sh` enforces native vs WASM agreement within 1e-5. ## Gotchas diff --git a/docs/AGENT-REFERENCE.md b/docs/AGENT-REFERENCE.md index 894433d..c811574 100644 --- a/docs/AGENT-REFERENCE.md +++ b/docs/AGENT-REFERENCE.md @@ -114,12 +114,11 @@ Two WASM instances at runtime: C API is in `nisps/wasm/bindings.cpp`. Build: `bash scripts/build-wasm.sh` (~94KB output to `manifold/public/`). -The WASM target is fixed at `MLP<32u, 10u, 14u, 18u, 126u>` (`nisps_ml_create` ignores requested dims — see `docs/specs/plans/one-core-engine-refactor.md` P2). Modes with smaller `input_size`/`output_size` use a slice. +The browser MLP is runtime-shaped since P2 (`MLPCore`): `nisps_ml_create` honours `(input, output, hidden[3])`; non-positive/null args default to `32→[10,14,18]→126`. `nisps_ml_reshape` swaps in a new shape warm-started from the overlapping weights (examples + feedback state reset). Modes currently still slice the default 126 outputs; per-mode dims become schema-real at P5. ### Known limitations - Loss history not yet plumbed through C API; only the final loss of a training run reaches TS. -- Engine MLP architecture is fixed at compile time — runtime-shaped browser MLP arrives at P2 of `docs/specs/plans/one-core-engine-refactor.md`. - Mic input through the worklet for XIASRI / SoundAnalysisMIDI is not wired in manifold. - C15 has no home on main (see `ALIGNMENT.md` defect #1). - The browser Jolt/OU controls in manifold reimplement the gesture math in TS (interim, ported from the retired playground) rather than calling the C++ `ml::Jolt`/`ml::OUNoise` through WASM. They drive weights via the existing `nisps_ml_get/set_weights` bindings — the P3 phase of the one-core-engine plan replaces them with `nisps_ml_jolt_press/release` + `nisps_ml_explore_intensity` bindings. diff --git a/docs/specs/plans/one-core-engine-refactor.md b/docs/specs/plans/one-core-engine-refactor.md index 88cd167..b496f29 100644 --- a/docs/specs/plans/one-core-engine-refactor.md +++ b/docs/specs/plans/one-core-engine-refactor.md @@ -99,7 +99,7 @@ Each phase ends green on its test gate and is independently landable. File phase - **Gate met:** full `run-all-tests.sh` green with playground gone (ctest 4/4, parity PASS, lint clean, manifold 9 unit + 20 e2e). -### P2 — Storage-policy split: templated hardware, dynamic browser (the structural centre, ≈1 wk) +### P2 — Storage-policy split: templated hardware, dynamic browser (the structural centre, ≈1 wk) — ✅ landed 2026-07-14 (hardware timing spot-check of chokepoint B pending physical MEMLNaut) - ✅ (landed 2026-07-13) Refactor `nisps/ml/` so algorithms (forward, backprop/SGD, init, `move_weights`) are written once against a storage policy (`mlp.hpp` `MLPCore`; jolt/OU/feedback already operate @@ -118,11 +118,16 @@ Each phase ends green on its test gate and is independently landable. File phase firmware/tests, `DynamicFeedbackStorage` for the browser). `nisps_ml_describe` now takes the handle (null → default shape). Verified: reshape ABI smoke (dims honoured, overlap survives, invalid dims rejected), warm-start ctest (grow+shrink), parity PASS unchanged, firmware `.text` unchanged. -- Manifold drops input clamping/phantom-channel handling; XIASRI/sound-analysis multi-input modes become - browser-viable. -- **Gate:** parity — fixed and dynamic storage produce bit-identical outputs for identical shapes/seeds - (new ctest); `parity-check.sh` native↔WASM ≤1e-5 unchanged; firmware builds byte-comparable (chokepoint B: - compile PAFSynth, compare `.text` size ±1%). +- ✅ (landed 2026-07-14) Manifold reshape wiring: `WasmIML.reshape` + `EngineApi.reshape` (buffers/worker/ + spine follow the new dims; training-worker protocol carries `hidden`); reset-on-reshape confirm modal + offered on genuine input-layout changes (default 32-in over-provisioned head untouched on load/decline); + stale even/odd-blend copy deleted. The mean-blend clamping itself was already gone (dedicated dims); + the `min(n, inputSize)` guard + zero-padding stay as the legitimate declined-reshape behaviour. + XIASRI/sound-analysis multi-input modes become browser-viable (schema-real dims arrive at P5). +- **Gate met:** fixed↔dynamic bit-identical ctest (+ warm-start grow/shrink ctest + reshape ABI smoke); + `parity-check.sh` native↔WASM PASS unchanged (2.4e-7); PAFSynth `.text` 122324→122692 (+0.30%, within + ±1%); manifold 9 unit + 25 e2e. Hardware audio-callback timing check deferred to the chokepoint-B + hardware session (operator, physical MEMLNaut). ### P3 — Exploration + feedback fully in core (≈3–4 days) diff --git a/nisps/wasm/README.md b/nisps/wasm/README.md index 07f0d68..6438cb9 100644 --- a/nisps/wasm/README.md +++ b/nisps/wasm/README.md @@ -28,30 +28,23 @@ Both files are committed (so the browser apps work from a fresh clone without a C++ toolchain). Re-run `build-wasm.sh` after changes to `nisps/{core,ml,engines,wasm}`. -## Architecture limit (read this) +## Architecture (runtime-shaped since one-core-engine P2) -The MLP class template is parametrised on `(input_size, hidden1, hidden2, -hidden3, output_size)`. WASM cannot recompile templates at runtime, so -this build instantiates exactly ONE configuration: +The browser MLP is `MLPCore` (`nisps/ml/dynamic_storage.hpp`): +`nisps_ml_create(input, output, hidden[3], n, seed)` HONOURS its dimensions. +The 4-layer topology (ReLU×3 + Sigmoid) is fixed; only the dimensions are +runtime, capped at 4096 per dim. Non-positive/null arguments fall back to the +historical defaults: - nisps::ml::MLP<32, 10, 14, 18, 126> + 32 inputs → [10, 14, 18] hidden → 126 outputs -That serves the browser use case (up to 32 input axes → up to 126 synth -parameters). `nisps_ml_create()` accepts caller-supplied dimensions for -forward compatibility but currently ignores them — see comment at the top -of `bindings.cpp`. The schemas in `schemas/modes/*.json` use up to -`output_size=126`; modes whose output_size is < 126 simply ignore the -trailing entries. - -To support additional architectures, either: - -1. Compile multiple wasm modules (`nisps_small.wasm`, - `nisps_default.wasm`, …) and let the playground load the right one - based on the active mode. -2. Add a runtime-shape MLP variant to `nisps/ml` (heap allocation only at - `create()`; no impact on hot paths). - -Both options are deferred to a future stream. +`nisps_ml_reshape(ml, in, out, hidden, n, spread)` constructs a new net at +the requested shape, warm-starts it by copying the overlapping weight region +(`nisps/ml/warm_start.hpp`), and swaps it in. The C-side dataset and the +feedback controller state RESET on reshape (front-end shows a confirm modal). +Heap is used only at create/reshape time, never per-call; the firmware target +never compiles the dynamic storage at all (`#error` under +`NISPS_TARGET_EMBEDDED`). ## C API surface @@ -59,7 +52,7 @@ See `bindings.cpp` for the full list. Summary: | Group | Functions | |-----------|------------------------------------------------------------------------------| -| ML life | `nisps_ml_create`, `nisps_ml_destroy`, `nisps_ml_reset` | +| ML life | `nisps_ml_create`, `nisps_ml_destroy`, `nisps_ml_reset`, `nisps_ml_reshape` | | ML I/O | `nisps_ml_set_input`, `nisps_ml_process`, `nisps_ml_outputs`, `nisps_ml_infer_batch` | | Training | `nisps_ml_add_example`, `nisps_ml_train`, `nisps_ml_eval_loss`, `nisps_ml_clear_examples`, `nisps_ml_example_count` | | Weights | `nisps_ml_weight_count`, `nisps_ml_get_weights`, `nisps_ml_set_weights`, `nisps_ml_draw_weights`, `nisps_ml_move_weights` |