c19d84619e
The §7.5 decision was "rebase our commits onto upstream, then vendor". On inspection there was no rebase to do, and the inventory that recommended it (7a30da9) was wrong on two counts — both corrected in this commit. All three fork commits touch only `examples/`. That directory is not in the sketch symlink forest (firmware/MEMLNaut-NISPS/src/ links exactly audio, hardware, interface, synth, utils, PicoDefs.hpp), so it is never compiled. They existed to let the RL code build against nisps/core *while it was being ported*, and that port is done — nisps/ml/{jolt,ou_noise,feedback,geo_push}.hpp cite the upstream sources directly. Two of the 31 commits I flagged as "work we are missing", `9fcd459 jolts` and `d0d8a72 noise`, are precisely the ones already absorbed. So the question was never rebase-or-not but which snapshot to vendor. Current upstream, because the staleness was already costing us: - upstream has DisplayDriver::NavigateToView; the pin does not. The SelfTest variant called it and had been failing to compile (worked around inb953681). It was written against a newer memllib, not against nothing. - e291192 "l r input swap" is a hardware bug fix: the physical L/R input sockets are wired to opposite codec ADC channels. Every mode on the old pin sees its stereo input backwards. Verified by building all three variants with arduino-cli: SLPWorkshop 145348 flash (+320) 87388 RAM (+4) PAFSynth 145300 flash (+312) 107060 RAM (+4) SelfTest 141840 flash (+320) 12028 RAM (+4) Exactly one compile error: the .ino used kSampleRate in a constexpr, and upstream 1997699 made it a runtime `extern size_t` so a mode can pick its own rate. constexpr -> const; it is a once-per-second diagnostic print. The uniform +316 is the AudioDriver/DisplayDriver deltas — the bulky new upstream code (GrainDelayI16, ReverbI16, ModFXI16, CCSelectView, RLView, VUMeterView, PSRAMManager) is header-only and unreferenced, so the linker drops all of it. .gitmodules points at upstream again: with the pin on an upstream commit, the fork holds nothing the firmware compiles. Phase 0 pointed it at the fork only because b37fc53 existed on no remote. The fork's feat/nisps-core-swap branch stays pushed; nothing is destroyed. Refs: plan §5, §7.5; ALIGNMENT Q4.
115 lines
12 KiB
Markdown
115 lines
12 KiB
Markdown
# ALIGNMENT
|
||
|
||
> Opinionated diagnosis of how well the codebase serves its mission, ranked by impact. Dated entries; remove when resolved rather than checking off. **Pruned every few weeks** — a stale diagnosis is worse than none.
|
||
|
||
## Mission
|
||
|
||
A research platform for interactive ML control of audio. We're building it to figure out what works and what doesn't — different ergonomics and ergodynamics of parameter sets, modes, ML architectures, audio engines, UI, and UX. Therefore: keep most/all parameters tweakable, ML/engine/UI/UX should each be configurable on their own axis, and the codebase has to enable/assist agentic AI coding patterns (confident changes, verifiable without hardware).
|
||
|
||
**Target vision (operator, 2026-07-20):** (1) one C++20 NISPS core serving RP2350 firmware and the browser, performance-sensitive on the MCU; (2) firmware modes runnable as modes in Manifold; (3) Manifold defaults to curated presets/modes, with the maximalist surface behind an "advanced" dev mode used to author them; (4) PlatformIO for hardware, no more .ino; (5) Manifold doubles as interface/editor for the hardware MEMLNaut (settings, presets, training, examples, visualisation).
|
||
|
||
The clean-slate rewrite (2026-04-29) consolidated everything into one C++20 codebase compiling to firmware AND WASM. Since 2026-07-13 (P1) the sole browser app is the React Manifold. JSON schemas remain the firmware↔browser parameter contract. A full-repo audit (2026-07-21, `docs/specs/recon/simplification-audit-2026-07.md`) grounds the entries below; mitigations are phased in `docs/specs/plans/simplification-plan.md`.
|
||
|
||
## Top defects (ranked by mission impact)
|
||
|
||
### 1. The mode layer is not shared: WASM re-orchestrates modes by hand (2026-07-21)
|
||
|
||
**What.** `nisps/modes/` — the CRTP layer binding ML config, engine, voice-space and I/O — compiles only into firmware. `nisps/wasm/bindings.cpp` includes engines and ML primitives but zero mode headers, and Manifold re-assembles mode behaviour (jolt stepping, OU, routing) in TS. "Firmware and WASM share the same modes" is true only at the engine level; every ModeBase behaviour must be mirrored browser-side by hand.
|
||
|
||
**Why it blocks the mission.** Vision bullet 2 is precisely this. Until the control-tick orchestration exists once in C++, every new mode behaviour is a dual implementation with drift risk.
|
||
|
||
**Rough cost.** Spec first, then ~a week: storage-policy the ModeBase orchestration the way P2 did MLPCore (verified shape in plan §6.5a — *not* binding monolithic mode objects, which would contradict the locked two-instance RT architecture). Related honesty gap: Manifold currently catalogues 4 modes that structurally cannot run in the browser (no mic input, event-only engines) — plan §6.5b (absorbs the old C15/mic-input defect; C15 itself lives on `archive/playground-solidjs`).
|
||
|
||
### 2. No curated/advanced split and no in-UI mode picker — the UI fights vision 3 (2026-07-21)
|
||
|
||
**What.** Manifold is 100% dev-maximalist: five drawers of everything, no preset data model to author against, and mode switching exists only via the debug hook — there is no instrument picker in the UI at all (the plumbing, `ctx.modes`/`setModeId`, already exists unused). A stratum of decorative controls (training-param sliders, master volume, bpm, A/B, snapshots, fabricated gradient health) renders real-looking UI that drives nothing.
|
||
|
||
**Why it blocks the mission.** The default experience is supposed to be curated presets; the advanced surface is the authoring tool. Neither exists, and the decorative stratum actively misleads research use.
|
||
|
||
**Rough cost.** Product-model decision first (plan §7.6), then incremental: picker is days; the curated-preset model seeds from `backends/presets.ts` + schemas; disclosure via per-drawer depth levels. Deleting the decorative stratum is part of the Phase-1 sweep.
|
||
|
||
### 3. Arduino-CLI build machinery is actively hostile — vision 4 unstarted (2026-07-21)
|
||
|
||
**What.** The build script sed-mutates the committed `.ino` to select variants (polluting history), the mode list is triple-bookkept (a `NISPS_ST_*` token-paste table is already silently missing the currently-active SLPWorkshop variant), a symlink forest works around Arduino's include rules, and the toolchain globally mutates the installed TFT_eSPI library. Firmware compilation is in no automated gate anywhere.
|
||
|
||
**Why it blocks the mission.** Fragile-by-design builds are the opposite of "confident agentic changes"; the vision names PlatformIO explicitly. `firmware/useq-celium/` already proves the PIO pattern in-repo.
|
||
|
||
**Rough cost.** 2–3 days, one cut (plan §5): env-per-variant `platformio.ini`, delete ~400 lines of hackery, then a firmware CI job. Gated on the memllib ownership decision (plan §7.5).
|
||
|
||
### 4. Manifold-as-hardware-editor is a facade (2026-07-21)
|
||
|
||
**What.** Vision bullet 5 exists as a 237-line Web Serial shell: sound connect lifecycle, zero protocol (`saveModel`/`restoreModel`/`getSettings` are literal stubs), and firmware has no serial command surface or on-device persistence to talk to.
|
||
|
||
**Why it blocks the mission.** The hardware research loop (train on device, inspect/curate in browser) is closed only by this bridge.
|
||
|
||
**Rough cost.** Week+, spec-first (plan §6.5d). The right discipline already exists in-repo: useq-celium's C-header wire truth + TS mirror + parity test; settings payloads should derive from schema codegen.
|
||
|
||
### 5. Dead mass and registry sprawl across every layer (2026-07-21)
|
||
|
||
**What.** *Phase 1 landed 2026-07-21 and removed the bulk of this:* the dead focus/altitude UI system, the decorative control stratum, 12 dead WASM API entries across the 5-file registration chain, the vendored daisysp tree, retired-playground artifacts and root planning relics, 5 unused primitives, the duplicate backend editor and catalogue, `voice_space.hpp`, `fixed_buffer.hpp`, the dead perf-macro regime, and the OSC bridge twin. What remains is the *registry* half: mode identity spread across ~6 hand-maintained registries with demonstrated drift, MLP dims typed twice, per-mode schema blocks hand-written in C++, and assorted stale specs presenting a deleted world as present tense.
|
||
|
||
**Why it blocks the mission.** The registries are dual-truth bugs waiting to fire (one already did: the selftest table). Stale specs are agent-confusing surface area.
|
||
|
||
**Rough cost.** Plan phase 3 (~2–3 days, codegen takes ownership) plus the docs disposition pass (§8). The behaviour bugs found en route (dataset-cap divergence, VCV 2-D input truncation, VCV audio-thread race and JSON) were fixed in phase 2 on 2026-07-21.
|
||
|
||
### 6. No performance measurement despite a performance-defined mission (2026-07-21)
|
||
|
||
**What.** The "super performance-sensitive" constraint is enforced only by static discipline (no-heap lint — itself with proven false negatives — and section attrs, 3/5 of which are dead macros). No benchmark, no CPU-load assertion, no flash/RAM size report on either target; the 16 KB dead buffer was found by reading, not by any gate.
|
||
|
||
**Rough cost.** ~A day for a host-side blocks-per-second benchmark + a per-variant size report in `build-firmware.sh` (plan §6.5f).
|
||
|
||
### 7. Training-health telemetry: one product decision fragmented into four half-features (2026-07-21)
|
||
|
||
**What.** A 16 KB loss-history buffer in every firmware MLP that nothing reads; a WASM worker faking a 1-element loss history; decorative gradient-health UI; and a real `get_layer_stats` API plumbed end-to-end and consumed by nobody.
|
||
|
||
**Why it blocks the mission.** "Is the network learning?" is a core research affordance — currently it *looks* answered while being fake. Decide feature-or-delete once (plan §7.3) and collapse all four limbs accordingly.
|
||
|
||
### 8. RMSProp still deferred from `nisps/ml/` (2026-04-29; reaffirmed 2026-07-21)
|
||
|
||
**What.** `training.hpp` ships SGD only; the legacy firmware used RMSProp for `TrainBatch`. Optimizer choice is a research axis. Not blocking current fits; will matter for harder loss landscapes. Port target: upstream MusicallyEmbodiedML `memlp` (the in-repo `src/memlp` copy is deleted; use the GitHub remote or archive branch).
|
||
|
||
**Rough cost.** A day, plus batch-convergence tests.
|
||
|
||
## Open mission questions
|
||
|
||
### Q1: Per-mode MLP architectures or one shared shape? (2026-04-29)
|
||
|
||
Schemas declare per-mode dims and since P5.3 both targets honour them. Is the mission served by maintaining per-mode shapes (research diversity) or collapsing to one (simpler ops)? Note the audit found all 9 mode schemas share copy-pasted ML defaults and 20 params are anonymous placeholders — the per-mode diversity is currently nominal (plan L40).
|
||
|
||
### Q2: Engine event taxonomy (2026-04-29)
|
||
|
||
`ControlEvent` is a flat enum consumed by the two sequencer modes. Revisit when a third event-emitting mode lands.
|
||
|
||
### Q3: Should Manifold stay desktop-first? (2026-04-29)
|
||
|
||
Legacy a-immersive was mobile-first; Manifold is desktop-first. Defer until user data exists.
|
||
|
||
### Q4: Who owns memllib? — DECIDED, half-executed (2026-07-21)
|
||
|
||
Operator decision: **vendor**, self-contained in this repo. The inventory
|
||
(`docs/specs/recon/memllib-usage-inventory.md`) settled the shape: there is no small load-bearing
|
||
subset — it is all of memllib bar `examples/` (~1.8 MB, 24/24 compiled TUs link). The fork is
|
||
dissolved: its three commits touch only `examples/`, which the firmware never compiles and whose
|
||
content already lives in `nisps/ml/{jolt,ou_noise,feedback,geo_push}.hpp`, so the submodule now
|
||
points at upstream and is pinned to current `main` (verified: all three variants build, +316 bytes
|
||
flash, and it brings the `l r input swap` hardware fix plus the `NavigateToView` the SelfTest
|
||
variant was already written against). **Remaining: the vendoring copy itself**, which lands with the
|
||
PlatformIO cut (plan §5). Delete this entry when it does.
|
||
|
||
### Q5: Legacy feedback modes — delete or keep for A/B? (2026-07-21)
|
||
|
||
`RandomiseOutputs`/`RandomiseMlp`/`Diffuse`/`on_drag` have no product consumer, but `docs/adr/rl-feedback-design.md` explicitly kept Diffuse for A/B comparison. Deleting reverses a recorded decision — operator call (plan §7.1).
|
||
|
||
## Deferred / accepted debt
|
||
|
||
- **EOC effects chain, ShapeSeq sequencer, modular engine (Phase E)** — legacy features consciously out of the v1 rewrite; revisit only if a mode wants them.
|
||
- **Inputs multi-source composition** (2026-06-28, reaffirmed 2026-07-21) — mix-and-match pad+gamepad+MIDI is a recorded, unreversed decision; the UI currently enforces exclusive single-source and the composition machinery sits dormant *by design*. Schedule or keep dormant — but the inputs-spec must stop presenting composition as current behaviour (plan §8).
|
||
- **Schema content is partially placeholder** (2026-07-21) — 20 anonymous "Param NN" slots across paf_synth/channel_strip/xiasri and copy-pasted ML defaults across all 9 modes. Name them during the first curated-preset pass per mode (plan §6.5c), or shrink `output_size` where the engine allows.
|
||
- **Geometric-dislike deliberate divergences** (2026-07-14, one-core P3): (1) the degenerate-branch RNG draws from the controller's deterministic `nisps::Rng`, not libc `rand()` — native==WASM parity holds; (2) upstream's async shuffled two-LR `optimise()` is collapsed into one synchronous `dislike_geometric()` training only the pressed negative's target — behavioural, not bitwise, parity with firmware upstream, by design; (3) `RandomiseMlp` uses `draw_weights(spread)` rather than the old asymmetric ranges. All intentional.
|
||
- **Manifold dock splits `state`/`muted`/`armed`** (2026-06-28) — deliberate divergence from the deployed conflated `frozen`↔`muted` model (dock-spec §3.3). `muted`-downstream and the `soloMode` gradient-mask variants remain UI-only; the C API exposes `set_focus` but no per-mode gradient masking yet. (The audit found `soloMode` behaviourally inert in the controller — plan L20 trims it until `train_masked` exists.)
|
||
|
||
## Recently resolved (delete after a few weeks)
|
||
|
||
- 2026-07-21: Full-repo simplification audit landed (recon + plan + this rewrite). Superseded entries removed: "browser-only engines incomplete" (→ defect 2/plan 5b), "loss curve not plumbed" (→ defect 8), "NISPS_AUDIO_FUNC misshapen" (→ plan Phase 1, S21/L13), stale "VCV not currently maintained" note (vcv/ is active and consumes `nisps/` directly post-P6).
|
||
- 2026-07-18: Browser curve maths unified onto the canonical `nisps/core/math.hpp` catalog at P4; four silently-divergent TS curves re-baselined.
|
||
- 2026-07-14: WASM MLP fixed-architecture defect resolved by P2 (`MLPCore<Storage>`; browser runtime-shaped, firmware zero-heap fixed).
|