From 1f4513802f781b35748e2faa5846e5b365179b0f Mon Sep 17 00:00:00 2001 From: monkey-w1n5t0n Date: Tue, 14 Jul 2026 04:56:18 +0200 Subject: [PATCH] =?UTF-8?q?docs:=20P3=20software=20burned=20down=20?= =?UTF-8?q?=E2=80=94=20ADR=20=C2=A78=20ALIGNMENT=20updates,=20MAP=20feedba?= =?UTF-8?q?ck/replay=20entries?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - ALIGNMENT: retract the 2026-06-18 'geometric push not ported' accepted divergence (it IS ported); record the two deliberate divergences (useRandom via nisps::Rng; synchronous per-press SGD vs upstream's shuffled TrainBatch — behavioural parity with firmware, 1e-5 native↔WASM) - MAP: nisps/ml gains replay/geo_push/warm_start; feedback is storage- policied with Geometric default + Diffuse legacy - AGENT-REFERENCE: Jolt/OU/dislike TS-math limitation closed - plan: P3 software marked landed; chokepoint A pending hardware + bug 10c3e55c (explore wiring linker-GC'd out of PAFSynth ELF, pre-existing) --- ALIGNMENT.md | 5 +++- MAP.md | 2 +- docs/AGENT-REFERENCE.md | 2 +- docs/specs/plans/one-core-engine-refactor.md | 31 ++++++++++++-------- 4 files changed, 25 insertions(+), 15 deletions(-) diff --git a/ALIGNMENT.md b/ALIGNMENT.md index 5699a96..d65d937 100644 --- a/ALIGNMENT.md +++ b/ALIGNMENT.md @@ -70,7 +70,10 @@ The original a-immersive was mobile-first ("designed for touch / foldable phone - **Modular engine (Phase E)** — newer JS-side feature in legacy; out of v1. - **Engine configuration panel** (SPEC-controls Part 8) — backlog. Would let users tune network architecture, loss, optimizer at runtime. Currently compile-time only. - **VCV Rack module** — used to consume `nisps-core/`. Now gone. If revived, it'd consume `nisps/` directly via CMake; not currently maintained. -- **Negative-feedback "Avoid" uses `move_weights`, not the firmware's k-NN geometric centroid push** (2026-06-18) — porting the 3-mode "Down Action" feature (`nisps/ml/feedback.hpp`, from upstream `InterfaceRL` branch `feat/feedback-explore-modes`) deliberately mapped `Avoid` to the existing `MLP::move_weights` Gaussian perturbation rather than re-porting the old geometric-centroid-toward-liked-prototype push (which depended on a firmware-only `ReplayMemory` with reward accumulation/decay). `RandomiseMlp` likewise uses `draw_weights(spread)` rather than the old asymmetric `RandomiseWeightsAndBiasesLin(-0.9,1.1,-0.9,0.3)`. Accepted divergence — **not a bug**. If richer geometric/contrastive avoidance is wanted later, it belongs in a separate replay-memory component, not `FeedbackController`. +- **Geometric-dislike deliberate divergences** (2026-07-14, one-core-engine P3; supersedes and RETRACTS the 2026-06-18 "Avoid = move_weights, geometric push not ported" note — the k-NN centroid push IS now ported, upstream `InterfaceRL` @ `0a541cc`, into `nisps/ml/{replay,geo_push}.hpp` + `feedback.hpp` `AvoidStyle::Geometric` default). Two divergences are by design: + 1. The upstream `useRandom` degenerate branch (disliked action exactly on the centroid) draws from the controller's deterministic `nisps::Rng`, not libc `rand()` — native==WASM parity holds (parity Stage 6); the value is generated, never compared against upstream. + 2. Upstream trains via async `optimise()` with shuffled `TrainBatch` over positive+geometric batches at two LRs; nisps collapses press+optimise into ONE synchronous `dislike_geometric()` that trains only the pressed negative's target (per-sample SGD, no shuffle). Behavioural — not bitwise — parity with firmware upstream, by design; `native == WASM` is pinned at 1e-5 instead. + Also: `RandomiseMlp` still uses `draw_weights(spread)` rather than the old asymmetric `RandomiseWeightsAndBiasesLin(-0.9,1.1,-0.9,0.3)` (unchanged accepted divergence). - **Manifold dock splits `state`/`muted`/`armed` into three fields, diverging from the deployed conflated `frozen`↔`muted`** (2026-06-28) — the deployed a-immersive override system maps the heatmap-popup `frozen` and the group-drawer `muted` onto ONE underlying field. The Manifold per-output model (`manifold/src/dock/output-state.ts`, folded onto `MFParam`) deliberately separates them: `status` carries the off/fixed/live tri-state, `muted` is downstream-silence (still computed + visible), `armed` is solo/focus-training. Cleaner semantics; intentional divergence (dock-spec §3.3, open choice 3). Note `muted`-downstream and the `soloMode` gradient-mask variants (mask-gradients / zero-loss / dont-care) are UI+state only so far — the engine C API exposes `set_focus` but not per-mode gradient masking nor a downstream mute gate yet (TODOs in `ConsoleApp.tsx` / `Drawers.tsx` reference rl-feedback-design §3 and dock-spec §3.3). diff --git a/MAP.md b/MAP.md index 2781521..2e1b94e 100644 --- a/MAP.md +++ b/MAP.md @@ -6,7 +6,7 @@ MEMLNaut-NISPS — Neural Interactive Shaping of Parameter Spaces. One C++20 cod ### `nisps/` — platform-agnostic C++20 library (the only ML/DSP/engine code) - `nisps/core/` — `perf.hpp` (memory section attrs), `types.hpp`, `concepts.hpp` (`MLEngine`, `AudioEngine`, `Mode`), `fixed_buffer.hpp`, `ring_buffer.hpp` (SPSC lock-free, replaces pico/util/queue), `rng.hpp` (xoshiro256+ deterministic), `math.hpp` (fast_sigmoid, `Curve` enum + `apply_curve`). -- `nisps/ml/` — the MLP core, written once against a storage policy (`mlp.hpp` `MLPCore`): `storage.hpp` (`FixedStorage` — template-sized `std::array`, zero heap; `MLP` alias preserves the classic compile-time surface) and `dynamic_storage.hpp` (`DynamicStorage` — runtime dims, single arena alloc at construction; `#error`s on RP2350 builds, sole lint heap-allowlist entry). Fixed↔dynamic bit-parity enforced by `tests/cpp/test_mlp_storage_parity.cpp`. Files: `mlp.hpp`, `activations.hpp`, `loss.hpp` (MSE, no double-scaling), `training.hpp` (SGD + grad clipping), `init.hpp` (spread-aware uniform↔Xavier), `rl.hpp` (`move_weights` with output pin mask + per-layer scaling + weight decay), `jolt.hpp` (`Jolt` — held continuous weight-morph over the flat weight buffer + post-release LR ramp; ported from upstream InterfaceRL), `ou_noise.hpp` (`OUNoise` — Ornstein-Uhlenbeck exploration walk on the output vector; ported from upstream InterfaceRL), `feedback.hpp` (`FeedbackController` — the 3-mode "Down Action" negative-feedback state machine: Avoid / RandomiseOutputs / RandomiseMlp; header-only, zero-heap, own deterministic RNG, exposed via `nisps_ml_feedback_*` C API), `stats.hpp`. Jolt + OU are inert by default and wired into `ModeBase`, so every mode exposes `jolt_press/jolt_release`, `jolt_lr_scale`, and `set_explore_intensity`. +- `nisps/ml/` — the MLP core, written once against a storage policy (`mlp.hpp` `MLPCore`): `storage.hpp` (`FixedStorage` — template-sized `std::array`, zero heap; `MLP` alias preserves the classic compile-time surface) and `dynamic_storage.hpp` (`DynamicStorage` — runtime dims, single arena alloc at construction; `#error`s on RP2350 builds, sole lint heap-allowlist entry). Fixed↔dynamic bit-parity enforced by `tests/cpp/test_mlp_storage_parity.cpp`. Files: `mlp.hpp`, `activations.hpp`, `loss.hpp` (MSE, no double-scaling), `training.hpp` (SGD + grad clipping), `init.hpp` (spread-aware uniform↔Xavier), `rl.hpp` (`move_weights` with output pin mask + per-layer scaling + weight decay), `jolt.hpp` (`Jolt` — held continuous weight-morph over the flat weight buffer + post-release LR ramp; ported from upstream InterfaceRL), `ou_noise.hpp` (`OUNoise` — Ornstein-Uhlenbeck exploration walk on the output vector; ported from upstream InterfaceRL), `feedback.hpp` (`FeedbackControllerCore` — the "Down Action" state machine: Avoid (geometric push-away default / Diffuse legacy) / RandomiseOutputs / RandomiseMlp / ExploreAndPlace; storage-policied like the MLP, own deterministic RNG, exposed via `nisps_ml_feedback_*` C API), `replay.hpp` (`ReplayView` — reward-tagged memory: dedup/deepen, k-NN positive centroid with deterministic tie-break, proportional decay+eviction), `geo_push.hpp` (push-away target computation, upstream InterfaceRL @ 0a541cc), `warm_start.hpp` (overlapping-weights copy for reshape), `stats.hpp`. Jolt + OU are inert by default and wired into `ModeBase`, so every mode exposes `jolt_press/jolt_release`, `jolt_lr_scale`, and `set_explore_intensity`. - `nisps/dsp/` — `biquad.hpp`, `delay.hpp`, `reverb.hpp`, `filter.hpp`, `env.hpp`, `osc.hpp`, `pitch_shift.hpp`, `dc_blocker.hpp`. Lean primitives extracted from maximilian; daisysp PitchShifter replaced with custom granular impl. - `nisps/engines/` — eight audio engines, each satisfying `AudioEngine`: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `breakor.hpp` (sequencer, NoOp audio), `elysiamorf.hpp` (sequencer, NoOp audio), `analysis.hpp` (input-side spectral features). Plus `base.hpp` (`NoOpEngine`, engine_id "thru"). - `nisps/modes/` — platform-agnostic modes binding `{ML config, engine, voice space lambdas, abstract I/O channels}`. Files: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `slp_workshop.hpp` (`SLPWorkshopMode` — the Synth Library Portland workshop build; reuses the MEMLCelium engine + MLP shape, foregrounds the Jolt + OU explore gestures), `breakor.hpp`, `elysiamorf.hpp`, `sound_analysis_midi.hpp`, `external_synth_midi.hpp` (`ExternalSynthMIDIMode` — joystick→MLP→MIDI CC for an external synth; compile-time device from `nisps/midi`; `consteval pick_cc_slots` curates which params fill the NOut slots; NoOpEngine, `kRouteOutputsToEngine=false`). `base.hpp` provides a CRTP scaffold eliminating the duplication that previously plagued firmware modes. `voice_space.hpp` holds engine-side voice space dispatch helpers. `generated/` contains codegen output (do not edit by hand). diff --git a/docs/AGENT-REFERENCE.md b/docs/AGENT-REFERENCE.md index c811574..58b30d0 100644 --- a/docs/AGENT-REFERENCE.md +++ b/docs/AGENT-REFERENCE.md @@ -121,7 +121,7 @@ The browser MLP is runtime-shaped since P2 (`MLPCore`): `nisps_m - Loss history not yet plumbed through C API; only the final loss of a training run reaches TS. - Mic input through the worklet for XIASRI / SoundAnalysisMIDI is not wired in manifold. - C15 has no home on main (see `ALIGNMENT.md` defect #1). -- The browser Jolt/OU controls in manifold reimplement the gesture math in TS (interim, ported from the retired playground) rather than calling the C++ `ml::Jolt`/`ml::OUNoise` through WASM. They drive weights via the existing `nisps_ml_get/set_weights` bindings — the P3 phase of the one-core-engine plan replaces them with `nisps_ml_jolt_press/release` + `nisps_ml_explore_intensity` bindings. +- (P3, 2026-07-14) The browser Jolt/OU gestures and the geometric dislike run the C++ core through WASM: `nisps_ml_jolt_*`, `nisps_ml_explore_*`, `nisps_ml_feedback_dislike_geometric` — no TS gesture math remains. ## URL parameters (manifold) diff --git a/docs/specs/plans/one-core-engine-refactor.md b/docs/specs/plans/one-core-engine-refactor.md index b496f29..48bad35 100644 --- a/docs/specs/plans/one-core-engine-refactor.md +++ b/docs/specs/plans/one-core-engine-refactor.md @@ -129,19 +129,26 @@ Each phase ends green on its test gate and is independently landable. File phase ±1%); manifold 9 unit + 25 e2e. Hardware audio-callback timing check deferred to the chokepoint-B hardware session (operator, physical MEMLNaut). -### P3 — Exploration + feedback fully in core (≈3–4 days) +### P3 — Exploration + feedback fully in core (≈3–4 days) — ✅ software landed 2026-07-14; ⏳ chokepoint A hardware spot-check pending (operator, physical MEMLNaut) -- **Geometric dislike:** port firmware k-NN centroid push (upstream `InterfaceRL.cpp:602-738`) into - `nisps/ml/geo_push.hpp` + `replay.hpp` + `mlp.train_targets`, per `docs/adr/rl-feedback-design.md` §4. - Expose via `nisps_ml_feedback_dislike_geometric`. Delete the TS approximation + both `C++ GAP` blocks in - `manifold/src/feedback/controller.ts`. -- **Jolt / OU:** expose `nisps/ml/jolt.hpp` + `ou_noise.hpp` through bindings - (`nisps_ml_jolt_press/release`, `nisps_ml_explore_intensity`); manifold's P1 UI shells switch to them. - The playground TS ports die with P1. -- **RNG:** feedback/exploration randomness in the browser comes from the core's seeded `Rng` streams - (already per-instance, ctor-seeded). Delete `manifold/src/feedback/rng.ts`. -- **Gate:** new cross-platform parity test — scripted feedback session (seeded) produces identical weight - trajectories native↔WASM; firmware spot-check on hardware (chokepoint A) for geometric dislike feel. +- ✅ **Geometric dislike:** firmware k-NN centroid push (upstream `InterfaceRL` @ `0a541cc`, verbatim + constants) ported into `nisps/ml/geo_push.hpp` + `replay.hpp` + `mlp.train_targets`; the press+optimise + halves collapsed into one synchronous `dislike_geometric()` per the ADR. `AvoidStyle::Geometric` is the + Avoid default; `Diffuse` (legacy move_weights) kept for A/B. Exposed via + `nisps_ml_feedback_dislike_geometric` (+ store_positive/counts/set_avoid_style). TS approximation + + both `C++ GAP` blocks deleted; two call-site bugs fixed en route (raw vs HEARD post-pipeline vector — + raw gives a zero-derivative no-op). Cold-start prompt per ADR §7. +- ✅ **Jolt / OU:** exposed through bindings (`nisps_ml_jolt_press/step/release/lr_scale/...`, + `nisps_ml_explore_intensity/apply`; browser `OUNoise<4096>` over-provisioned); manifold's P1 UI shells + switched at the marked swap point; interim TS ports (`jolt.ts`, `ou-explore.ts`) deleted. OU applies + through the spine's output morph hook. +- ✅ **RNG:** `manifold/src/feedback/rng.ts` deleted; all feedback/exploration randomness is the core's + seeded per-instance `Rng` streams. (Debug-only fixed seed/dt under `?debug=1` for deterministic e2e.) +- **Gate:** ✅ parity Stage 6 (v4) — scripted seeded feedback session (2 likes → centroid → 2 dislikes) + produces identical weight trajectories native↔WASM (961 floats, 2.4e-7); ctest geo suite; manifold 9 + unit + 27 e2e. ⏳ Hardware spot-check (chokepoint A) BLOCKED on the physical MEMLNaut — and on ergo bug + `10c3e55c`: the explore/place wiring is linker-GC'd out of the PAFSynth ELF (pre-existing, discovered + during P3; the gestures are dead code on that build until fixed). ### P4 — Input/output pipelines + curves into core (≈3 days)