docs: P3 software burned down — ADR §8 ALIGNMENT updates, MAP feedback/replay entries
- ALIGNMENT: retract the 2026-06-18 'geometric push not ported' accepted divergence (it IS ported); record the two deliberate divergences (useRandom via nisps::Rng; synchronous per-press SGD vs upstream's shuffled TrainBatch — behavioural parity with firmware, 1e-5 native↔WASM) - MAP: nisps/ml gains replay/geo_push/warm_start; feedback is storage- policied with Geometric default + Diffuse legacy - AGENT-REFERENCE: Jolt/OU/dislike TS-math limitation closed - plan: P3 software marked landed; chokepoint A pending hardware + bug 10c3e55c (explore wiring linker-GC'd out of PAFSynth ELF, pre-existing)
This commit is contained in:
parent
a36b58dce8
commit
1f4513802f
4 changed files with 25 additions and 15 deletions
|
|
@ -70,7 +70,10 @@ The original a-immersive was mobile-first ("designed for touch / foldable phone
|
||||||
- **Modular engine (Phase E)** — newer JS-side feature in legacy; out of v1.
|
- **Modular engine (Phase E)** — newer JS-side feature in legacy; out of v1.
|
||||||
- **Engine configuration panel** (SPEC-controls Part 8) — backlog. Would let users tune network architecture, loss, optimizer at runtime. Currently compile-time only.
|
- **Engine configuration panel** (SPEC-controls Part 8) — backlog. Would let users tune network architecture, loss, optimizer at runtime. Currently compile-time only.
|
||||||
- **VCV Rack module** — used to consume `nisps-core/`. Now gone. If revived, it'd consume `nisps/` directly via CMake; not currently maintained.
|
- **VCV Rack module** — used to consume `nisps-core/`. Now gone. If revived, it'd consume `nisps/` directly via CMake; not currently maintained.
|
||||||
- **Negative-feedback "Avoid" uses `move_weights`, not the firmware's k-NN geometric centroid push** (2026-06-18) — porting the 3-mode "Down Action" feature (`nisps/ml/feedback.hpp`, from upstream `InterfaceRL` branch `feat/feedback-explore-modes`) deliberately mapped `Avoid` to the existing `MLP::move_weights` Gaussian perturbation rather than re-porting the old geometric-centroid-toward-liked-prototype push (which depended on a firmware-only `ReplayMemory` with reward accumulation/decay). `RandomiseMlp` likewise uses `draw_weights(spread)` rather than the old asymmetric `RandomiseWeightsAndBiasesLin(-0.9,1.1,-0.9,0.3)`. Accepted divergence — **not a bug**. If richer geometric/contrastive avoidance is wanted later, it belongs in a separate replay-memory component, not `FeedbackController`.
|
- **Geometric-dislike deliberate divergences** (2026-07-14, one-core-engine P3; supersedes and RETRACTS the 2026-06-18 "Avoid = move_weights, geometric push not ported" note — the k-NN centroid push IS now ported, upstream `InterfaceRL` @ `0a541cc`, into `nisps/ml/{replay,geo_push}.hpp` + `feedback.hpp` `AvoidStyle::Geometric` default). Two divergences are by design:
|
||||||
|
1. The upstream `useRandom` degenerate branch (disliked action exactly on the centroid) draws from the controller's deterministic `nisps::Rng`, not libc `rand()` — native==WASM parity holds (parity Stage 6); the value is generated, never compared against upstream.
|
||||||
|
2. Upstream trains via async `optimise()` with shuffled `TrainBatch` over positive+geometric batches at two LRs; nisps collapses press+optimise into ONE synchronous `dislike_geometric()` that trains only the pressed negative's target (per-sample SGD, no shuffle). Behavioural — not bitwise — parity with firmware upstream, by design; `native == WASM` is pinned at 1e-5 instead.
|
||||||
|
Also: `RandomiseMlp` still uses `draw_weights(spread)` rather than the old asymmetric `RandomiseWeightsAndBiasesLin(-0.9,1.1,-0.9,0.3)` (unchanged accepted divergence).
|
||||||
|
|
||||||
- **Manifold dock splits `state`/`muted`/`armed` into three fields, diverging from the deployed conflated `frozen`↔`muted`** (2026-06-28) — the deployed a-immersive override system maps the heatmap-popup `frozen` and the group-drawer `muted` onto ONE underlying field. The Manifold per-output model (`manifold/src/dock/output-state.ts`, folded onto `MFParam`) deliberately separates them: `status` carries the off/fixed/live tri-state, `muted` is downstream-silence (still computed + visible), `armed` is solo/focus-training. Cleaner semantics; intentional divergence (dock-spec §3.3, open choice 3). Note `muted`-downstream and the `soloMode` gradient-mask variants (mask-gradients / zero-loss / dont-care) are UI+state only so far — the engine C API exposes `set_focus` but not per-mode gradient masking nor a downstream mute gate yet (TODOs in `ConsoleApp.tsx` / `Drawers.tsx` reference rl-feedback-design §3 and dock-spec §3.3).
|
- **Manifold dock splits `state`/`muted`/`armed` into three fields, diverging from the deployed conflated `frozen`↔`muted`** (2026-06-28) — the deployed a-immersive override system maps the heatmap-popup `frozen` and the group-drawer `muted` onto ONE underlying field. The Manifold per-output model (`manifold/src/dock/output-state.ts`, folded onto `MFParam`) deliberately separates them: `status` carries the off/fixed/live tri-state, `muted` is downstream-silence (still computed + visible), `armed` is solo/focus-training. Cleaner semantics; intentional divergence (dock-spec §3.3, open choice 3). Note `muted`-downstream and the `soloMode` gradient-mask variants (mask-gradients / zero-loss / dont-care) are UI+state only so far — the engine C API exposes `set_focus` but not per-mode gradient masking nor a downstream mute gate yet (TODOs in `ConsoleApp.tsx` / `Drawers.tsx` reference rl-feedback-design §3 and dock-spec §3.3).
|
||||||
|
|
||||||
|
|
|
||||||
2
MAP.md
2
MAP.md
|
|
@ -6,7 +6,7 @@ MEMLNaut-NISPS — Neural Interactive Shaping of Parameter Spaces. One C++20 cod
|
||||||
|
|
||||||
### `nisps/` — platform-agnostic C++20 library (the only ML/DSP/engine code)
|
### `nisps/` — platform-agnostic C++20 library (the only ML/DSP/engine code)
|
||||||
- `nisps/core/` — `perf.hpp` (memory section attrs), `types.hpp`, `concepts.hpp` (`MLEngine`, `AudioEngine`, `Mode`), `fixed_buffer.hpp`, `ring_buffer.hpp` (SPSC lock-free, replaces pico/util/queue), `rng.hpp` (xoshiro256+ deterministic), `math.hpp` (fast_sigmoid, `Curve` enum + `apply_curve`).
|
- `nisps/core/` — `perf.hpp` (memory section attrs), `types.hpp`, `concepts.hpp` (`MLEngine`, `AudioEngine`, `Mode`), `fixed_buffer.hpp`, `ring_buffer.hpp` (SPSC lock-free, replaces pico/util/queue), `rng.hpp` (xoshiro256+ deterministic), `math.hpp` (fast_sigmoid, `Curve` enum + `apply_curve`).
|
||||||
- `nisps/ml/` — the MLP core, written once against a storage policy (`mlp.hpp` `MLPCore<Storage>`): `storage.hpp` (`FixedStorage` — template-sized `std::array`, zero heap; `MLP<NIn,NH1,NH2,NH3,NOut>` alias preserves the classic compile-time surface) and `dynamic_storage.hpp` (`DynamicStorage` — runtime dims, single arena alloc at construction; `#error`s on RP2350 builds, sole lint heap-allowlist entry). Fixed↔dynamic bit-parity enforced by `tests/cpp/test_mlp_storage_parity.cpp`. Files: `mlp.hpp`, `activations.hpp`, `loss.hpp` (MSE, no double-scaling), `training.hpp` (SGD + grad clipping), `init.hpp` (spread-aware uniform↔Xavier), `rl.hpp` (`move_weights` with output pin mask + per-layer scaling + weight decay), `jolt.hpp` (`Jolt` — held continuous weight-morph over the flat weight buffer + post-release LR ramp; ported from upstream InterfaceRL), `ou_noise.hpp` (`OUNoise<N>` — Ornstein-Uhlenbeck exploration walk on the output vector; ported from upstream InterfaceRL), `feedback.hpp` (`FeedbackController<MLP_T>` — the 3-mode "Down Action" negative-feedback state machine: Avoid / RandomiseOutputs / RandomiseMlp; header-only, zero-heap, own deterministic RNG, exposed via `nisps_ml_feedback_*` C API), `stats.hpp`. Jolt + OU are inert by default and wired into `ModeBase`, so every mode exposes `jolt_press/jolt_release`, `jolt_lr_scale`, and `set_explore_intensity`.
|
- `nisps/ml/` — the MLP core, written once against a storage policy (`mlp.hpp` `MLPCore<Storage>`): `storage.hpp` (`FixedStorage` — template-sized `std::array`, zero heap; `MLP<NIn,NH1,NH2,NH3,NOut>` alias preserves the classic compile-time surface) and `dynamic_storage.hpp` (`DynamicStorage` — runtime dims, single arena alloc at construction; `#error`s on RP2350 builds, sole lint heap-allowlist entry). Fixed↔dynamic bit-parity enforced by `tests/cpp/test_mlp_storage_parity.cpp`. Files: `mlp.hpp`, `activations.hpp`, `loss.hpp` (MSE, no double-scaling), `training.hpp` (SGD + grad clipping), `init.hpp` (spread-aware uniform↔Xavier), `rl.hpp` (`move_weights` with output pin mask + per-layer scaling + weight decay), `jolt.hpp` (`Jolt` — held continuous weight-morph over the flat weight buffer + post-release LR ramp; ported from upstream InterfaceRL), `ou_noise.hpp` (`OUNoise<N>` — Ornstein-Uhlenbeck exploration walk on the output vector; ported from upstream InterfaceRL), `feedback.hpp` (`FeedbackControllerCore<FbStorage>` — the "Down Action" state machine: Avoid (geometric push-away default / Diffuse legacy) / RandomiseOutputs / RandomiseMlp / ExploreAndPlace; storage-policied like the MLP, own deterministic RNG, exposed via `nisps_ml_feedback_*` C API), `replay.hpp` (`ReplayView` — reward-tagged memory: dedup/deepen, k-NN positive centroid with deterministic tie-break, proportional decay+eviction), `geo_push.hpp` (push-away target computation, upstream InterfaceRL @ 0a541cc), `warm_start.hpp` (overlapping-weights copy for reshape), `stats.hpp`. Jolt + OU are inert by default and wired into `ModeBase`, so every mode exposes `jolt_press/jolt_release`, `jolt_lr_scale`, and `set_explore_intensity`.
|
||||||
- `nisps/dsp/` — `biquad.hpp`, `delay.hpp`, `reverb.hpp`, `filter.hpp`, `env.hpp`, `osc.hpp`, `pitch_shift.hpp`, `dc_blocker.hpp`. Lean primitives extracted from maximilian; daisysp PitchShifter replaced with custom granular impl.
|
- `nisps/dsp/` — `biquad.hpp`, `delay.hpp`, `reverb.hpp`, `filter.hpp`, `env.hpp`, `osc.hpp`, `pitch_shift.hpp`, `dc_blocker.hpp`. Lean primitives extracted from maximilian; daisysp PitchShifter replaced with custom granular impl.
|
||||||
- `nisps/engines/` — eight audio engines, each satisfying `AudioEngine`: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `breakor.hpp` (sequencer, NoOp audio), `elysiamorf.hpp` (sequencer, NoOp audio), `analysis.hpp` (input-side spectral features). Plus `base.hpp` (`NoOpEngine`, engine_id "thru").
|
- `nisps/engines/` — eight audio engines, each satisfying `AudioEngine`: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `breakor.hpp` (sequencer, NoOp audio), `elysiamorf.hpp` (sequencer, NoOp audio), `analysis.hpp` (input-side spectral features). Plus `base.hpp` (`NoOpEngine`, engine_id "thru").
|
||||||
- `nisps/modes/` — platform-agnostic modes binding `{ML config, engine, voice space lambdas, abstract I/O channels}`. Files: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `slp_workshop.hpp` (`SLPWorkshopMode` — the Synth Library Portland workshop build; reuses the MEMLCelium engine + MLP shape, foregrounds the Jolt + OU explore gestures), `breakor.hpp`, `elysiamorf.hpp`, `sound_analysis_midi.hpp`, `external_synth_midi.hpp` (`ExternalSynthMIDIMode<const MidiDevice&, NOut>` — joystick→MLP→MIDI CC for an external synth; compile-time device from `nisps/midi`; `consteval pick_cc_slots` curates which params fill the NOut slots; NoOpEngine, `kRouteOutputsToEngine=false`). `base.hpp` provides a CRTP scaffold eliminating the duplication that previously plagued firmware modes. `voice_space.hpp` holds engine-side voice space dispatch helpers. `generated/` contains codegen output (do not edit by hand).
|
- `nisps/modes/` — platform-agnostic modes binding `{ML config, engine, voice space lambdas, abstract I/O channels}`. Files: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `slp_workshop.hpp` (`SLPWorkshopMode` — the Synth Library Portland workshop build; reuses the MEMLCelium engine + MLP shape, foregrounds the Jolt + OU explore gestures), `breakor.hpp`, `elysiamorf.hpp`, `sound_analysis_midi.hpp`, `external_synth_midi.hpp` (`ExternalSynthMIDIMode<const MidiDevice&, NOut>` — joystick→MLP→MIDI CC for an external synth; compile-time device from `nisps/midi`; `consteval pick_cc_slots` curates which params fill the NOut slots; NoOpEngine, `kRouteOutputsToEngine=false`). `base.hpp` provides a CRTP scaffold eliminating the duplication that previously plagued firmware modes. `voice_space.hpp` holds engine-side voice space dispatch helpers. `generated/` contains codegen output (do not edit by hand).
|
||||||
|
|
|
||||||
|
|
@ -121,7 +121,7 @@ The browser MLP is runtime-shaped since P2 (`MLPCore<DynamicStorage>`): `nisps_m
|
||||||
- Loss history not yet plumbed through C API; only the final loss of a training run reaches TS.
|
- Loss history not yet plumbed through C API; only the final loss of a training run reaches TS.
|
||||||
- Mic input through the worklet for XIASRI / SoundAnalysisMIDI is not wired in manifold.
|
- Mic input through the worklet for XIASRI / SoundAnalysisMIDI is not wired in manifold.
|
||||||
- C15 has no home on main (see `ALIGNMENT.md` defect #1).
|
- C15 has no home on main (see `ALIGNMENT.md` defect #1).
|
||||||
- The browser Jolt/OU controls in manifold reimplement the gesture math in TS (interim, ported from the retired playground) rather than calling the C++ `ml::Jolt`/`ml::OUNoise` through WASM. They drive weights via the existing `nisps_ml_get/set_weights` bindings — the P3 phase of the one-core-engine plan replaces them with `nisps_ml_jolt_press/release` + `nisps_ml_explore_intensity` bindings.
|
- (P3, 2026-07-14) The browser Jolt/OU gestures and the geometric dislike run the C++ core through WASM: `nisps_ml_jolt_*`, `nisps_ml_explore_*`, `nisps_ml_feedback_dislike_geometric` — no TS gesture math remains.
|
||||||
|
|
||||||
## URL parameters (manifold)
|
## URL parameters (manifold)
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -129,19 +129,26 @@ Each phase ends green on its test gate and is independently landable. File phase
|
||||||
±1%); manifold 9 unit + 25 e2e. Hardware audio-callback timing check deferred to the chokepoint-B
|
±1%); manifold 9 unit + 25 e2e. Hardware audio-callback timing check deferred to the chokepoint-B
|
||||||
hardware session (operator, physical MEMLNaut).
|
hardware session (operator, physical MEMLNaut).
|
||||||
|
|
||||||
### P3 — Exploration + feedback fully in core (≈3–4 days)
|
### P3 — Exploration + feedback fully in core (≈3–4 days) — ✅ software landed 2026-07-14; ⏳ chokepoint A hardware spot-check pending (operator, physical MEMLNaut)
|
||||||
|
|
||||||
- **Geometric dislike:** port firmware k-NN centroid push (upstream `InterfaceRL.cpp:602-738`) into
|
- ✅ **Geometric dislike:** firmware k-NN centroid push (upstream `InterfaceRL` @ `0a541cc`, verbatim
|
||||||
`nisps/ml/geo_push.hpp` + `replay.hpp` + `mlp.train_targets`, per `docs/adr/rl-feedback-design.md` §4.
|
constants) ported into `nisps/ml/geo_push.hpp` + `replay.hpp` + `mlp.train_targets`; the press+optimise
|
||||||
Expose via `nisps_ml_feedback_dislike_geometric`. Delete the TS approximation + both `C++ GAP` blocks in
|
halves collapsed into one synchronous `dislike_geometric()` per the ADR. `AvoidStyle::Geometric` is the
|
||||||
`manifold/src/feedback/controller.ts`.
|
Avoid default; `Diffuse` (legacy move_weights) kept for A/B. Exposed via
|
||||||
- **Jolt / OU:** expose `nisps/ml/jolt.hpp` + `ou_noise.hpp` through bindings
|
`nisps_ml_feedback_dislike_geometric` (+ store_positive/counts/set_avoid_style). TS approximation +
|
||||||
(`nisps_ml_jolt_press/release`, `nisps_ml_explore_intensity`); manifold's P1 UI shells switch to them.
|
both `C++ GAP` blocks deleted; two call-site bugs fixed en route (raw vs HEARD post-pipeline vector —
|
||||||
The playground TS ports die with P1.
|
raw gives a zero-derivative no-op). Cold-start prompt per ADR §7.
|
||||||
- **RNG:** feedback/exploration randomness in the browser comes from the core's seeded `Rng` streams
|
- ✅ **Jolt / OU:** exposed through bindings (`nisps_ml_jolt_press/step/release/lr_scale/...`,
|
||||||
(already per-instance, ctor-seeded). Delete `manifold/src/feedback/rng.ts`.
|
`nisps_ml_explore_intensity/apply`; browser `OUNoise<4096>` over-provisioned); manifold's P1 UI shells
|
||||||
- **Gate:** new cross-platform parity test — scripted feedback session (seeded) produces identical weight
|
switched at the marked swap point; interim TS ports (`jolt.ts`, `ou-explore.ts`) deleted. OU applies
|
||||||
trajectories native↔WASM; firmware spot-check on hardware (chokepoint A) for geometric dislike feel.
|
through the spine's output morph hook.
|
||||||
|
- ✅ **RNG:** `manifold/src/feedback/rng.ts` deleted; all feedback/exploration randomness is the core's
|
||||||
|
seeded per-instance `Rng` streams. (Debug-only fixed seed/dt under `?debug=1` for deterministic e2e.)
|
||||||
|
- **Gate:** ✅ parity Stage 6 (v4) — scripted seeded feedback session (2 likes → centroid → 2 dislikes)
|
||||||
|
produces identical weight trajectories native↔WASM (961 floats, 2.4e-7); ctest geo suite; manifold 9
|
||||||
|
unit + 27 e2e. ⏳ Hardware spot-check (chokepoint A) BLOCKED on the physical MEMLNaut — and on ergo bug
|
||||||
|
`10c3e55c`: the explore/place wiring is linker-GC'd out of the PAFSynth ELF (pre-existing, discovered
|
||||||
|
during P3; the gestures are dead code on that build until fixed).
|
||||||
|
|
||||||
### P4 — Input/output pipelines + curves into core (≈3 days)
|
### P4 — Input/output pipelines + curves into core (≈3 days)
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue