diff --git a/ALIGNMENT.md b/ALIGNMENT.md index ce15fc0..840abbb 100644 --- a/ALIGNMENT.md +++ b/ALIGNMENT.md @@ -63,30 +63,6 @@ command surface to report through. **Rough cost.** Host half is done. On-device: ~a day, and it wants defect 3's serial protocol to have somewhere to send the number. -### 6. SGD-vs-RMSProp is not a research axis — it silently invalidated every ported hyperparameter (2026-04-29; **re-ranked 2026-07-25**) - -**What.** `training.hpp` ships SGD only. Upstream `memlp` (pinned `ea777502` by -`upstream/main`) applies gradients with **RMSProp everywhere** — `Layer.h:239`, the -`m_sq_grad_avg` running squared-gradient average at `Layer.h:601`, `StaticMLP.h:268` -"Mini-batch RMSProp training". This was filed as an optimiser-choice research question -and ranked last. That was wrong, and the 2026-07-25 benchmark work shows why. - -**Why it blocks the mission.** Every learning rate ported from upstream landed in a -different optimiser than the one it was tuned for. RMSProp normalises each step by the -running gradient magnitude, so `lr=1e-3` there is a normalised step; under SGD it is -literally `1e-3 x raw gradient`. The two numbers are unrelated. Concretely: -`feedback.hpp:789` carries `geo_lr_ = 0.001f // upstream InterfaceRL.hpp:312` — an -RMSProp LR pasted into a single SGD step. `tests/cpp/ml_bench.cpp` D1 measures the -result: the geometric dislike aims at a target 0.5 output-units away and moves the -mapping by **5.1e-5**, linearly, so ~10,000 presses would be needed for one press's -intended effect. Meanwhile likes train at `lr 1.0 x 1000 iterations`, making a like -~2e6x stronger than a dislike and heaving the whole mapping on every press -(`ml_bench` U1/U4: `lurch_max` ~1.1 against a [0,1] output range). - -**Rough cost.** A day for the port plus batch-convergence tests — but it must come -BEFORE any retuning of `geo_lr`, `kGeometricPushScale` or the neg-LR base, or those -constants get tuned twice. - ### 6b. The geometric dislike was ported from a superseded upstream design (2026-07-25) **What.** `geo_push.hpp`/`replay.hpp` cite `memllib @ 0a541cc`. `upstream/main` now pins @@ -100,11 +76,34 @@ as a **batch over ALL of them every tick** rather than one item one step; and a ported. On the shared constants (dedup radius 0.05, `kCentroidK` 4) we match. **Why it blocks the mission.** We are carrying a design upstream diagnosed and fixed, -and the fix is documented in their source comments. Compounded with defect 6 the ported -dislike is ~9.4x weaker on constants alone before the optimiser mismatch. +and the fix is documented in their source comments. On the constants alone the ported +dislike is ~9.4x weaker than upstream's. (The optimiser half of this — an RMSProp LR +pasted into an SGD step — is fixed as of 2026-07-25; see "Recently resolved". A single +dislike now moves the mapping 1.6e-2 instead of 5.3e-5, but that is still ~14x weaker +than the legacy Diffuse design measures in one press, `ml_bench` A4.) -**Rough cost.** Small once defect 6 lands — mostly deleting the taper and re-basing -three constants, then re-running `scripts/bench-ml.sh` D1/A4/A7 to confirm. +**Rough cost.** Small — mostly deleting the taper and re-basing three constants, then +re-running `scripts/bench-ml.sh` D1/A4/A7 to confirm. + +### 6d. One like still heaves the whole mapping (2026-07-25) + +**What.** A thumbs-up trains at `lr 1.0 x 1000 iterations` on every gesture. `ml_bench` +U1/U4 measure **lurch** — how far the mapping the musician is playing moves per single +gesture, averaged over the field: `lurch_max` 1.08 against a [0,1] output range, i.e. +one thumbs-up can move the mapping somewhere in the space by more than the entire output +range. Retention (how much of the previous teaching survives) is 0.38; at `iters=1` it +is 0.80. + +**Why it blocks the mission.** This is the other half of the feedback asymmetry, and +RMSProp did NOT fix it — normalising the step size does not change the dose. Upstream +keeps both directions on small repeated steps; we take one enormous positive step and +one small negative one, so teaching feels like a lurch and correcting feels like +nothing. The two numbers now differ by ~70x rather than ~2e6x, which is progress and +still not a design. + +**Rough cost.** Cheap to change, expensive to choose: the tuning space is now measurable +(`ml_bench` U4 sweeps dose; U1 sweeps upstream's soft-target alpha, where alpha=1 is +NISPS today). It wants a matched-N head-to-head, not a guess. ### 6c. `InterfaceRL` — the reference implementation — is not in the tree (2026-07-25) @@ -138,6 +137,19 @@ Legacy a-immersive was mobile-first; Manifold is desktop-first. Defer until user ## Recently resolved (delete after a few weeks) +- 2026-07-25: **The optimiser mismatch (defect 6) is fixed.** `nisps/ml/training.hpp` was + SGD-only while upstream `memlp` (`ea777502`) applies **RMSProp everywhere**, so every + learning rate we ported landed in an optimiser that reads it differently — an RMSProp + `lr` is a normalised step, an SGD `lr` multiplies the raw gradient. `rmsprop_step()` + now ports `Layer.h:239 ApplyAccumulatedGradients` exactly (decay 0.9, eps 1e-6, + sq-avg clamp 1e6, one-sided adjusted-LR clamp 1.0), with the per-weight running + squared-gradient average living in the storage policies so the zero-heap contract + holds. Measured on `ml_bench` D1: one geometric dislike moves the mapping **1.6e-2, + up from 5.3e-5**, and repeated presses now converge on the intended 0.5 push (0.12 at + 10 presses, 0.56 at 100) instead of creeping linearly. Golden vector stages 2 and 3 + were re-captured; stages 0 and 1 are pre-training and did not move. What this does NOT + fix: the dose asymmetry, now tracked as defect 6d. + - 2026-07-21: **Q4 (who owns memllib) closed.** Vendored at `firmware/MEMLNaut-NISPS/lib/memllib/` from upstream `e291192`; the submodule and the fork are both gone. **Q5 (legacy feedback modes) closed** — operator kept all four (`RandomiseOutputs`/`RandomiseMlp`/`Diffuse`/`on_drag`) as diff --git a/MAP.md b/MAP.md index 0a031f6..7337f51 100644 --- a/MAP.md +++ b/MAP.md @@ -6,7 +6,7 @@ MEMLNaut-NISPS — Neural Interactive Shaping of Parameter Spaces. One C++20 cod ### `nisps/` — platform-agnostic C++20 library (the only ML/DSP/engine code) - `nisps/core/` — `perf.hpp` (hot-path/inlining attrs), `types.hpp`, `concepts.hpp` (`MLEngine`, `AudioEngine`, `Mode`), `ring_buffer.hpp` (SPSC lock-free cross-core channel, replaces pico/util/queue), `event_queue.hpp` (single-threaded in-engine event FIFO — deliberately NOT RingBuffer, which is an atomics-based cross-thread channel), `rng.hpp` (xoshiro256+ deterministic), `math.hpp` (fast_sigmoid, `Curve` enum + `apply_curve`). -- `nisps/ml/` — the MLP core, written once against a storage policy (`mlp.hpp` `MLPCore`): `storage.hpp` (`FixedStorage` — template-sized `std::array`, zero heap; `MLP` alias preserves the classic compile-time surface) and `dynamic_storage.hpp` (`DynamicStorage` — runtime dims, single arena alloc at construction; `#error`s on RP2350 builds, sole lint heap-allowlist entry). Fixed↔dynamic bit-parity enforced by `tests/cpp/test_mlp_storage_parity.cpp`. Files: `mlp.hpp`, `activations.hpp`, `loss.hpp` (MSE, no double-scaling), `training.hpp` (SGD + grad clipping), `init.hpp` (spread-aware uniform↔Xavier), `rl.hpp` (`move_weights` with output pin mask + per-layer scaling + weight decay), `jolt.hpp` (`Jolt` — held continuous weight-morph over the flat weight buffer + post-release LR ramp; ported from upstream InterfaceRL), `ou_noise.hpp` (`OUNoise` — Ornstein-Uhlenbeck exploration walk on the output vector; ported from upstream InterfaceRL), `feedback.hpp` (`FeedbackControllerCore` — the "Down Action" state machine: Avoid (geometric push-away default / Diffuse legacy) / RandomiseOutputs / RandomiseMlp / ExploreAndPlace; storage-policied like the MLP, own deterministic RNG, exposed via `nisps_ml_feedback_*` C API), `replay.hpp` (`ReplayView` — reward-tagged memory: dedup/deepen, k-NN positive centroid with deterministic tie-break, proportional decay+eviction), `geo_push.hpp` (push-away target computation, upstream InterfaceRL @ 0a541cc), `warm_start.hpp` (overlapping-weights copy for reshape), `stats.hpp`. `generated/ml_defaults.hpp` is codegen output (do not edit): `nisps::ml::generated::kMlTrainDefaults`, the ONE learning-rate / max-iterations / min-error default shared by firmware, WASM and VCV (source `schemas/ml_defaults.json`); `MLPCore::set_train_config()` and `nisps_ml_set_train_config()` override it at runtime. It lives under `ml/` rather than `modes/generated/` because `nisps/ml` sits below `nisps/modes` — mlp.hpp must not include upward. Jolt + OU are inert by default and wired into `ModeBase`, so every mode exposes `jolt_press/jolt_release`, `jolt_lr_scale`, and `set_explore_intensity`. +- `nisps/ml/` — the MLP core, written once against a storage policy (`mlp.hpp` `MLPCore`): `storage.hpp` (`FixedStorage` — template-sized `std::array`, zero heap; `MLP` alias preserves the classic compile-time surface) and `dynamic_storage.hpp` (`DynamicStorage` — runtime dims, single arena alloc at construction; `#error`s on RP2350 builds, sole lint heap-allowlist entry). Fixed↔dynamic bit-parity enforced by `tests/cpp/test_mlp_storage_parity.cpp`. Files: `mlp.hpp`, `activations.hpp`, `loss.hpp` (MSE, no double-scaling), `training.hpp` (**RMSProp** + grad clipping — `rmsprop_step()` is a line-for-line port of upstream memlp `Layer.h:239 ApplyAccumulatedGradients` @ `ea777502`; the per-weight running squared-gradient average is optimiser state held in the storage policies, NOT part of `weight_count()`/`get_weights()`, and `MLPCore::reset_optimizer_state()` clears it. `draw_weights()` deliberately does not, matching upstream `DrawWeights`), `init.hpp` (spread-aware uniform↔Xavier), `rl.hpp` (`move_weights` with output pin mask + per-layer scaling + weight decay), `jolt.hpp` (`Jolt` — held continuous weight-morph over the flat weight buffer + post-release LR ramp; ported from upstream InterfaceRL), `ou_noise.hpp` (`OUNoise` — Ornstein-Uhlenbeck exploration walk on the output vector; ported from upstream InterfaceRL), `feedback.hpp` (`FeedbackControllerCore` — the "Down Action" state machine: Avoid (geometric push-away default / Diffuse legacy) / RandomiseOutputs / RandomiseMlp / ExploreAndPlace; storage-policied like the MLP, own deterministic RNG, exposed via `nisps_ml_feedback_*` C API), `replay.hpp` (`ReplayView` — reward-tagged memory: dedup/deepen, k-NN positive centroid with deterministic tie-break, proportional decay+eviction), `geo_push.hpp` (push-away target computation, upstream InterfaceRL @ 0a541cc), `warm_start.hpp` (overlapping-weights copy for reshape), `stats.hpp`. `generated/ml_defaults.hpp` is codegen output (do not edit): `nisps::ml::generated::kMlTrainDefaults`, the ONE learning-rate / max-iterations / min-error default shared by firmware, WASM and VCV (source `schemas/ml_defaults.json`); `MLPCore::set_train_config()` and `nisps_ml_set_train_config()` override it at runtime. It lives under `ml/` rather than `modes/generated/` because `nisps/ml` sits below `nisps/modes` — mlp.hpp must not include upward. Jolt + OU are inert by default and wired into `ModeBase`, so every mode exposes `jolt_press/jolt_release`, `jolt_lr_scale`, and `set_explore_intensity`. - `nisps/pipeline/` — the control-rate input/output processing chains (P4): `input_chain.hpp` (`InputChain` — invert→deadzone→circular clamp→momentum-modulated zoom→centred power→EMA→momentum; caller-supplied dt, internal clock, fixed velocity ring, serialisable state) and `output_chain.hpp` (`OutputChain` — curve→EMA→slew→freeze(+mask), capacity-templated). Behaviour contract = the retired manifold TS pipelines, pinned by `manifold/tests/fixtures/` and parity stage 7. - `nisps/dsp/` — `biquad.hpp`, `delay.hpp`, `reverb.hpp`, `filter.hpp`, `env.hpp`, `osc.hpp`, `pitch_shift.hpp`, `dc_blocker.hpp`, plus the sequencer primitives shared by the sequencer engines: `ratio_seq.hpp` and `seq_clock.hpp` (bar phasor + MIDI clock + bpm). Lean primitives extracted from maximilian; daisysp PitchShifter replaced with custom granular impl. - `nisps/engines/` — eight audio engines, each satisfying `AudioEngine`: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `breakor.hpp` (sequencer, NoOp audio), `elysiamorf.hpp` (sequencer, NoOp audio), `analysis.hpp` (input-side spectral features). Plus `base.hpp` (`NoOpEngine`, engine_id "thru"). @@ -126,14 +126,14 @@ includes; no `nisps-core`. - Per-component tests: `test_dsp_*.cpp`, `test_engine_*.cpp`, `test_mlp_*.cpp`, `test_mode_*.cpp`, `test_ring_buffer.cpp`, `test_rng.cpp`, `test_math.cpp`. Helpers in `test_helpers.hpp`. - Verification: `ml_golden_vectors.cpp`, `engine_impulse.cpp` (+ `engine_impulse_baseline.bin`), `parity_check.cpp` + `parity_wasm.mjs` + `parity_diff.mjs` — native-vs-WASM bit-equivalence within 1e-5. - Behaviour: `test_ml_behaviour.cpp` — 20 INVARIANTS of the interaction model (not tuning). Asserts what must hold however the knobs are set: a like is reachable and a re-like overwrites; dislike never yields non-finite weights/outputs in either `AvoidStyle`, and is safe at cold start; repeat dislikes inside `kReplayDedupRadius` deepen ONE negative while distant ones store separately; explore→reroll→undo→exit restores weights *bit-exactly* and over-undoing is safe; `RandomiseOutputs` leaves weights untouched (the outputs-vs-weights randomisation distinction); same seed + same gesture sequence ⇒ bit-identical weights (what makes any behavioural benchmark comparable); example-ring overflow keeps the newest; contradictory examples stay finite; the ExploreAndPlace accessor contract (`placed_output()` while Placing, `committed_output()` after commit — for both place and reposition); a FULLY-masked focus gate freezes every weight; switching mode mid-exploration never strands the net in a randomised scratchpad; and all of it holds at 1×1, 2×8, 1×33, 8×2 and 32×8 shapes. -- Measurement (asserts nothing): `ml_bench.cpp` — the BEHAVIOURAL benchmark. NISPS is a controller, so this measures the shape of the control→parameter mapping and how interaction journeys deform it, never loss alone. Shape-agnostic (`--shape N_IN,H1,H2,H3,N_OUT`, default `2,16,16,16,8`) via `MLPCore`, so "does a wider/deeper net change the UX?" becomes a number; sample points come from a deterministic Kronecker low-discrepancy sequence rather than a raster, which is what makes it work at any input arity. Field metrics: local gain p50/p95, cliff index, dead fraction, range utilisation, rail occupancy, effective dimensionality (participation ratio — trace²/‖C‖²_F, no eigendecomposition). Displacement metrics: at-point, rings, global, **blast ratio**, and collateral damage at the protected positives. **61 scenarios.** Diagnostic (D1 — the geometric-dislike dose decomposed: intended push vs effective LR vs measured movement at 1/10/100/1000 presses). Atomic probes A1–A14: at/around/far-from an example; one dislike under BOTH candidate designs; twice at one point; adjacent-then-return across the dedup radius; near a protected positive; roll-a-patch-and-place; the full explore→audition→place→commit lifecycle; explore-then-cancel; reposition; like-then-dislike in place; dislike-then-repair; focus/solo mask leakage. Journeys J1–J11: positive-only retention curve, randomise-place-only, mixed, branch (one shared prefix, three divergent gestures, replayed from scratch per branch because that is exact under a deterministic RNG), explore-place-only, 120-gesture long session with drift + weight-norm checkpoints, dislike storm, revisit-after-wandering, two-region interference, sweep-and-teach along a continuous path, undo-heavy. Edge cases E1–E13: cold start, single example, contradictory, collinear, corners, capacity overflow, undo exhaustion, minimal shape, identical targets, rail targets, rapid like/dislike alternation, mode-switch mid-exploration, fully-masked dislike. Upstream comparison U1–U3: the older memllib `interfaceRL` (a DDPG actor-critic, recoverable from this repo's own git history at blob `755ff8b`) differs structurally — the user HEARS `actorTarget`, a soft copy updated `target += alpha*(online-target)` at alpha=0.005, and it trains a batch of 4 from replay only every `optimiseDivisor=40` gestures. The critic half is not reproducible here (MLPCore has `train_targets` but not the per-layer gradient extraction the policy-gradient step needs), but both OUTPUT-PATH ideas are: U1 sweeps the soft-target alpha (alpha=1 IS NISPS today, a free control), U2 sweeps the train-every-Nth divisor, U3 compares actor shapes, U4 sweeps the positive-path training dose. **U4 carries a load-bearing caveat**: upstream LRs are RMSProp LRs (memlp `Layer.h:239`, `StaticMLP.h:268`) and `nisps/ml/training.hpp` is SGD-only, so the numbers are not comparable — see ALIGNMENT.md defect 6. Their shared metric is **lurch** — how far the mapping the musician is playing moves per single gesture, averaged over the whole field. Knobs: `--spread` (1 = Xavier, 0 = uniform with NO fan_in coupling — i.e. the post-removal behaviour, measurable before paying for the refactor), `--geo-lr`, `--geo-iters`. Driven by `scripts/bench-ml.sh`. **Two contracts that fail SILENTLY and are pinned by tests:** (1) a thumbs-up must go through BOTH `mlp.add_example` AND `fb.store_positive` — `dislike_geometric` k-NNs the replay buffer, not the MLP dataset, so a harness that only calls `add_example` measures the cold-start branch instead; (2) `placed_output()` is valid ONLY while state is `Placing` — after `commit_place()`/`commit_reposition()` the vector moves to `committed_output()`, and reading the wrong one yields an empty span whose `l2()` is 0, i.e. a broken lifecycle scored as a perfect placement. +- Measurement (asserts nothing): `ml_bench.cpp` — the BEHAVIOURAL benchmark. NISPS is a controller, so this measures the shape of the control→parameter mapping and how interaction journeys deform it, never loss alone. Shape-agnostic (`--shape N_IN,H1,H2,H3,N_OUT`, default `2,16,16,16,8`) via `MLPCore`, so "does a wider/deeper net change the UX?" becomes a number; sample points come from a deterministic Kronecker low-discrepancy sequence rather than a raster, which is what makes it work at any input arity. Field metrics: local gain p50/p95, cliff index, dead fraction, range utilisation, rail occupancy, effective dimensionality (participation ratio — trace²/‖C‖²_F, no eigendecomposition). Displacement metrics: at-point, rings, global, **blast ratio**, and collateral damage at the protected positives. **61 scenarios.** Diagnostic (D1 — the geometric-dislike dose decomposed: intended push vs effective LR vs measured movement at 1/10/100/1000 presses). Atomic probes A1–A14: at/around/far-from an example; one dislike under BOTH candidate designs; twice at one point; adjacent-then-return across the dedup radius; near a protected positive; roll-a-patch-and-place; the full explore→audition→place→commit lifecycle; explore-then-cancel; reposition; like-then-dislike in place; dislike-then-repair; focus/solo mask leakage. Journeys J1–J11: positive-only retention curve, randomise-place-only, mixed, branch (one shared prefix, three divergent gestures, replayed from scratch per branch because that is exact under a deterministic RNG), explore-place-only, 120-gesture long session with drift + weight-norm checkpoints, dislike storm, revisit-after-wandering, two-region interference, sweep-and-teach along a continuous path, undo-heavy. Edge cases E1–E13: cold start, single example, contradictory, collinear, corners, capacity overflow, undo exhaustion, minimal shape, identical targets, rail targets, rapid like/dislike alternation, mode-switch mid-exploration, fully-masked dislike. Upstream comparison U1–U3: the older memllib `interfaceRL` (a DDPG actor-critic, recoverable from this repo's own git history at blob `755ff8b`) differs structurally — the user HEARS `actorTarget`, a soft copy updated `target += alpha*(online-target)` at alpha=0.005, and it trains a batch of 4 from replay only every `optimiseDivisor=40` gestures. The critic half is not reproducible here (MLPCore has `train_targets` but not the per-layer gradient extraction the policy-gradient step needs), but both OUTPUT-PATH ideas are: U1 sweeps the soft-target alpha (alpha=1 IS NISPS today, a free control), U2 sweeps the train-every-Nth divisor, U3 compares actor shapes, U4 sweeps the positive-path training dose. U4's numbers became comparable to upstream's on 2026-07-25, when `nisps/ml/training.hpp` stopped being SGD-only and ported upstream's RMSProp — before that an upstream LR meant something different here than there, which is what made the geometric dislike inert (D1: 5.3e-5 per press before, 1.6e-2 after). Their shared metric is **lurch** — how far the mapping the musician is playing moves per single gesture, averaged over the whole field. Knobs: `--spread` (1 = Xavier, 0 = uniform with NO fan_in coupling — i.e. the post-removal behaviour, measurable before paying for the refactor), `--geo-lr`, `--geo-iters`. Driven by `scripts/bench-ml.sh`. **Two contracts that fail SILENTLY and are pinned by tests:** (1) a thumbs-up must go through BOTH `mlp.add_example` AND `fb.store_positive` — `dislike_geometric` k-NNs the replay buffer, not the MLP dataset, so a harness that only calls `add_example` measures the cold-start branch instead; (2) `placed_output()` is valid ONLY while state is `Placing` — after `commit_place()`/`commit_reposition()` the vector moves to `committed_output()`, and reading the wrong one yields an empty span whose `l2()` is 0, i.e. a broken lifecycle scored as a perfect placement. - Measurement (asserts nothing): `engine_bench.cpp` + `bench_report.mjs` — per-engine throughput (ns/sample, blocks/s, realtime factor) for the `process()` hot path. ONE source compiled twice (CMake `nisps_engine_bench` natively, emcc for WASM) so the two targets are comparable without adding a single export to `nisps/wasm/bindings.cpp`. Engines are driven into a working state (transport running + event drain for the sequencers, periodic `note_on` for paf_synth, a noise+sine input bed for the fx/analysis engines) and every row prints its own working-state evidence, so a number produced by an idle engine is visible rather than plausible. Driven by `scripts/bench-engines.sh`. ### `scripts/` — build + verify entry points - `build-firmware.sh`, `flash-firmware.sh`, `build-and-flash-firmware.sh`, `firmware-common.sh` — Arduino-CLI wrapper for RP2350 target with C++20 flag. - `build-wasm.sh` — Emscripten compile producing `manifold/public/nisps.{wasm,js}`. - `build-cpp-tests.sh` — CMake configure + build + ctest (Ninja). -- `parity-check.sh` — runs native + WASM and diffs binary outputs. +- `parity-check.sh` — runs native + WASM and diffs binary outputs. **Gotcha**: it only builds `manifold/public/nisps.{js,wasm}` when they are MISSING, never when they are stale, so after any change under `nisps/` you must run `build-wasm.sh` yourself or you are diffing fresh native against an old WASM — which reports a parity FAILURE that is really a staleness failure (this is how the RMSProp port first "broke" parity). - `bench-ml.sh` — the ML BEHAVIOUR benchmark on native + WASM from one source (same trick as `bench-engines.sh`). `--shape`, `--scenario`, `--smoke`, `--seed`, `--compare`, and `--sweep-shape` (runs the corpus across a ladder of architectures and arities — the knob-sensitivity instrument). **Reports, never asserts**: a cliff index is a description, not a pass/fail. Invariants live in `tests/cpp/test_ml_behaviour.cpp` instead. Reports land in `nisps/build/bench-ml/`. - `bench-engines.sh` — engine throughput on native + WASM; `--compare ` prints per-engine Δ%. **Reports, never asserts** (a wall-clock threshold on shared hardware is meaningless or flaky — same call as the firmware size job). Reports land in `nisps/build/bench/`. - `lint-cpp.sh` — `.f` literal warn + heap/`Arduino.h` violation fail. diff --git a/manifold/public/nisps.wasm b/manifold/public/nisps.wasm index 82c21bf..593193b 100755 Binary files a/manifold/public/nisps.wasm and b/manifold/public/nisps.wasm differ diff --git a/nisps/ml/dynamic_storage.hpp b/nisps/ml/dynamic_storage.hpp index 942c377..daf1fdf 100644 --- a/nisps/ml/dynamic_storage.hpp +++ b/nisps/ml/dynamic_storage.hpp @@ -67,6 +67,8 @@ class DynamicStorage { off_a_[l] = claim(fan_out(l)); off_gw_[l] = claim(fan_in(l) * fan_out(l)); off_gb_[l] = claim(fan_out(l)); + off_sw_[l] = claim(fan_in(l) * fan_out(l)); + off_sb_[l] = claim(fan_out(l)); off_d_[l] = claim(fan_in(l)); off_e_[l] = claim(fan_out(l)); } @@ -143,6 +145,14 @@ class DynamicStorage { template std::span grad_b_l() noexcept { return {arena_ + off_gb_[L], fan_out_l()}; } + // RMSProp running squared-gradient averages (training.hpp). Optimiser + // state, not model state: excluded from weight_count()/copy_weights_to(). + template std::span sq_grad_w_l() noexcept { + return {arena_ + off_sw_[L], fan_in_l() * fan_out_l()}; + } + template std::span sq_grad_b_l() noexcept { + return {arena_ + off_sb_[L], fan_out_l()}; + } template std::span delta_l() noexcept { return {arena_ + off_d_[L], fan_in_l()}; } @@ -186,6 +196,7 @@ class DynamicStorage { off_w_[l] = o.off_w_[l]; off_b_[l] = o.off_b_[l]; off_pa_[l] = o.off_pa_[l]; off_a_[l] = o.off_a_[l]; off_gw_[l] = o.off_gw_[l]; off_gb_[l] = o.off_gb_[l]; + off_sw_[l] = o.off_sw_[l]; off_sb_[l] = o.off_sb_[l]; off_d_[l] = o.off_d_[l]; off_e_[l] = o.off_e_[l]; } off_input_ = o.off_input_; off_output_ = o.off_output_; @@ -200,6 +211,7 @@ class DynamicStorage { std::size_t max_iter_ = 0u; std::size_t off_w_[kNumLayers]{}, off_b_[kNumLayers]{}, off_pa_[kNumLayers]{}, off_a_[kNumLayers]{}, off_gw_[kNumLayers]{}, off_gb_[kNumLayers]{}, + off_sw_[kNumLayers]{}, off_sb_[kNumLayers]{}, off_d_[kNumLayers]{}, off_e_[kNumLayers]{}; std::size_t off_input_ = 0u, off_output_ = 0u, off_dsf_ = 0u, off_dsl_ = 0u, off_flat_ = 0u, off_lh_ = 0u; diff --git a/nisps/ml/mlp.hpp b/nisps/ml/mlp.hpp index 245f36a..560f17f 100644 --- a/nisps/ml/mlp.hpp +++ b/nisps/ml/mlp.hpp @@ -182,7 +182,8 @@ class MLPCore : public Storage { } const TrainConfig& train_config() const noexcept { return train_config_; } - // Full SGD training. `sample_weights`, if non-empty, must size to the + // Full per-sample training (RMSProp — see training.hpp). `sample_weights`, + // if non-empty, must size to the // current example count and sum to 1.0 (caller's responsibility — we // do NOT renormalize). // @@ -204,7 +205,7 @@ class MLPCore : public Storage { float epoch_loss = 0.f; for (std::size_t iter = 0; iter < max_iter; ++iter) { epoch_loss = 0.f; - // SGD: per-sample forward → loss → backprop+update. The order + // Per-sample forward → loss → backprop+update. The order // is the dataset insertion order; we do not shuffle (matches // the legacy `Train()` exactly — `TrainBatch` shuffles, but // we're not implementing batch yet). @@ -231,7 +232,7 @@ class MLPCore : public Storage { // Backprop with the same w as the gradient scaler. backprop_(x, deriv, w); - // Apply gradient (per-sample, SGD). + // Apply gradient (per-sample, RMSProp). apply_grad_<3u>(lr); apply_grad_<2u>(lr); apply_grad_<1u>(lr); @@ -316,11 +317,25 @@ class MLPCore : public Storage { clear_grad_<3u>(); } - // Concept reset: clear weights, dataset, and loss history. Seed is - // intentionally NOT reset (use `seed()` for that). + // Zero the RMSProp running squared-gradient averages (upstream + // `MLP::ResetOptimizerState`, MLP.h:205). Note `draw_weights()` + // deliberately does NOT call this: upstream's `DrawWeights` leaves the + // optimiser state alone, so a randomise gesture keeps the step-size + // statistics it had. Only a full `reset()` clears them. + void reset_optimizer_state() noexcept { + if (!storage_ok_()) return; + clear_sq_grad_<0u>(); + clear_sq_grad_<1u>(); + clear_sq_grad_<2u>(); + clear_sq_grad_<3u>(); + } + + // Concept reset: clear weights, dataset, loss history and optimiser + // state. Seed is intentionally NOT reset (use `seed()` for that). void reset() noexcept { if (!storage_ok_()) return; clear_dataset_(); + reset_optimizer_state(); loss_history_count_ = 0u; // Re-init weights from current rng state with default spread. draw_weights(1.f); @@ -527,24 +542,27 @@ class MLPCore : public Storage { backprop_layer_<0u>(input, this->template delta_l<1u>(), 1.f); } - // Apply accumulated gradient to weights+biases with clipping. Resets - // the accumulators to zero for the next sample/iteration. + // Apply accumulated gradient to weights+biases via RMSProp (clip, advance + // the running squared-gradient average, normalise the step — see + // training.hpp for the ported formula and why it is not SGD). Resets the + // accumulators to zero for the next sample/iteration; the squared-gradient + // averages PERSIST, which is the whole point of the optimiser. template NISPS_FORCE_INLINE void apply_grad_(float lr) noexcept { auto w = this->template weights_l(); auto b = this->template biases_l(); auto gw = this->template grad_w_l(); auto gb = this->template grad_b_l(); + auto sw = this->template sq_grad_w_l(); + auto sb = this->template sq_grad_b_l(); const std::size_t nw = gw.size(); const std::size_t nb = gb.size(); for (std::size_t i = 0; i < nw; ++i) { - const float g = clip_gradient(gw[i]); - w[i] -= lr * g; + w[i] -= rmsprop_step(gw[i], sw[i], lr); gw[i] = 0.f; } for (std::size_t i = 0; i < nb; ++i) { - const float g = clip_gradient(gb[i]); - b[i] -= lr * g; + b[i] -= rmsprop_step(gb[i], sb[i], lr); gb[i] = 0.f; } } @@ -557,6 +575,14 @@ class MLPCore : public Storage { for (std::size_t i = 0; i < gb.size(); ++i) gb[i] = 0.f; } + template + NISPS_FORCE_INLINE void clear_sq_grad_() noexcept { + auto sw = this->template sq_grad_w_l(); + auto sb = this->template sq_grad_b_l(); + for (std::size_t i = 0; i < sw.size(); ++i) sw[i] = 0.f; + for (std::size_t i = 0; i < sb.size(); ++i) sb[i] = 0.f; + } + // const forward pass for diagnostics — writes into the mutable eval // scratch, never the real caches. template diff --git a/nisps/ml/storage.hpp b/nisps/ml/storage.hpp index 175d96a..50f7358 100644 --- a/nisps/ml/storage.hpp +++ b/nisps/ml/storage.hpp @@ -19,7 +19,11 @@ // dims: n_in(), n_out(), fan_in_l(), fan_out_l(), // max_examples(), max_iter_train(), weight_count() // layers: weights_l(), biases_l(), pre_act_l(), act_l(), -// grad_w_l(), grad_b_l(), delta_l() [backprop scratch, +// grad_w_l(), grad_b_l(), +// sq_grad_w_l(), sq_grad_b_l() [RMSProp running +// squared-gradient averages, same shape as the gradient +// accumulators — see nisps/ml/training.hpp], +// delta_l() [backprop scratch, // sized fan_in(L)], eval_act_l() [const-eval scratch, // sized fan_out(L), mutable] // global: input_buf(), output_buf(), ds_features(), ds_labels(), @@ -139,6 +143,16 @@ class FixedStorage { if constexpr (L == 0u) return gb0_; else if constexpr (L == 1u) return gb1_; else if constexpr (L == 2u) return gb2_; else return gb3_; } + // RMSProp running squared-gradient averages (training.hpp). Optimiser + // state, not model state: excluded from weight_count()/copy_weights_to(). + template NISPS_FORCE_INLINE std::span sq_grad_w_l() noexcept { + if constexpr (L == 0u) return sw0_; else if constexpr (L == 1u) return sw1_; + else if constexpr (L == 2u) return sw2_; else return sw3_; + } + template NISPS_FORCE_INLINE std::span sq_grad_b_l() noexcept { + if constexpr (L == 0u) return sb0_; else if constexpr (L == 1u) return sb1_; + else if constexpr (L == 2u) return sb2_; else return sb3_; + } // Backprop scratch (delta into layer L's input), sized fan_in(L). template NISPS_FORCE_INLINE std::span delta_l() noexcept { if constexpr (L == 0u) return d0_; else if constexpr (L == 1u) return d1_; @@ -209,6 +223,14 @@ class FixedStorage { std::array gb1_{}; std::array gb2_{}; std::array gb3_{}; + std::array sw0_{}; + std::array sw1_{}; + std::array sw2_{}; + std::array sw3_{}; + std::array sb0_{}; + std::array sb1_{}; + std::array sb2_{}; + std::array sb3_{}; std::array d0_{}; std::array d1_{}; std::array d2_{}; diff --git a/nisps/ml/training.hpp b/nisps/ml/training.hpp index f0adea7..abee198 100644 --- a/nisps/ml/training.hpp +++ b/nisps/ml/training.hpp @@ -1,37 +1,95 @@ -// nisps/ml/training.hpp — gradient-clipping helper and per-layer SGD update. +// nisps/ml/training.hpp — gradient clipping and the RMSProp weight update. // // The MLP class owns the training loop because it knows the dataset layout -// and weight buffers. This header provides: -// - kGradClip: ±10.0, matches the legacy firmware/Layer.h clamp. -// - clip_gradient(): scalar clipper, applied to each accumulated gradient -// before the weight update. +// and weight buffers. This header provides the pieces that are pure scalar +// arithmetic over one accumulated gradient: +// - kGradClip: ±10.0, matches upstream Layer.h's gradientClipValue. +// - clip_gradient(): scalar clipper, applied before the update. +// - rmsprop_step(): the optimiser step — returns the amount to SUBTRACT +// from the weight and advances that weight's running +// squared-gradient average in place. // // The full training loop (forward, loss, backprop, weight update) is // implemented inline in mlp.hpp because everything it touches is either a // member array or layer-templated. Splitting it across translation units // would require type-erasing the layers, which we don't want. // -// Optimizer choice: this MVP ships SGD only. RMSProp is planned (the legacy -// firmware uses it for `TrainBatch`) but the playground was using plain SGD -// until very recently and the XOR-convergence benchmark in the test suite -// is the clearer target. RMSProp can land as a follow-up — see the Ergo -// task notes. For now `train()` is SGD with optional sample-weight -// scaling and gradient clipping. +// OPTIMISER: RMSProp, ported from upstream memlp `Layer.h` +// --------------------------------------------------------------------- +// (github.com/MusicallyEmbodiedML/memlp @ ea777502 — the commit +// MEMLNaut-NISPS `upstream/main` pins). Upstream applies RMSProp EVERYWHERE: +// `Layer.h:239 ApplyAccumulatedGradients`, the `m_sq_grad_avg` running +// average at `Layer.h:601`, `StaticMLP.h:268` "Mini-batch RMSProp training". +// +// This file previously shipped SGD only and called the difference an +// optimiser-choice research question. It was not one. RMSProp divides each +// step by the running gradient magnitude, so an upstream `lr` is a +// NORMALISED step size, whereas under SGD the same number multiplies the raw +// gradient. Every learning rate we ported from upstream — most visibly +// `feedback.hpp`'s `geo_lr_ = 0.001f // upstream InterfaceRL.hpp:312` — +// therefore landed in an optimiser that interprets it completely +// differently, which is why the geometric dislike measured 5.1e-5 of +// movement against an intended 0.5 (`tests/cpp/ml_bench.cpp` D1). +// +// The constants and the clamp order below are upstream's, exactly: +// grad = clip(accumulated_grad) (±10) +// sq = min(0.9*sq + 0.1*grad², 1e6) +// adj = min(lr / (sqrt(sq) + 1e-6), 1.0) +// w -= adj * grad +// Note the adjusted-LR clamp is one-sided, matching upstream's +// `std::min(adj_lr, maxAdjustedLR)`. A NEGATIVE lr (the "train away from +// this target" path in `MLPCore::train_targets`, used by the geometric +// dislike's cold-start fallback) is therefore left unclamped in magnitude, +// exactly as upstream leaves it. +// +// The per-weight squared-gradient average is new persistent STATE. It lives +// in the storage policy (`storage.hpp` FixedStorage / `dynamic_storage.hpp` +// DynamicStorage) alongside the gradient accumulators, so the firmware's +// zero-heap contract holds. It is optimiser state, not model state: it is +// NOT part of `weight_count()` / `get_weights()` / `set_weights()`, matching +// upstream, and `MLPCore::reset_optimizer_state()` (upstream +// `MLP::ResetOptimizerState`) zeroes it. #pragma once +#include + #include "../core/perf.hpp" namespace nisps::ml { -// Per-element gradient clip threshold. Matches the legacy firmware's -// gradientClipValue in Layer.h::ApplyAccumulatedGradients. +// Per-element gradient clip threshold. Matches upstream's gradientClipValue +// in Layer.h::ApplyAccumulatedGradients. inline constexpr float kGradClip = 10.f; +// RMSProp constants — upstream Layer.h:242-244 and the local constants at +// the top of ApplyAccumulatedGradients. +inline constexpr float kRmsPropDecay = 0.9f; +inline constexpr float kRmsPropDecayInv = 0.1f; +inline constexpr float kRmsPropEpsilon = 1.e-6f; +inline constexpr float kMaxSqGradAvg = 1.e6f; +inline constexpr float kMaxAdjustedLr = 1.f; + NISPS_FORCE_INLINE float clip_gradient(float g) noexcept { if (g > kGradClip) return kGradClip; if (g < -kGradClip) return -kGradClip; return g; } +// One RMSProp update for a single weight or bias. `sq_avg` is that element's +// running squared-gradient average and is advanced in place. Returns the +// value to SUBTRACT from the parameter (upstream writes `w -= adj_lr * g`). +NISPS_FORCE_INLINE float rmsprop_step(float grad, float& sq_avg, float lr) noexcept { + const float g = clip_gradient(grad); + + float sq = (kRmsPropDecay * sq_avg) + (kRmsPropDecayInv * g * g); + if (sq > kMaxSqGradAvg) sq = kMaxSqGradAvg; + sq_avg = sq; + + float adj_lr = lr / (std::sqrt(sq) + kRmsPropEpsilon); + if (adj_lr > kMaxAdjustedLr) adj_lr = kMaxAdjustedLr; // one-sided, as upstream + + return adj_lr * g; +} + } // namespace nisps::ml diff --git a/tests/cpp/ml_golden_vectors.cpp b/tests/cpp/ml_golden_vectors.cpp index a5dbc73..0db2909 100644 --- a/tests/cpp/ml_golden_vectors.cpp +++ b/tests/cpp/ml_golden_vectors.cpp @@ -57,6 +57,13 @@ constexpr float kInputY = 0.5f; constexpr float kTol = 1.0e-5f; // Golden vectors captured 2026-04-29 from a clean build of the worktree. +// Stages 2 and 3 RE-CAPTURED 2026-07-25 when the optimiser changed from SGD +// to RMSProp (nisps/ml/training.hpp — the port of upstream memlp Layer.h +// @ ea777502). That is a deliberate behaviour change, not a regression: an +// upstream-tuned learning rate is a normalised step under RMSProp and a raw +// gradient multiplier under SGD, so every ported hyperparameter was landing +// in the wrong optimiser. Stages 0 and 1 are pre-training and did NOT move, +// which is the cross-check that only the update rule changed. // To regenerate: NISPS_REGEN_GOLDEN=1 ./nisps_golden_tests // // The arrays below are the post-process() output vectors at each stage @@ -89,24 +96,24 @@ constexpr std::array kExpectedStage1 = { // Stage 2: after add_example x4 and train(lr=0.5, max_iter=100). constexpr std::array kExpectedStage2 = { - 0.58816862f, 0.54550838f, 0.52971077f, 0.57996541f, 0.51933926f, - 0.52855897f, 0.52070957f, 0.51254886f, 0.63541287f, 0.60544819f, - 0.62713605f, 0.50966084f, 0.54484981f, 0.62081128f, 0.46142119f, - 0.58908224f, 0.53818786f, 0.63540941f, 0.56438410f, 0.48750070f, - 0.57746446f, 0.56682873f, 0.54530638f, 0.62427443f, 0.62183237f, - 0.47161084f, 0.62285376f, 0.63356918f, 0.60930848f, 0.54802805f, - 0.60707289f, 0.61082870f, 0.63076299f, + 0.66427404f, 0.62863928f, 0.63347638f, 0.68701935f, 0.62282497f, + 0.59066784f, 0.57926202f, 0.61092430f, 0.75877625f, 0.71219701f, + 0.76448023f, 0.64224732f, 0.67476881f, 0.75132567f, 0.60728127f, + 0.69433212f, 0.66885883f, 0.75783861f, 0.70804310f, 0.65609163f, + 0.69490194f, 0.71871793f, 0.70532608f, 0.78637725f, 0.78410172f, + 0.65154189f, 0.75753516f, 0.78313172f, 0.77768368f, 0.69148761f, + 0.74804193f, 0.76963025f, 0.77354264f, }; // Stage 3: after move_weights(0.1, 0.3) and re-inference. constexpr std::array kExpectedStage3 = { - 0.59288090f, 0.51427215f, 0.53329450f, 0.54284835f, 0.54659188f, - 0.55931354f, 0.49446660f, 0.55678725f, 0.65046465f, 0.57125282f, - 0.59887666f, 0.52882028f, 0.56914681f, 0.65517074f, 0.51438135f, - 0.51590335f, 0.47392485f, 0.63500941f, 0.56648540f, 0.53441441f, - 0.54152828f, 0.55973053f, 0.52789825f, 0.60794514f, 0.62089235f, - 0.43689638f, 0.56897777f, 0.65621388f, 0.60184997f, 0.60134500f, - 0.63753480f, 0.53896642f, 0.60946816f, + 0.66872150f, 0.58160317f, 0.61222225f, 0.60297203f, 0.64177775f, + 0.64809793f, 0.54033780f, 0.63073188f, 0.76591563f, 0.64906687f, + 0.69643915f, 0.63856536f, 0.67001587f, 0.76458490f, 0.63405144f, + 0.61661869f, 0.56021708f, 0.74601728f, 0.64519984f, 0.62616533f, + 0.64741832f, 0.68293601f, 0.62627184f, 0.74945569f, 0.75258166f, + 0.56667519f, 0.69179827f, 0.80808818f, 0.72934091f, 0.74348420f, + 0.75629526f, 0.67320448f, 0.75111967f, }; // Inference helper: set both inputs, run process(), copy outputs into a