diff --git a/ALIGNMENT.md b/ALIGNMENT.md index f6d05e9..82a82d2 100644 --- a/ALIGNMENT.md +++ b/ALIGNMENT.md @@ -63,28 +63,6 @@ command surface to report through. **Rough cost.** Host half is done. On-device: ~a day, and it wants defect 3's serial protocol to have somewhere to send the number. -### 6b. The geometric dislike was ported from a superseded upstream design (2026-07-25) - -**What.** `geo_push.hpp`/`replay.hpp` cite `memllib @ 0a541cc`. `upstream/main` now pins -`e291192`, where the same code has been deliberately redesigned. Upstream: -`kGeometricPushScale` 1.0 (ours 0.5); neg-LR base 1.5 (ours 0.5, `geo_push.hpp:92`); -the `/(1+len)` taper **deleted**, with the comment "a 'no' should clearly move the -mapping away even from a sound already far from the liked region (the taper used to -kill exactly that case)" — ours still applies it at `geo_push.hpp:66`; negatives trained -as a **batch over ALL of them every tick** rather than one item one step; and a fixed -`kDislikeLifetimeMs = 2500` full-strength lifetime replacing the proportional decay we -ported. On the shared constants (dedup radius 0.05, `kCentroidK` 4) we match. - -**Why it blocks the mission.** We are carrying a design upstream diagnosed and fixed, -and the fix is documented in their source comments. On the constants alone the ported -dislike is ~9.4x weaker than upstream's. (The optimiser half of this — an RMSProp LR -pasted into an SGD step — is fixed as of 2026-07-25; see "Recently resolved". A single -dislike now moves the mapping 1.6e-2 instead of 5.3e-5, but that is still ~14x weaker -than the legacy Diffuse design measures in one press, `ml_bench` A4.) - -**Rough cost.** Small — mostly deleting the taper and re-basing three constants, then -re-running `scripts/bench-ml.sh` D1/A4/A7 to confirm. - ### 6d. One like still heaves the whole mapping (2026-07-25) **What.** A thumbs-up trains at `lr 1.0 x 1000 iterations` on every gesture. `ml_bench` @@ -124,11 +102,22 @@ Legacy a-immersive was mobile-first; Manifold is desktop-first. Defer until user - **EOC effects chain, ShapeSeq sequencer, modular engine (Phase E)** — legacy features consciously out of the v1 rewrite; revisit only if a mode wants them. - **Inputs multi-source composition** (2026-06-28, reaffirmed 2026-07-21) — mix-and-match pad+gamepad+MIDI is a recorded, unreversed decision; the UI currently enforces exclusive single-source and the composition machinery sits dormant *by design*. Schedule or keep dormant — but the inputs-spec must stop presenting composition as current behaviour (plan §8). - **Schema content is partially placeholder** (2026-07-21) — 20 anonymous "Param NN" slots across paf_synth/channel_strip/xiasri and copy-pasted ML defaults across all 9 modes. Name them during the first curated-preset pass per mode (plan §6.5c), or shrink `output_size` where the engine allows. -- **Geometric-dislike deliberate divergences** (2026-07-14, one-core P3): (1) the degenerate-branch RNG draws from the controller's deterministic `nisps::Rng`, not libc `rand()` — native==WASM parity holds; (2) upstream's async shuffled two-LR `optimise()` is collapsed into one synchronous `dislike_geometric()` training only the pressed negative's target — behavioural, not bitwise, parity with firmware upstream, by design; (3) `RandomiseMlp` uses `draw_weights(spread)` rather than the old asymmetric ranges. All intentional. +- **Geometric-dislike deliberate divergences** (2026-07-14, one-core P3; reaffirmed 2026-07-25): (1) the degenerate-branch RNG draws from the controller's deterministic `nisps::Rng`, not libc `rand()` — native==WASM parity holds; (2) upstream's async shuffled two-LR `optimise()` is collapsed into one synchronous `dislike_geometric()` training only the pressed negative's target — behavioural, not bitwise, parity with firmware upstream, by design; (3) `RandomiseMlp` uses `draw_weights(spread)` rather than the old asymmetric ranges. All intentional. The 2026-07-25 re-base onto `e291192` took upstream's constants and deleted the taper but did NOT adopt (2)'s counterpart — upstream retrains a batch over ALL live negatives on every 200 Hz tick and holds each negative at full strength for a fixed `kDislikeLifetimeMs = 2500` instead of decaying it proportionally. Adopting that needs a per-tick call site and a millisecond clock inside `nisps/ml`, which the core does not have; it is a real interface decision, not a constant, and is tracked as its own task. - **Manifold dock splits `state`/`muted`/`armed`** (2026-06-28) — deliberate divergence from the deployed conflated `frozen`↔`muted` model (dock-spec §3.3). `muted`-downstream and the `soloMode` gradient-mask variants remain UI-only; the C API exposes `set_focus` but no per-mode gradient masking yet. (The audit found `soloMode` behaviourally inert in the controller — plan L20 trims it until `train_masked` exists.) ## Recently resolved (delete after a few weeks) +- 2026-07-25: **The geometric dislike is re-based on upstream `e291192` (defect 6b).** + `kGeometricPushScale` 0.5 -> 1.0, `kNegLRBase` 0.5 -> 1.5, and the `/(1+len)` taper + deleted — upstream's own comment is that a "no" should clearly move the mapping away + even from a sound already far from the liked region, which is exactly the case the + taper killed. Cold start folded into the same path (random direction when nothing is + liked yet) instead of the superseded negative-LR branch, so a "no" now moves the + mapping before any likes exist (`ml_bench` E1: 0 -> 2.3e-3). One dislike moves the + mapping **5.3e-2**, up from 1.6e-2 after the RMSProp fix and 5.3e-5 before it — a + ~1000x change end to end, and now within ~4x of the legacy Diffuse design instead of + ~4100x (`ml_bench` A4). Not adopted: upstream's per-tick batch retraining and fixed + 2500 ms dislike lifetime — see "Deferred / accepted debt". - 2026-07-25: **`InterfaceRL` is back in the tree (defect 6c).** Vendored verbatim from memllib `e291192` at `firmware/MEMLNaut-NISPS/lib/memllib/reference/` — outside `src/`, so PlatformIO never compiles it. Upstream drift in the feedback subsystem is a `diff` diff --git a/MAP.md b/MAP.md index 26f11a9..286386e 100644 --- a/MAP.md +++ b/MAP.md @@ -6,7 +6,7 @@ MEMLNaut-NISPS — Neural Interactive Shaping of Parameter Spaces. One C++20 cod ### `nisps/` — platform-agnostic C++20 library (the only ML/DSP/engine code) - `nisps/core/` — `perf.hpp` (hot-path/inlining attrs), `types.hpp`, `concepts.hpp` (`MLEngine`, `AudioEngine`, `Mode`), `ring_buffer.hpp` (SPSC lock-free cross-core channel, replaces pico/util/queue), `event_queue.hpp` (single-threaded in-engine event FIFO — deliberately NOT RingBuffer, which is an atomics-based cross-thread channel), `rng.hpp` (xoshiro256+ deterministic), `math.hpp` (fast_sigmoid, `Curve` enum + `apply_curve`). -- `nisps/ml/` — the MLP core, written once against a storage policy (`mlp.hpp` `MLPCore`): `storage.hpp` (`FixedStorage` — template-sized `std::array`, zero heap; `MLP` alias preserves the classic compile-time surface) and `dynamic_storage.hpp` (`DynamicStorage` — runtime dims, single arena alloc at construction; `#error`s on RP2350 builds, sole lint heap-allowlist entry). Fixed↔dynamic bit-parity enforced by `tests/cpp/test_mlp_storage_parity.cpp`. Files: `mlp.hpp`, `activations.hpp`, `loss.hpp` (MSE, no double-scaling), `training.hpp` (**RMSProp** + grad clipping — `rmsprop_step()` is a line-for-line port of upstream memlp `Layer.h:239 ApplyAccumulatedGradients` @ `ea777502`; the per-weight running squared-gradient average is optimiser state held in the storage policies, NOT part of `weight_count()`/`get_weights()`, and `MLPCore::reset_optimizer_state()` clears it. `draw_weights()` deliberately does not, matching upstream `DrawWeights`), `init.hpp` (spread-aware uniform↔Xavier), `rl.hpp` (`move_weights` with output pin mask + per-layer scaling + weight decay), `jolt.hpp` (`Jolt` — held continuous weight-morph over the flat weight buffer + post-release LR ramp; ported from upstream InterfaceRL), `ou_noise.hpp` (`OUNoise` — Ornstein-Uhlenbeck exploration walk on the output vector; ported from upstream InterfaceRL), `feedback.hpp` (`FeedbackControllerCore` — the "Down Action" state machine: Avoid (geometric push-away default / Diffuse legacy) / RandomiseOutputs / RandomiseMlp / ExploreAndPlace; storage-policied like the MLP, own deterministic RNG, exposed via `nisps_ml_feedback_*` C API), `replay.hpp` (`ReplayView` — reward-tagged memory: dedup/deepen, k-NN positive centroid with deterministic tie-break, proportional decay+eviction), `geo_push.hpp` (push-away target computation, upstream InterfaceRL @ 0a541cc), `warm_start.hpp` (overlapping-weights copy for reshape), `stats.hpp`. `generated/ml_defaults.hpp` is codegen output (do not edit): `nisps::ml::generated::kMlTrainDefaults`, the ONE learning-rate / max-iterations / min-error default shared by firmware, WASM and VCV (source `schemas/ml_defaults.json`); `MLPCore::set_train_config()` and `nisps_ml_set_train_config()` override it at runtime. It lives under `ml/` rather than `modes/generated/` because `nisps/ml` sits below `nisps/modes` — mlp.hpp must not include upward. Jolt + OU are inert by default and wired into `ModeBase`, so every mode exposes `jolt_press/jolt_release`, `jolt_lr_scale`, and `set_explore_intensity`. +- `nisps/ml/` — the MLP core, written once against a storage policy (`mlp.hpp` `MLPCore`): `storage.hpp` (`FixedStorage` — template-sized `std::array`, zero heap; `MLP` alias preserves the classic compile-time surface) and `dynamic_storage.hpp` (`DynamicStorage` — runtime dims, single arena alloc at construction; `#error`s on RP2350 builds, sole lint heap-allowlist entry). Fixed↔dynamic bit-parity enforced by `tests/cpp/test_mlp_storage_parity.cpp`. Files: `mlp.hpp`, `activations.hpp`, `loss.hpp` (MSE, no double-scaling), `training.hpp` (**RMSProp** + grad clipping — `rmsprop_step()` is a line-for-line port of upstream memlp `Layer.h:239 ApplyAccumulatedGradients` @ `ea777502`; the per-weight running squared-gradient average is optimiser state held in the storage policies, NOT part of `weight_count()`/`get_weights()`, and `MLPCore::reset_optimizer_state()` clears it. `draw_weights()` deliberately does not, matching upstream `DrawWeights`), `init.hpp` (spread-aware uniform↔Xavier), `rl.hpp` (`move_weights` with output pin mask + per-layer scaling + weight decay), `jolt.hpp` (`Jolt` — held continuous weight-morph over the flat weight buffer + post-release LR ramp; ported from upstream InterfaceRL), `ou_noise.hpp` (`OUNoise` — Ornstein-Uhlenbeck exploration walk on the output vector; ported from upstream InterfaceRL), `feedback.hpp` (`FeedbackControllerCore` — the "Down Action" state machine: Avoid (geometric push-away default / Diffuse legacy) / RandomiseOutputs / RandomiseMlp / ExploreAndPlace; storage-policied like the MLP, own deterministic RNG, exposed via `nisps_ml_feedback_*` C API), `replay.hpp` (`ReplayView` — reward-tagged memory: dedup/deepen, k-NN positive centroid with deterministic tie-break, proportional decay+eviction), `geo_push.hpp` (push-away target computation, upstream InterfaceRL @ `e291192` — re-based from `0a541cc` on 2026-07-25: `kGeometricPushScale` 1.0, `kNegLRBase` 1.5, and NO `/(1+len)` taper, so distance from the liked centroid does not shrink a "no". Cold start is the same push in a random direction, not a separate branch), `warm_start.hpp` (overlapping-weights copy for reshape), `stats.hpp`. `generated/ml_defaults.hpp` is codegen output (do not edit): `nisps::ml::generated::kMlTrainDefaults`, the ONE learning-rate / max-iterations / min-error default shared by firmware, WASM and VCV (source `schemas/ml_defaults.json`); `MLPCore::set_train_config()` and `nisps_ml_set_train_config()` override it at runtime. It lives under `ml/` rather than `modes/generated/` because `nisps/ml` sits below `nisps/modes` — mlp.hpp must not include upward. Jolt + OU are inert by default and wired into `ModeBase`, so every mode exposes `jolt_press/jolt_release`, `jolt_lr_scale`, and `set_explore_intensity`. - `nisps/pipeline/` — the control-rate input/output processing chains (P4): `input_chain.hpp` (`InputChain` — invert→deadzone→circular clamp→momentum-modulated zoom→centred power→EMA→momentum; caller-supplied dt, internal clock, fixed velocity ring, serialisable state) and `output_chain.hpp` (`OutputChain` — curve→EMA→slew→freeze(+mask), capacity-templated). Behaviour contract = the retired manifold TS pipelines, pinned by `manifold/tests/fixtures/` and parity stage 7. - `nisps/dsp/` — `biquad.hpp`, `delay.hpp`, `reverb.hpp`, `filter.hpp`, `env.hpp`, `osc.hpp`, `pitch_shift.hpp`, `dc_blocker.hpp`, plus the sequencer primitives shared by the sequencer engines: `ratio_seq.hpp` and `seq_clock.hpp` (bar phasor + MIDI clock + bpm). Lean primitives extracted from maximilian; daisysp PitchShifter replaced with custom granular impl. - `nisps/engines/` — eight audio engines, each satisfying `AudioEngine`: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `breakor.hpp` (sequencer, NoOp audio), `elysiamorf.hpp` (sequencer, NoOp audio), `analysis.hpp` (input-side spectral features). Plus `base.hpp` (`NoOpEngine`, engine_id "thru"). diff --git a/manifold/public/nisps.wasm b/manifold/public/nisps.wasm index 593193b..7c290a0 100755 Binary files a/manifold/public/nisps.wasm and b/manifold/public/nisps.wasm differ diff --git a/nisps/ml/feedback.hpp b/nisps/ml/feedback.hpp index 9dcb9cd..94b15e3 100644 --- a/nisps/ml/feedback.hpp +++ b/nisps/ml/feedback.hpp @@ -416,11 +416,12 @@ class FeedbackControllerCore : public FbStorage { // Thumbs-down at the MLP's CURRENT input with heard action `current_out` // (empty ⇒ the MLP's live outputs). Runs the full upstream sequence: // 1. deepen-or-store the negative (dedup radius 0.05). - // 2. cold start (no positives): train AWAY from the heard action at - // lr * 0.1 * avgRewardNeg (negative LR — upstream fallback). - // 3. else: k-NN(4) positive centroid → push-away target → train toward - // it at lr * negLRRatio, gated by the focus/solo mask. - // 4. proportional decay + eviction of expired negatives. + // 2. k-NN(4) positive centroid — or zeros when nothing is liked yet — + // → push-away target → train toward it at lr * negLRRatio, gated by + // the focus/solo mask. With no positives the push direction is + // random per dim, which is upstream's cold start (there is no + // separate negative-LR branch any more; see the body). + // 3. proportional decay + eviction of expired negatives. template FeedbackAction dislike_geometric(M& mlp, std::span current_out, float lr) noexcept { @@ -439,29 +440,36 @@ class FeedbackControllerCore : public FbStorage { const std::size_t neg_total = replay.negative_count(); const float avg_neg = replay.avg_negative_reward(); - FeedbackAction action; - if (pos_total == 0u) { - // 2. cold-start fallback (InterfaceRL.cpp:746): negative-LR - // training away from the heard action; no geometric push. The - // caller shows the "like a few sounds first" prompt. - mlp.train_targets(x_neg, a_neg, lr * 0.1f * avg_neg, focus_span_()); - action = FeedbackAction::GeometricColdStart; + // 2+3. ONE path, as upstream (InterfaceRL.tpp:696-761). With no likes + // stored there is no centroid to push away from, so the centroid is + // zeros and `have_positives=false` sends every dim in a random + // direction — upstream's `useRandom = !havePositives || ...`. We still + // report GeometricColdStart so the caller can show its "like a few + // sounds first" prompt, but the TRAINING is the same push at the same + // LR either way. (Until 2026-07-25 the cold start instead trained AWAY + // from the heard action at a NEGATIVE lr — a port of the older + // `0a541cc` fallback that upstream has since replaced.) + auto mean = this->centroid_buf(); + const bool have_positives = (pos_total > 0u); + if (have_positives) { + (void)replay.knn_positive_centroid(x_neg, kCentroidK, mean); } else { - // 3. centroid → target → train (InterfaceRL.cpp:602-743). - auto mean = this->centroid_buf(); - const std::size_t used = - replay.knn_positive_centroid(x_neg, kCentroidK, mean); - auto target = this->target_buf(); - compute_push_target(a_neg.subspan(0, (a_neg.size() < n_out) ? a_neg.size() : n_out), - std::span(mean.data(), n_out), - focus_span_(), geo_push_step(avg_neg), rng_, target); - const float ratio = geo_neg_lr_ratio(neg_total, pos_total); - mlp.train_targets(x_neg, std::span(target.data(), n_out), - lr * ratio, focus_span_()); - (void)used; - action = FeedbackAction::GeometricPush; + for (std::size_t j = 0; j < n_out; ++j) mean[j] = 0.f; } + auto target = this->target_buf(); + compute_push_target(a_neg.subspan(0, (a_neg.size() < n_out) ? a_neg.size() : n_out), + std::span(mean.data(), n_out), + focus_span_(), geo_push_step(avg_neg), have_positives, + rng_, target); + const float ratio = geo_neg_lr_ratio(neg_total, pos_total); + mlp.train_targets(x_neg, std::span(target.data(), n_out), + lr * ratio, focus_span_()); + + const FeedbackAction action = have_positives + ? FeedbackAction::GeometricPush + : FeedbackAction::GeometricColdStart; + // 4. decay + evict expired negatives (InterfaceRL.cpp:752-760). replay.decay_negatives(); diff --git a/nisps/ml/geo_push.hpp b/nisps/ml/geo_push.hpp index 364aa0e..077dca1 100644 --- a/nisps/ml/geo_push.hpp +++ b/nisps/ml/geo_push.hpp @@ -2,22 +2,32 @@ // dislike gesture (docs/adr/rl-feedback-design.md §2.1/§4). // // Verbatim port of the per-negative target computation in upstream -// InterfaceRL.cpp:713-735 (memllib @ 0a541cc): +// InterfaceRL.tpp:723-760 (memllib @ e291192 — vendored read-only at +// firmware/MEMLNaut-NISPS/lib/memllib/reference/): // -// pushStep = clamp(|avgRewardNeg|, 0.25, 1.0) * kGeometricPushScale -// dir[j] = neg_action[j] - meanPositiveAction[j] -// len = ||dir|| -// useRandom = (len <= 1e-4) (disliked ON the centroid) -// effectivePushStep = pushStep / (1 + len) (taper for far items) -// d = useRandom ? random ∈ [-1,1] : dir[j] / len -// target[j] = clamp(neg_action[j] + d * effectivePushStep, 0, 1) +// pushStep = clamp(|avgRewardNeg|, 0.25, 1.0) * kGeometricPushScale +// dir[j] = neg_action[j] - meanPositiveAction[j] (zeros when no likes) +// len = ||dir|| +// useRandom = !havePositives || (len <= 1e-4) +// d = useRandom ? random ∈ [-1,1] : dir[j] / len +// target[j] = clamp(neg_action[j] + d * pushStep, 0, 1) // inactive dims keep neg_action[j] // +// RE-BASED 2026-07-25 from `0a541cc` to `e291192`, where upstream had +// deliberately redesigned this and we had not noticed (the reference impl was +// out of tree — see reference/README.md). Three changes, all upstream's: +// kGeometricPushScale 0.5 -> 1.0, kNegLRBase 0.5 -> 1.5, and the `/(1+len)` +// TAPER DELETED. Upstream's own comment on the taper, at InterfaceRL.tpp:724: +// +// "No taper: a 'no' should clearly move the mapping away even from a sound +// already far from the liked region (the taper used to kill exactly that +// case). Bigger kGeometricPushScale + higher negLRRatio => the sound +// slides away faster/further." +// // SINGLE DELIBERATE FIRMWARE DIVERGENCE (recorded in ALIGNMENT.md): the // upstream `useRandom` branch draws libc `rand() & 0xFF`; we draw from the // caller's deterministic per-instance `nisps::Rng` so native == WASM parity -// holds. The branch only fires when a disliked action sits exactly on the -// centroid. +// holds. // // Pure free functions over spans — no replay/centroid logic in the MLP // kernel, no state, no heap (Anchor-First graft, ADR §0). @@ -34,10 +44,13 @@ namespace nisps::ml { -// Upstream InterfaceRL.hpp:293. -inline constexpr float kGeometricPushScale = 0.5f; +// Upstream InterfaceRL.hpp:409. +inline constexpr float kGeometricPushScale = 1.f; -// pushStep from the mean negative reward (InterfaceRL.cpp:713). +// Upstream InterfaceRL.hpp:410 — `negLRRatio = kNegLRBase - 0.4*negFraction`. +inline constexpr float kNegLRBase = 1.5f; + +// pushStep from the mean negative reward (InterfaceRL.tpp:733). inline float geo_push_step(float avg_reward_neg) noexcept { float mag = (avg_reward_neg < 0.f) ? -avg_reward_neg : avg_reward_neg; if (mag < 0.25f) mag = 0.25f; @@ -47,11 +60,17 @@ inline float geo_push_step(float avg_reward_neg) noexcept { // Compute the push-away target for ONE disliked action. `active_mask` // (1 = active) gates which dims move — empty ⇒ all active (this is the solo/ -// focus mask; upstream `activeDims_`). Writes n_out floats into `target`. +// focus mask; upstream `activeDims_`). `have_positives` is upstream's +// `posMemCount > 0`: with no likes stored yet there is no centroid to push +// away FROM, so every dim goes in a random direction instead (upstream passes +// an all-zero `meanPositiveAction` in that case, and `mean_positive` may be +// zeros here too — the flag, not the vector, is what selects the branch). +// Writes n_out floats into `target`. inline void compute_push_target(std::span neg_action, std::span mean_positive, std::span active_mask, float push_step, + bool have_positives, Rng& rng, std::span target) noexcept { const std::size_t n = neg_action.size(); @@ -62,8 +81,7 @@ inline void compute_push_target(std::span neg_action, len_sq += d * d; } const float len = std::sqrt(len_sq); - const bool use_random = (len <= 1e-4f); - const float effective = push_step / (1.0f + len); + const bool use_random = !have_positives || (len <= 1e-4f); for (std::size_t j = 0; j < n; ++j) { const bool active = @@ -75,21 +93,21 @@ inline void compute_push_target(std::span neg_action, const float d = use_random ? rng.next_float_signed() : ((neg_action[j] - mean_positive[j]) / len); - float t = neg_action[j] + d * effective; + float t = neg_action[j] + d * push_step; if (t < 0.f) t = 0.f; if (t > 1.f) t = 1.f; target[j] = t; } } -// Dynamic LR ratio (InterfaceRL.cpp:742-743): push harder when dislikes are +// Dynamic LR ratio (InterfaceRL.tpp:758-760): push harder when dislikes are // rare, gentler when they flood the buffer. inline float geo_neg_lr_ratio(std::size_t neg_count, std::size_t pos_count) noexcept { const std::size_t total = neg_count + pos_count; const float neg_fraction = (total > 0u) ? static_cast(neg_count) / static_cast(total) : 0.f; - return 0.5f - 0.4f * neg_fraction; + return kNegLRBase - 0.4f * neg_fraction; } } // namespace nisps::ml diff --git a/tests/cpp/test_mlp_geo_dislike.cpp b/tests/cpp/test_mlp_geo_dislike.cpp index 9e42852..f8cb4df 100644 --- a/tests/cpp/test_mlp_geo_dislike.cpp +++ b/tests/cpp/test_mlp_geo_dislike.cpp @@ -149,20 +149,50 @@ NISPS_TEST(geo_push_target_moves_away_from_centroid) { neg[j] = 0.6f; mean[j] = 0.4f; // dir = +0.2 per dim → push increases values } - const float step = nisps::ml::geo_push_step(-1.f); // clamp(1,0.25,1)*0.5 = 0.5 - NISPS_EXPECT_NEAR(step, 0.5f, 1e-7); - nisps::ml::compute_push_target(neg, mean, {}, step, rng, target); + const float step = nisps::ml::geo_push_step(-1.f); // clamp(1,0.25,1)*1.0 = 1.0 + NISPS_EXPECT_NEAR(step, 1.f, 1e-7); + nisps::ml::compute_push_target(neg, mean, {}, step, /*have_positives=*/true, rng, target); for (std::size_t j = 0; j < kNOut; ++j) { NISPS_EXPECT(target[j] > neg[j]); // strictly away from the centroid NISPS_EXPECT(target[j] <= 1.f); } - // Taper: a far-away negative moves LESS than a near one for the same step. - std::array mean_far{}; - std::array target_far{}; - for (std::size_t j = 0; j < kNOut; ++j) mean_far[j] = 0.0f; // larger len - nisps::ml::compute_push_target(neg, mean_far, {}, step, rng, target_far); - NISPS_EXPECT((target_far[0] - neg[0]) < (target[0] - neg[0])); + // NO TAPER (upstream e291192, InterfaceRL.tpp:724): distance from the + // liked centroid must NOT shrink the push. The direction is a unit + // vector either way, so a far-away negative is displaced exactly as far + // as a near one — which is the case the deleted /(1+len) used to kill. + // (Both are clamped at 1.0 here, so compare an unsaturated dim.) + std::array near_neg{}, far_mean{}, near_mean{}; + std::array t_near{}, t_far{}; + for (std::size_t j = 0; j < kNOut; ++j) { + near_neg[j] = 0.5f; + near_mean[j] = 0.49f; // len ~= 0.024 across 6 dims + far_mean[j] = 0.0f; // len ~= 1.22 + } + nisps::ml::compute_push_target(near_neg, near_mean, {}, 0.1f, true, rng, t_near); + nisps::ml::compute_push_target(near_neg, far_mean, {}, 0.1f, true, rng, t_far); + NISPS_EXPECT_NEAR(t_far[0] - near_neg[0], t_near[0] - near_neg[0], 1e-6); +} + +// With nothing liked yet there is no centroid to push away from, so upstream +// pushes in a RANDOM direction per dim rather than doing nothing +// (`useRandom = !havePositives || ...`, InterfaceRL.tpp:745). +NISPS_TEST(geo_push_target_random_direction_when_no_positives) { + nisps::Rng rng(11ull); + std::array neg{}, mean{}, target{}; + for (std::size_t j = 0; j < kNOut; ++j) { neg[j] = 0.5f; mean[j] = 0.5f; } + + nisps::ml::compute_push_target(neg, mean, {}, 0.25f, /*have_positives=*/false, + rng, target); + bool any_moved = false, any_down = false, any_up = false; + for (std::size_t j = 0; j < kNOut; ++j) { + if (target[j] != neg[j]) any_moved = true; + if (target[j] < neg[j]) any_down = true; + if (target[j] > neg[j]) any_up = true; + NISPS_EXPECT(target[j] >= 0.f && target[j] <= 1.f); + } + NISPS_EXPECT(any_moved); + NISPS_EXPECT(any_down && any_up); // per-dim signs, not one global direction } NISPS_TEST(geo_push_respects_active_mask_and_clamps) { @@ -175,7 +205,8 @@ NISPS_TEST(geo_push_respects_active_mask_and_clamps) { mean[j] = 0.01f; } std::array mask{1u, 0u, 1u, 0u, 1u, 0u}; - nisps::ml::compute_push_target(neg, mean, mask, 0.5f, rng, target); + nisps::ml::compute_push_target(neg, mean, mask, 0.5f, /*have_positives=*/true, + rng, target); for (std::size_t j = 0; j < kNOut; ++j) { if (mask[j]) { NISPS_EXPECT(target[j] >= neg[j]); // pushed (and clamped at 1) @@ -197,27 +228,30 @@ NISPS_TEST(geo_dislike_cold_start_then_push) { set_inputs(m, 0.25f, 0.75f); m.process(); - // Cold start: no positives yet → negative-LR fallback. + // Cold start: no positives yet. Since the 2026-07-25 re-base onto + // upstream e291192 this is NOT a separate inert branch — it is the same + // push, in a random direction because there is no centroid to move away + // from. So a 'no' moves the mapping even before anything is liked, which + // is exactly what upstream's comment at InterfaceRL.tpp:724 argues for. auto before = m.get_weights(); std::array snap{}; for (std::size_t i = 0; i < snap.size(); ++i) snap[i] = before[i]; - // With the heard action == the net's own output the MSE derivative is - // zero, so the fallback is INERT — the conservative cold start the ADR - // mandates (never destabilise before any positives exist). const FeedbackAction a1 = fb.on_down(m, {}, 0.1f, 0.5f, {}); NISPS_EXPECT(a1 == FeedbackAction::GeometricColdStart); NISPS_EXPECT(fb.negative_count() == 1u); + bool moved = false; { auto after = m.get_weights(); for (std::size_t i = 0; i < snap.size(); ++i) { - NISPS_ASSERT(after[i] == snap[i]); + if (after[i] != snap[i]) { moved = true; break; } } } + NISPS_EXPECT(moved); - // When the HEARD action differs from the net's raw output (the real - // browser/firmware case — the user hears the post-pipeline vector), the - // negative-LR fallback trains AWAY from it: weights move. + // The same holds when the HEARD action differs from the net's raw output + // (the real browser/firmware case — the user hears the post-pipeline + // vector), which used to be the ONLY case that moved anything. std::array heard{}; { auto outs = m.outputs(); @@ -227,14 +261,6 @@ NISPS_TEST(geo_dislike_cold_start_then_push) { } const FeedbackAction a1b = fb.on_down(m, heard, 0.1f, 0.5f, {}); NISPS_EXPECT(a1b == FeedbackAction::GeometricColdStart); - bool moved = false; - { - auto after = m.get_weights(); - for (std::size_t i = 0; i < snap.size(); ++i) { - if (after[i] != snap[i]) { moved = true; break; } - } - } - NISPS_EXPECT(moved); // Feed positives via the like path, then dislike → geometric push. set_inputs(m, 0.2f, 0.2f);