feat(manifold): replay geometric dislikes over time

This commit is contained in:
monkey-w1n5t0n 2026-07-25 16:14:35 +02:00
parent 7190932191
commit a1cd26ff68
24 changed files with 720 additions and 114 deletions

View file

@ -72,12 +72,11 @@ one thumbs-up can move the mapping somewhere in the space by more than the entir
range. Retention (how much of the previous teaching survives) is 0.38; at `iters=1` it range. Retention (how much of the previous teaching survives) is 0.38; at `iters=1` it
is 0.80. is 0.80.
**Why it blocks the mission.** This is the other half of the feedback asymmetry, and **Why it blocks the mission.** RMSProp did NOT fix the positive lurch — normalising the
RMSProp did NOT fix it — normalising the step size does not change the dose. Upstream step size does not change its dose. The negative path now exposes upstream-style repeated
keeps both directions on small repeated steps; we take one enormous positive step and small steps (rate/lifetime/LR are live controls), so its old fixed ~70x comparison is no
one small negative one, so teaching feels like a lurch and correcting feels like longer current. Positive teaching remains one enormous blocking train, and the two doses
nothing. The two numbers now differ by ~70x rather than ~2e6x, which is progress and still need a matched head-to-head rather than independent tuning.
still not a design.
**Rough cost.** Cheap to change, expensive to choose: the tuning space is now measurable **Rough cost.** Cheap to change, expensive to choose: the tuning space is now measurable
(`ml_bench` U4 sweeps dose; U1 sweeps upstream's soft-target alpha, where alpha=1 is (`ml_bench` U4 sweeps dose; U1 sweeps upstream's soft-target alpha, where alpha=1 is
@ -102,7 +101,7 @@ Legacy a-immersive was mobile-first; Manifold is desktop-first. Defer until user
- **EOC effects chain, ShapeSeq sequencer, modular engine (Phase E)** — legacy features consciously out of the v1 rewrite; revisit only if a mode wants them. - **EOC effects chain, ShapeSeq sequencer, modular engine (Phase E)** — legacy features consciously out of the v1 rewrite; revisit only if a mode wants them.
- **Inputs multi-source composition** (2026-06-28, reaffirmed 2026-07-21) — mix-and-match pad+gamepad+MIDI is a recorded, unreversed decision; the UI currently enforces exclusive single-source and the composition machinery sits dormant *by design*. Schedule or keep dormant — but the inputs-spec must stop presenting composition as current behaviour (plan §8). - **Inputs multi-source composition** (2026-06-28, reaffirmed 2026-07-21) — mix-and-match pad+gamepad+MIDI is a recorded, unreversed decision; the UI currently enforces exclusive single-source and the composition machinery sits dormant *by design*. Schedule or keep dormant — but the inputs-spec must stop presenting composition as current behaviour (plan §8).
- **Schema content is partially placeholder** (2026-07-21) — 20 anonymous "Param NN" slots across paf_synth/channel_strip/xiasri and copy-pasted ML defaults across all 9 modes. Name them during the first curated-preset pass per mode (plan §6.5c), or shrink `output_size` where the engine allows. - **Schema content is partially placeholder** (2026-07-21) — 20 anonymous "Param NN" slots across paf_synth/channel_strip/xiasri and copy-pasted ML defaults across all 9 modes. Name them during the first curated-preset pass per mode (plan §6.5c), or shrink `output_size` where the engine allows.
- **Geometric-dislike deliberate divergences** (2026-07-14, one-core P3; reaffirmed 2026-07-25): (1) the degenerate-branch RNG draws from the controller's deterministic `nisps::Rng`, not libc `rand()` — native==WASM parity holds; (2) upstream's async shuffled two-LR `optimise()` is collapsed into one synchronous `dislike_geometric()` training only the pressed negative's target — behavioural, not bitwise, parity with firmware upstream, by design; (3) `RandomiseMlp` uses `draw_weights(spread)` rather than the old asymmetric ranges. All intentional. The 2026-07-25 re-base onto `e291192` took upstream's constants and deleted the taper but did NOT adopt (2)'s counterpart — upstream retrains a batch over ALL live negatives on every 200 Hz tick and holds each negative at full strength for a fixed `kDislikeLifetimeMs = 2500` instead of decaying it proportionally. Adopting that needs a per-tick call site and a millisecond clock inside `nisps/ml`, which the core does not have; it is a real interface decision, not a constant, and is tracked as its own task. - **Geometric-dislike deliberate divergences** (2026-07-25): (1) the degenerate-branch RNG draws from deterministic `nisps::Rng`, not libc `rand()`; (2) each live rejection computes its liked centroid at its own stored input, rather than upstream reinterpreting every old rejection around the cursor's current position; (3) live negatives are applied as deterministic per-item RMSProp steps rather than one shuffled `TrainBatch`; (4) a repeated nearby rejection refreshes its lifetime (upstream's current dedup path leaves the original timestamp untouched); (5) upstream's default removal of a nearby positive is not yet adopted because Manifold also has a separate positive MLP dataset that would keep pulling; (6) `RandomiseMlp` uses `draw_weights(spread)` rather than old asymmetric ranges. Native↔WASM parity covers the elapsed-time replay path.
- **Manifold dock splits `state`/`muted`/`armed`** (2026-06-28) — deliberate divergence from the deployed conflated `frozen`↔`muted` model (dock-spec §3.3). `muted`-downstream and the `soloMode` gradient-mask variants remain UI-only; the C API exposes `set_focus` but no per-mode gradient masking yet. (The audit found `soloMode` behaviourally inert in the controller — plan L20 trims it until `train_masked` exists.) - **Manifold dock splits `state`/`muted`/`armed`** (2026-06-28) — deliberate divergence from the deployed conflated `frozen`↔`muted` model (dock-spec §3.3). `muted`-downstream and the `soloMode` gradient-mask variants remain UI-only; the C API exposes `set_focus` but no per-mode gradient masking yet. (The audit found `soloMode` behaviourally inert in the controller — plan L20 trims it until `train_masked` exists.)
## Recently resolved (delete after a few weeks) ## Recently resolved (delete after a few weeks)
@ -116,8 +115,10 @@ Legacy a-immersive was mobile-first; Manifold is desktop-first. Defer until user
mapping before any likes exist (`ml_bench` E1: 0 -> 2.3e-3). One dislike moves the mapping before any likes exist (`ml_bench` E1: 0 -> 2.3e-3). One dislike moves the
mapping **5.3e-2**, up from 1.6e-2 after the RMSProp fix and 5.3e-5 before it — a mapping **5.3e-2**, up from 1.6e-2 after the RMSProp fix and 5.3e-5 before it — a
~1000x change end to end, and now within ~4x of the legacy Diffuse design instead of ~1000x change end to end, and now within ~4x of the legacy Diffuse design instead of
~4100x (`ml_bench` A4). Not adopted: upstream's per-tick batch retraining and fixed ~4100x (`ml_bench` A4). The follow-up now adopts upstream's repeated-all-negatives
2500 ms dislike lifetime — see "Deferred / accepted debt". schedule and full-strength wall-clock lifetime through a deterministic elapsed-time
core seam. Manifold defaults to 0.001 LR, 200 Hz and 2500 ms, exposes all three in the
expanded Learning panel, and allows rate/lifetime zero as an explicit one-shot A/B.
- 2026-07-25: **`InterfaceRL` is back in the tree (defect 6c).** Vendored verbatim from - 2026-07-25: **`InterfaceRL` is back in the tree (defect 6c).** Vendored verbatim from
memllib `e291192` at `firmware/MEMLNaut-NISPS/lib/memllib/reference/` — outside `src/`, memllib `e291192` at `firmware/MEMLNaut-NISPS/lib/memllib/reference/` — outside `src/`,
so PlatformIO never compiles it. Upstream drift in the feedback subsystem is a `diff` so PlatformIO never compiles it. Upstream drift in the feedback subsystem is a `diff`

6
MAP.md
View file

@ -6,7 +6,7 @@ MEMLNaut-NISPS — Neural Interactive Shaping of Parameter Spaces. One C++20 cod
### `nisps/` — platform-agnostic C++20 library (the only ML/DSP/engine code) ### `nisps/` — platform-agnostic C++20 library (the only ML/DSP/engine code)
- `nisps/core/``perf.hpp` (hot-path/inlining attrs), `types.hpp`, `concepts.hpp` (`MLEngine`, `AudioEngine`, `Mode`), `ring_buffer.hpp` (SPSC lock-free cross-core channel, replaces pico/util/queue), `event_queue.hpp` (single-threaded in-engine event FIFO — deliberately NOT RingBuffer, which is an atomics-based cross-thread channel), `rng.hpp` (xoshiro256+ deterministic), `math.hpp` (fast_sigmoid, `Curve` enum + `apply_curve`). - `nisps/core/``perf.hpp` (hot-path/inlining attrs), `types.hpp`, `concepts.hpp` (`MLEngine`, `AudioEngine`, `Mode`), `ring_buffer.hpp` (SPSC lock-free cross-core channel, replaces pico/util/queue), `event_queue.hpp` (single-threaded in-engine event FIFO — deliberately NOT RingBuffer, which is an atomics-based cross-thread channel), `rng.hpp` (xoshiro256+ deterministic), `math.hpp` (fast_sigmoid, `Curve` enum + `apply_curve`).
- `nisps/ml/` — the MLP core, written once against a storage policy (`mlp.hpp` `MLPCore<Storage>`): `storage.hpp` (`FixedStorage` — template-sized `std::array`, zero heap; `MLP<NIn,NH1,NH2,NH3,NOut>` alias preserves the classic compile-time surface) and `dynamic_storage.hpp` (`DynamicStorage` — runtime dims, single arena alloc at construction; `#error`s on RP2350 builds, sole lint heap-allowlist entry). Fixed↔dynamic bit-parity enforced by `tests/cpp/test_mlp_storage_parity.cpp`. Files: `mlp.hpp`, `activations.hpp`, `loss.hpp` (MSE, no double-scaling), `training.hpp` (**RMSProp** + grad clipping — `rmsprop_step()` is a line-for-line port of upstream memlp `Layer.h:239 ApplyAccumulatedGradients` @ `ea777502`; the per-weight running squared-gradient average is optimiser state held in the storage policies, NOT part of `weight_count()`/`get_weights()`, and `MLPCore::reset_optimizer_state()` clears it. `draw_weights()` deliberately does not, matching upstream `DrawWeights`), `init.hpp` (spread-aware uniform↔Xavier), `rl.hpp` (`move_weights` with output pin mask + per-layer scaling + weight decay), `jolt.hpp` (`Jolt` — held continuous weight-morph over the flat weight buffer + post-release LR ramp; ported from upstream InterfaceRL), `ou_noise.hpp` (`OUNoise<N>` — Ornstein-Uhlenbeck exploration walk on the output vector; ported from upstream InterfaceRL), `feedback.hpp` (`FeedbackControllerCore<FbStorage>` — the "Down Action" state machine: Avoid (geometric push-away default / Diffuse legacy) / RandomiseOutputs / RandomiseMlp / ExploreAndPlace; storage-policied like the MLP, own deterministic RNG, exposed via `nisps_ml_feedback_*` C API), `replay.hpp` (`ReplayView` — reward-tagged memory: dedup/deepen, k-NN positive centroid with deterministic tie-break, proportional decay+eviction), `geo_push.hpp` (push-away target computation, upstream InterfaceRL @ `e291192` — re-based from `0a541cc` on 2026-07-25: `kGeometricPushScale` 1.0, `kNegLRBase` 1.5, and NO `/(1+len)` taper, so distance from the liked centroid does not shrink a "no". Cold start is the same push in a random direction, not a separate branch), `warm_start.hpp` (overlapping-weights copy for reshape), `stats.hpp`. `generated/ml_defaults.hpp` is codegen output (do not edit): `nisps::ml::generated::kMlTrainDefaults`, the ONE learning-rate / max-iterations / min-error default shared by firmware, WASM and VCV (source `schemas/ml_defaults.json`); `MLPCore::set_train_config()` and `nisps_ml_set_train_config()` override it at runtime. It lives under `ml/` rather than `modes/generated/` because `nisps/ml` sits below `nisps/modes` — mlp.hpp must not include upward. Jolt + OU are inert by default and wired into `ModeBase`, so every mode exposes `jolt_press/jolt_release`, `jolt_lr_scale`, and `set_explore_intensity`. - `nisps/ml/` — the MLP core, written once against a storage policy (`mlp.hpp` `MLPCore<Storage>`): `storage.hpp` (`FixedStorage` — template-sized `std::array`, zero heap; `MLP<NIn,NH1,NH2,NH3,NOut>` alias preserves the classic compile-time surface) and `dynamic_storage.hpp` (`DynamicStorage` — runtime dims, single arena alloc at construction; `#error`s on RP2350 builds, sole lint heap-allowlist entry). Fixed↔dynamic bit-parity enforced by `tests/cpp/test_mlp_storage_parity.cpp`. Files: `mlp.hpp`, `activations.hpp`, `loss.hpp` (MSE, no double-scaling), `training.hpp` (**RMSProp** + grad clipping — `rmsprop_step()` is a line-for-line port of upstream memlp `Layer.h:239 ApplyAccumulatedGradients` @ `ea777502`; the per-weight running squared-gradient average is optimiser state held in the storage policies, NOT part of `weight_count()`/`get_weights()`, and `MLPCore::reset_optimizer_state()` clears it. `draw_weights()` deliberately does not, matching upstream `DrawWeights`), `init.hpp` (spread-aware uniform↔Xavier), `rl.hpp` (`move_weights` with output pin mask + per-layer scaling + weight decay), `jolt.hpp` (`Jolt` — held continuous weight-morph over the flat weight buffer + post-release LR ramp; ported from upstream InterfaceRL), `ou_noise.hpp` (`OUNoise<N>` — Ornstein-Uhlenbeck exploration walk on the output vector; ported from upstream InterfaceRL), `feedback.hpp` (`FeedbackControllerCore<FbStorage>` — the "Down Action" state machine: Avoid (geometric push-away default / Diffuse legacy) / RandomiseOutputs / RandomiseMlp / ExploreAndPlace; geometric replay is advanced deterministically by caller-supplied elapsed time with configurable rate/lifetime; storage-policied like the MLP, own deterministic RNG, exposed via `nisps_ml_feedback_*` C API), `replay.hpp` (`ReplayView` — reward-tagged memory: dedup/deepen, k-NN positive centroid with deterministic tie-break, wall-clock age/lifetime + eviction), `geo_push.hpp` (push-away target computation, upstream InterfaceRL @ `e291192` — re-based from `0a541cc` on 2026-07-25: `kGeometricPushScale` 1.0, `kNegLRBase` 1.5, and NO `/(1+len)` taper, so distance from the liked centroid does not shrink a "no". Cold start is the same push in a random direction, not a separate branch), `warm_start.hpp` (overlapping-weights copy for reshape), `stats.hpp`. `generated/ml_defaults.hpp` is codegen output (do not edit): `nisps::ml::generated::kMlTrainDefaults`, the ONE learning-rate / max-iterations / min-error default shared by firmware, WASM and VCV (source `schemas/ml_defaults.json`); `MLPCore::set_train_config()` and `nisps_ml_set_train_config()` override it at runtime. It lives under `ml/` rather than `modes/generated/` because `nisps/ml` sits below `nisps/modes` — mlp.hpp must not include upward. Jolt + OU are inert by default and wired into `ModeBase`, so every mode exposes `jolt_press/jolt_release`, `jolt_lr_scale`, and `set_explore_intensity`.
- `nisps/pipeline/` — the control-rate input/output processing chains (P4): `input_chain.hpp` (`InputChain` — invert→deadzone→circular clamp→momentum-modulated zoom→centred power→EMA→momentum; caller-supplied dt, internal clock, fixed velocity ring, serialisable state) and `output_chain.hpp` (`OutputChain<NMax>` — curve→EMA→slew→freeze(+mask), capacity-templated). Behaviour contract = the retired manifold TS pipelines, pinned by `manifold/tests/fixtures/` and parity stage 7. - `nisps/pipeline/` — the control-rate input/output processing chains (P4): `input_chain.hpp` (`InputChain` — invert→deadzone→circular clamp→momentum-modulated zoom→centred power→EMA→momentum; caller-supplied dt, internal clock, fixed velocity ring, serialisable state) and `output_chain.hpp` (`OutputChain<NMax>` — curve→EMA→slew→freeze(+mask), capacity-templated). Behaviour contract = the retired manifold TS pipelines, pinned by `manifold/tests/fixtures/` and parity stage 7.
- `nisps/dsp/``biquad.hpp`, `delay.hpp`, `reverb.hpp`, `filter.hpp`, `env.hpp`, `osc.hpp`, `pitch_shift.hpp`, `dc_blocker.hpp`, plus the sequencer primitives shared by the sequencer engines: `ratio_seq.hpp` and `seq_clock.hpp` (bar phasor + MIDI clock + bpm). Lean primitives extracted from maximilian; daisysp PitchShifter replaced with custom granular impl. - `nisps/dsp/``biquad.hpp`, `delay.hpp`, `reverb.hpp`, `filter.hpp`, `env.hpp`, `osc.hpp`, `pitch_shift.hpp`, `dc_blocker.hpp`, plus the sequencer primitives shared by the sequencer engines: `ratio_seq.hpp` and `seq_clock.hpp` (bar phasor + MIDI clock + bpm). Lean primitives extracted from maximilian; daisysp PitchShifter replaced with custom granular impl.
- `nisps/engines/` — eight audio engines, each satisfying `AudioEngine`: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `breakor.hpp` (sequencer, NoOp audio), `elysiamorf.hpp` (sequencer, NoOp audio), `analysis.hpp` (input-side spectral features). Plus `base.hpp` (`NoOpEngine`, engine_id "thru"). - `nisps/engines/` — eight audio engines, each satisfying `AudioEngine`: `paf_synth.hpp`, `channel_strip.hpp`, `xiasri.hpp`, `verb_fx.hpp`, `memlcelium.hpp`, `breakor.hpp` (sequencer, NoOp audio), `elysiamorf.hpp` (sequencer, NoOp audio), `analysis.hpp` (input-side spectral features). Plus `base.hpp` (`NoOpEngine`, engine_id "thru").
@ -85,8 +85,8 @@ anchor + locked decisions) and the `docs/specs/*-spec.md` set.
BATCH "MIDI Learn": every CC swept while armed becomes an axis, shown as read-only meters). `useInputLayer.ts` BATCH "MIDI Learn": every CC swept while armed becomes an axis, shown as read-only meters). `useInputLayer.ts`
is the React binding; `base-source.ts` shared status/action plumbing; `types.ts` the adapter contract. is the React binding; `base-source.ts` shared status/action plumbing; `types.ts` the adapter contract.
`backends/base-backend.ts` is its output-side counterpart (status + throttle + lastSent) used by the midi/osc/vcv transports. `backends/base-backend.ts` is its output-side counterpart (status + throttle + lastSent) used by the midi/osc/vcv transports.
- `manifold/src/feedback/``controller.ts` (Explore-and-place scratchpad + geometric-dislike + solo; a thin - `manifold/src/feedback/``controller.ts` (Explore-and-place scratchpad + geometric-dislike + solo; owns the
driver over the shared C++ core). elapsed-time timer for geometric replay while every weight-affecting step stays in the shared C++ core).
- `manifold/src/settings/``settings-store.ts` (monochrome icons, input-map shape, I/O resize policy, corner radius, and the opt-in legacy Xavier/spread feature flag; Manifold randomisation is full-range uniform by default). - `manifold/src/settings/``settings-store.ts` (monochrome icons, input-map shape, I/O resize policy, corner radius, and the opt-in legacy Xavier/spread feature flag; Manifold randomisation is full-range uniform by default).
- `manifold/src/serial/``memlnaut-serial.ts` Web Serial scaffold + `EditorPanel.tsx` (MEMLNaut Editor mode). - `manifold/src/serial/``memlnaut-serial.ts` Web Serial scaffold + `EditorPanel.tsx` (MEMLNaut Editor mode).
- `manifold/src/engine/exploration.ts` — Jolt press + OU explore gestures (Learning drawer): a thin - `manifold/src/engine/exploration.ts` — Jolt press + OU explore gestures (Learning drawer): a thin

View file

@ -4,6 +4,26 @@
--- ---
### 2026-07-25 implementation amendment
The verified upstream reference is now memllib `e291192`, vendored under
`firmware/MEMLNaut-NISPS/lib/memllib/reference/`. Its geometric constants and schedule
supersede the older `0a541cc` constants and synchronous-one-shot wording below:
- push scale `1.0`, negative-LR base `1.5`, no distance taper; cold start pushes in a
deterministic random direction;
- a press stores the rejection and performs one immediate update, then
`FeedbackControllerCore::advance_geometric(dt)` replays **all** live negatives at a
configurable rate for a full-strength wall-clock lifetime;
- defaults match upstream: LR `0.001`, `200 Hz`, `2500 ms`; rate or lifetime zero is
explicit one-shot mode;
- the host supplies elapsed time, but target computation and every weight mutation stay
in the allocation-free shared C++ core. Native↔WASM parity covers this seam.
Manifold exposes LR/rate/lifetime at expanded Learning depth. Deliberate divergences are
kept current in `ALIGNMENT.md`; source comparison evidence is immutable in
`docs/specs/recon/findings-push-away-upstream-comparison.md`.
## 0. Decision summary ## 0. Decision summary
| Setting | Default | Also selectable | | Setting | Default | Also selectable |
@ -222,4 +242,4 @@ Extend Stage 5: **(S5a Mode 1)** seed `ParityMLP`+`ReplayStore`, store 3 fixed p
2. Record the **single deliberate firmware divergence**: the `useRandom` degenerate branch uses `nisps::Rng`, not libc `rand()` (value generated, never compared; native==WASM holds). 2. Record the **single deliberate firmware divergence**: the `useRandom` degenerate branch uses `nisps::Rng`, not libc `rand()` (value generated, never compared; native==WASM holds).
3. Record the **TrainBatch-vs-unshuffled-SGD** behavioural divergence: nisps trains per-sample SGD, not shuffled two-LR batches — `browser != firmware` behaviourally, by design (separate verification targets). 3. Record the **TrainBatch-vs-unshuffled-SGD** behavioural divergence: nisps trains per-sample SGD, not shuffled two-LR batches — `browser != firmware` behaviourally, by design (separate verification targets).
4. State the **shared-trunk solo limit** as accepted: no realisation both perfectly isolates and stays expressive. 4. State the **shared-trunk solo limit** as accepted: no realisation both perfectly isolates and stays expressive.
5. Re-pin orphaned memllib `4733ca0``0a541cc`. 5. Re-pin orphaned memllib `4733ca0``0a541cc`.

View file

@ -0,0 +1,179 @@
---
kind: finding
date: 2026-07-25
immutable: true
---
# Findings — Manifold “Push away” vs upstream geometric dislike
_Read-only comparison, 2026-07-25. “Confirmed” claims cite source; “Inference” labels
interpretation. No implementation decision is made here._
## Scope and source identity
The hardware repository named in the question,
[`MusicallyEmbodiedML/MEMLNaut`](https://github.com/MusicallyEmbodiedML/MEMLNaut),
does not contain the active ML implementation. The firmware repository is
[`MusicallyEmbodiedML/MEMLNaut-NISPS`](https://github.com/MusicallyEmbodiedML/MEMLNaut-NISPS/tree/701f2d9f1b4698e0ddfa147193928489de12601f),
whose `src/memllib` submodule pins
[`MusicallyEmbodiedML/memllib` at `e291192`](https://github.com/MusicallyEmbodiedML/memllib/tree/e291192d8e4f2fca7b79670c4df9c2ec8bdf03cd).
The local read-only copies of upstream `InterfaceRL.hpp`, `.tpp`, and `.cpp` have Git
blob hashes `03e9255`, `9ec762b`, and `60204d1`, respectively—the same blobs returned
by GitHub for that pin. The comparison below is therefore against the exact upstream
source, not an approximation. Provenance is recorded locally in
`firmware/MEMLNaut-NISPS/lib/memllib/reference/README.md:1-15`.
## Confirmed behaviour
### The mental model is substantially right, but “different” needs a target
A negative verdict has no supervised label by itself. Upstream defines “different” as:
take the output that was heard, find the mean output of the four liked positions nearest
the current control input, and create a new target one unit farther from that liked
centroid in output space. With no likes—or a degenerate zero-length direction—it uses a
random direction. The target is clamped to `[0,1]`
([upstream `InterfaceRL.tpp:698-761`](https://github.com/MusicallyEmbodiedML/memllib/blob/e291192d8e4f2fca7b79670c4df9c2ec8bdf03cd/examples/InterfaceRL.tpp#L698-L761);
local mirror `firmware/MEMLNaut-NISPS/lib/memllib/reference/InterfaceRL.tpp:698-761`).
That target is then passed to `synthMapping.TrainBatch`. Inference subsequently reads
the same MLP through `synthMapping.GetOutput`. Therefore current upstream push-away is
**weight training on the mapping MLP**, not a separate rejection layer after inference
([upstream training call](https://github.com/MusicallyEmbodiedML/memllib/blob/e291192d8e4f2fca7b79670c4df9c2ec8bdf03cd/examples/InterfaceRL.tpp#L757-L761);
[upstream inference path](https://github.com/MusicallyEmbodiedML/memllib/blob/e291192d8e4f2fca7b79670c4df9c2ec8bdf03cd/examples/InterfaceRL.tpp#L869-L890)).
OU noise and `paramTransformHook` do exist after MLP inference, but are independent of
the dislike algorithm (`InterfaceRL.tpp:877-890`).
The Manifold path has the same basic semantics. The UI passes its current post-output-
pipeline vector to `FeedbackController.dislike`, which calls the WASM geometric-dislike
entry point (`manifold/src/console/ConsoleApp.tsx:509-535`;
`manifold/src/feedback/controller.ts:347-373`;
`manifold/src/engine/wasm-iml.ts:956-978`). The C++ controller computes the target and
calls `MLPCore::train_targets`, which runs forward propagation, backpropagation and one
RMSProp update on the same network weights (`nisps/ml/feedback.hpp:395-477`;
`nisps/ml/mlp.hpp:250-287`).
### The target maths now matches current upstream, but the training schedule does not
Current Manifold main now matches upstreams untapered target formula and constants:
`kGeometricPushScale=1.0`, `kNegLRBase=1.5`, no `/(1+distance)` taper
(`nisps/ml/geo_push.hpp:1-110`; upstream `InterfaceRL.hpp:406-414` and
`InterfaceRL.tpp:723-761`). It also now uses the upstream RMSProp update rather than
interpreting an upstream RMSProp learning rate as plain SGD
(`nisps/ml/training.hpp:1-92`).
The remaining major divergence is dose:
- Upstream stores the dislike at press time, then its main loop calls `optimise()` on
subsequent cycles (`InterfaceRL.tpp:38-54,186-232`).
- Every optimisation scans **all** live negatives and batch-trains them again
(`InterfaceRL.tpp:673-761`).
- Upstream computes one liked centroid around the **current live control input** and
applies it to every negative in that cycle, even if the user has moved away from the
original disliked position (`InterfaceRL.tpp:698-755`). Manifold instead computes the
centroid at the just-pressed negatives stored input (`nisps/ml/feedback.hpp:431-467`).
- A negative remains at full strength for 2500 ms, then expires; the number of updates
depends on the modes loop rate (`InterfaceRL.hpp:406-414`).
- Manifold collapses press and optimisation into **one synchronous
`train_targets` call for only the just-pressed negative**, then proportionally decays
stored negatives (`nisps/ml/feedback.hpp:395-477`; `nisps/ml/replay.hpp:165-185`).
There is no background/per-frame feedback optimiser.
Thus a Manifold click names a strongly displaced target, but takes only one step toward
it. Upstream keeps walking toward its target for the next 2.5 seconds. This is the most
direct explanation for a remaining perceptual strength difference.
As checked on 2026-07-25, the WASM served by
`https://meml.lnfinitemonkeys.org/next/nisps.wasm` has SHA-256
`d1c58a59517a00c6f51870ea1ec21194561b81058e22bbfb2e11de4af45c645a`, exactly matching
`manifold/public/nisps.wasm` on current main. The reported live behaviour therefore
cannot be explained by production still serving the pre-RMSProp or tapered binary.
### Upstream also cancels a nearby positive; Manifold does not
Upstreams default replay policy is `REPLACE_10_PERCENT`
(`InterfaceRL.hpp:404-405`). When a negative is stored, a positive within input-space
distance `0.10` is removed before the negative is added
(`InterfaceRL.tpp:904-929,950-979`). Manifolds replay method only deepens or adds a
negative and leaves positives intact (`nisps/ml/replay.hpp:102-121`). Manifold also
keeps liked examples in the separate MLP dataset
(`manifold/src/feedback/controller.ts:375-390`).
**Inference:** this is less about the first clicks amplitude than persistence. A later
positive training run can pull the mapping back toward a sound rejected near an
existing like, whereas upstream removes that local positive from its continuously
trained replay set.
### Current measured scale
On current main, the native behavioural benchmark at shape `2→16→16→16→8`, seed
`24301`, reports:
- one geometric dislike: at-point L2 movement `0.05335`;
- one legacy undirected Diffuse dislike: `0.22626`;
- repeated geometric dislikes: `0.05314` after 1, `0.33748` after 10, `1.09504`
after 100.
Commands:
```bash
scripts/bench-ml.sh --native-only --scenario A4_negative_once
scripts/bench-ml.sh --native-only --scenario D1_geo_anatomy
```
These numbers confirm that the current path is no longer inert, but also that a single
geometric press is still about 4.2× smaller than the legacy random-diffusion gesture in
this benchmark. They do not by themselves establish the right musical feel.
At Manifolds current default PAF shape (`4→10→10→14→33`), the same seeded scenarios
report one-click movement `0.09654` geometric versus `0.20788` Diffuse (about 2.2×
smaller), and geometric movement `0.55935` after ten presses. These are vector L2
distances across 33 parameters, so they establish that weights move; they do not prove
that the affected parameters produce a perceptually obvious timbral change.
The more revealing PAF-shape `A12_like_then_dislike` journey dislikes exactly where a
liked target was taught. With one update, distance from that rejected liked target
changes from `0.38291` to `0.36030` (`rejection_moved=-0.02261`): the mapping moves, but
slightly **toward** the particular target the user just rejected. Ten updates change the
distance to `0.67843` (`rejection_moved=+0.29552`). This is deterministic evidence that
one update is not sufficient to realise the user-facing semantic in an important
contradictory-feedback case; it also motivates testing upstreams nearby-like removal
separately.
## Why it likely felt weak
1. **Fixed today: optimiser mismatch.** Before `f57cddc`, the port used a tiny upstream
RMSProp learning rate inside plain SGD, reducing one press to roughly `5.3e-5`
movement.
2. **Fixed today: superseded push formula.** Before `ec31180`, the port halved the target
step, used one-third the negative-LR base, and tapered the step by distance—the exact
case upstream says made “no” ineffective.
3. **Still present: one update versus a time window of updates.** Manifold performs one
weight update per click; upstream replays all dislikes repeatedly for 2.5 seconds.
4. **Still present: highly asymmetric teaching dose.** A Manifold like trains the whole
positive dataset at `lr=1.0` for up to 1000 iterations, while a dislike takes one
roughly `0.0015` RMSProp step. Relative to the lurch caused by a like, correction
still feels small (`ALIGNMENT.md:66-84`).
5. **Potential persistence mismatch:** nearby likes are retained locally but removed by
upstreams default replay policy, so future positive retraining may partly undo the
rejection.
## Recommended next experiment
Do not start by increasing `geo_lr` blindly. That can make a click stronger but does not
answer whether the intended interaction is one discrete correction or a short-lived
repulsive constraint.
Add a deterministic “advance feedback by `dt`” core seam and benchmark three matched
variants from the same seeded prefix:
1. current one-shot update;
2. upstream-style replay of all negatives for 2500 ms, while separately testing whether
centroid lookup follows each stored negative or upstreams current live input;
3. a bounded discrete equivalent (for example 10/25/50 steps at press time) that avoids
wall-clock ownership in the core.
For each, report at-point movement, neighbourhood rings, global blast ratio, collateral
movement at liked positions, and like→dislike→like persistence. Then perform an
in-Manifold blind A/B at the real per-mode output arities and choose the smallest dose
that makes one rejection obvious without damaging liked regions. Test the nearby-like
removal policy as a separate axis rather than coupling it to dose.

File diff suppressed because one or more lines are too long

Binary file not shown.

View file

@ -29,7 +29,13 @@
*/ */
import { useEffect, useMemo, useRef, useState } from 'react'; import { useEffect, useMemo, useRef, useState } from 'react';
import type { CSSProperties } from 'react'; import type { CSSProperties } from 'react';
import { useEngine, useEngineVersion, ExplorationController } from '../engine'; import {
useEngine,
useEngineVersion,
ExplorationController,
DEFAULT_GEOMETRIC_FEEDBACK_CONFIG,
type GeometricFeedbackConfig,
} from '../engine';
import { MF_MODES, createOutputParam, modeEngineId, shapeValues } from './model'; import { MF_MODES, createOutputParam, modeEngineId, shapeValues } from './model';
import type { MFParam } from './model'; import type { MFParam } from './model';
import { CompositeStage } from './CompositeStage'; import { CompositeStage } from './CompositeStage';
@ -110,6 +116,9 @@ export function ConsoleApp() {
const [pos, setPos] = useState<[number, number]>([0.5, 0.5]); const [pos, setPos] = useState<[number, number]>([0.5, 0.5]);
const [noiseCap, setNoiseCap] = useState(0.12); const [noiseCap, setNoiseCap] = useState(0.12);
const [geometricConfig, setGeometricConfig] = useState<GeometricFeedbackConfig>(() => ({
...DEFAULT_GEOMETRIC_FEEDBACK_CONFIG,
}));
const [examples, setExamples] = useState(0); const [examples, setExamples] = useState(0);
const [addingExample, setAddingExample] = useState(false); const [addingExample, setAddingExample] = useState(false);
const [busy, setBusy] = useState(false); const [busy, setBusy] = useState(false);
@ -186,6 +195,7 @@ export function ConsoleApp() {
if (engine && !controllerRef.current) { if (engine && !controllerRef.current) {
controllerRef.current = new FeedbackController(engine, { controllerRef.current = new FeedbackController(engine, {
spread: randomisationSpread, spread: randomisationSpread,
geometricConfig,
}); });
} }
@ -210,6 +220,8 @@ export function ConsoleApp() {
} }
useEffect( useEffect(
() => () => { () => () => {
controllerRef.current?.dispose();
controllerRef.current = null;
explorationRef.current?.dispose(); explorationRef.current?.dispose();
explorationRef.current = null; explorationRef.current = null;
}, },
@ -248,6 +260,10 @@ export function ConsoleApp() {
controllerRef.current?.setSpread(randomisationSpread); controllerRef.current?.setSpread(randomisationSpread);
}, [engine, randomisationSpread]); }, [engine, randomisationSpread]);
useEffect(() => {
controllerRef.current?.setGeometricConfig(geometricConfig);
}, [engine, geometricConfig]);
// Push the selected solo-mode + the arm mask into the controller whenever they // Push the selected solo-mode + the arm mask into the controller whenever they
// change (dock-spec §1.2). The controller RESPECTS the arm mask at the example // change (dock-spec §1.2). The controller RESPECTS the arm mask at the example
// level in BOTH modes and forwards it to engine.feedback.setFocus. // level in BOTH modes and forwards it to engine.feedback.setFocus.
@ -968,6 +984,8 @@ export function ConsoleApp() {
xavierSpreadEnabled: settings.xavierSpreadEnabled, xavierSpreadEnabled: settings.xavierSpreadEnabled,
noiseCap, noiseCap,
setNoiseCap, setNoiseCap,
geometricConfig,
setGeometricConfig,
// learning-behaviour // learning-behaviour
feedbackMode, feedbackMode,
setFeedbackMode, setFeedbackMode,

View file

@ -31,7 +31,11 @@ import { outputModeDescriptor } from './output-mode';
import { useSettings, unfocusedIconCss } from '../settings/settings-store'; import { useSettings, unfocusedIconCss } from '../settings/settings-store';
import type { UnfocusedIconColour, InputMapMode } from '../settings/settings-store'; import type { UnfocusedIconColour, InputMapMode } from '../settings/settings-store';
import type { ExampleResizePolicy, NetworkResizePolicy } from '../engine/io-reshape'; import type { ExampleResizePolicy, NetworkResizePolicy } from '../engine/io-reshape';
import { useEngine, useEngineVersion } from '../engine'; import {
useEngine,
useEngineVersion,
DEFAULT_GEOMETRIC_FEEDBACK_CONFIG,
} from '../engine';
import { EditorPanel } from '../serial/EditorPanel'; import { EditorPanel } from '../serial/EditorPanel';
import { TrainingHealth } from './TrainingHealth'; import { TrainingHealth } from './TrainingHealth';
import { import {
@ -315,7 +319,76 @@ function LearningDrawer(ctx: ConsoleCtx, depth: DrawerDepth) {
</span> </span>
</div> </div>
<SectionLabel>Live training params</SectionLabel> <SectionLabel>Live training params</SectionLabel>
<Slider label="noise cap" value={ctx.noiseCap} min={0} max={0.5} step={0.01} onChange={ctx.setNoiseCap} /> {ctx.feedbackMode === 'geometric-dislike' ? (
<>
<Slider
label="push · learning rate"
value={ctx.geometricConfig.learningRate}
min={0.0001}
max={0.005}
step={0.0001}
format={(v) => v.toFixed(4)}
onChange={(learningRate) =>
ctx.setGeometricConfig({ ...ctx.geometricConfig, learningRate })
}
/>
<Slider
label="push · updates / second"
value={ctx.geometricConfig.updatesPerSecond}
min={0}
max={400}
step={10}
format={(v) => `${Math.round(v)} Hz`}
onChange={(updatesPerSecond) =>
ctx.setGeometricConfig({ ...ctx.geometricConfig, updatesPerSecond })
}
/>
<Slider
label="push · lifetime"
value={ctx.geometricConfig.lifetimeMs / 1000}
min={0}
max={5}
step={0.1}
format={(v) => `${v.toFixed(1)} s`}
onChange={(seconds) =>
ctx.setGeometricConfig({
...ctx.geometricConfig,
lifetimeMs: seconds * 1000,
})
}
/>
<div style={{ display: 'flex', gap: 8, alignItems: 'center', flexWrap: 'wrap' }}>
<Button
size="sm"
variant="secondary"
onClick={() =>
ctx.setGeometricConfig({ ...DEFAULT_GEOMETRIC_FEEDBACK_CONFIG })
}
>
Upstream defaults
</Button>
<span style={{ fontSize: 'var(--fs-xs)', color: 'var(--fg-dim)' }}>
{ctx.geometricConfig.updatesPerSecond <= 0 ||
ctx.geometricConfig.lifetimeMs <= 0
? 'one immediate update per press'
: `${Math.round(
(ctx.geometricConfig.updatesPerSecond *
ctx.geometricConfig.lifetimeMs) /
1000,
)} replay updates + the press`}
</span>
</div>
</>
) : (
<Slider
label="noise cap"
value={ctx.noiseCap}
min={0}
max={0.5}
step={0.01}
onChange={ctx.setNoiseCap}
/>
)}
</> </>
)} )}

View file

@ -5,6 +5,7 @@ import type { MFMode, MFParam } from './model';
import type { FeedbackMode } from '../engine/types'; import type { FeedbackMode } from '../engine/types';
import type { BackendStatus } from '../backends/backend'; import type { BackendStatus } from '../backends/backend';
import type { UseInputLayer } from '../inputs'; import type { UseInputLayer } from '../inputs';
import type { GeometricFeedbackConfig } from '../engine';
/** The two product feedback modes (dock-spec §1.1; rl-feedback-design §0). */ /** The two product feedback modes (dock-spec §1.1; rl-feedback-design §0). */
export type FeedbackModeUI = 'explore-and-place' | 'geometric-dislike'; export type FeedbackModeUI = 'explore-and-place' | 'geometric-dislike';
@ -124,6 +125,8 @@ export interface ConsoleCtx {
xavierSpreadEnabled: boolean; xavierSpreadEnabled: boolean;
noiseCap: number; noiseCap: number;
setNoiseCap: (v: number) => void; setNoiseCap: (v: number) => void;
geometricConfig: GeometricFeedbackConfig;
setGeometricConfig: (config: GeometricFeedbackConfig) => void;
// ---- Learning-behaviour (dock-spec §1; rl-feedback-design) ---- // ---- Learning-behaviour (dock-spec §1; rl-feedback-design) ----
feedbackMode: FeedbackModeUI; feedbackMode: FeedbackModeUI;

View file

@ -52,6 +52,8 @@ export interface DebugProbe {
* int (14=GeometricPush, 15=GeometricColdStart). * int (14=GeometricPush, 15=GeometricColdStart).
*/ */
dislikeGeometric(heardVec: ReadonlyArray<number>, lr?: number): number; dislikeGeometric(heardVec: ReadonlyArray<number>, lr?: number): number;
setGeometricConfig(learningRate: number, updatesPerSecond: number, lifetimeMs: number): void;
advanceGeometric(dtSeconds: number): number;
/** Feed a positive into the k-NN centroid (omit vec → live MLP output). */ /** Feed a positive into the k-NN centroid (omit vec → live MLP output). */
storePositive(vec?: ReadonlyArray<number>): void; storePositive(vec?: ReadonlyArray<number>): void;
/** Replay-memory sizes (Mode 1). */ /** Replay-memory sizes (Mode 1). */
@ -174,6 +176,20 @@ function makeProbe(engine: EngineApi): DebugProbe {
return a; return a;
}, },
setGeometricConfig(
learningRate: number,
updatesPerSecond: number,
lifetimeMs: number,
): void {
engine.feedback.setGeometricConfig({ learningRate, updatesPerSecond, lifetimeMs });
},
advanceGeometric(dtSeconds: number): number {
const steps = engine.feedback.advanceGeometric(dtSeconds);
if (steps > 0) engine.process();
return steps;
},
storePositive(vec?: ReadonlyArray<number>): void { storePositive(vec?: ReadonlyArray<number>): void {
engine.feedback.storePositive(vec ? Float32Array.from(vec) : undefined); engine.feedback.storePositive(vec ? Float32Array.from(vec) : undefined);
}, },

View file

@ -23,6 +23,18 @@ import type { EngineId, FeedbackMode, LayerStats } from './types';
import { WasmIML } from './wasm-iml'; import { WasmIML } from './wasm-iml';
import type { IoMigration } from './io-reshape'; import type { IoMigration } from './io-reshape';
export interface GeometricFeedbackConfig {
learningRate: number;
updatesPerSecond: number;
lifetimeMs: number;
}
export const DEFAULT_GEOMETRIC_FEEDBACK_CONFIG: Readonly<GeometricFeedbackConfig> = {
learningRate: 0.001,
updatesPerSecond: 200,
lifetimeMs: 2500,
};
export interface EngineFeedbackApi { export interface EngineFeedbackApi {
/** Positive feedback (thumbs-up). Returns the FeedbackAction int. */ /** Positive feedback (thumbs-up). Returns the FeedbackAction int. */
thumbsUp(): number; thumbsUp(): number;
@ -65,6 +77,10 @@ export interface EngineFeedbackApi {
* FeedbackAction int (14=GeometricPush, 15=GeometricColdStart). * FeedbackAction int (14=GeometricPush, 15=GeometricColdStart).
*/ */
dislikeGeometric(heardVec?: Float32Array, lr?: number): number; dislikeGeometric(heardVec?: Float32Array, lr?: number): number;
/** Configure upstream-style replay dose and wall-clock lifetime. */
setGeometricConfig(config: GeometricFeedbackConfig): void;
/** Advance replay by elapsed wall-clock time; returns optimise cycles run. */
advanceGeometric(dtSeconds: number): number;
/** Feed a positive (like) into the k-NN centroid (null → live MLP output). */ /** Feed a positive (like) into the k-NN centroid (null → live MLP output). */
storePositive(vec?: Float32Array): void; storePositive(vec?: Float32Array): void;
positiveCount(): number; positiveCount(): number;
@ -185,6 +201,13 @@ export class EngineApi {
placedOutput: () => this.iml.feedbackPlacedOutput(), placedOutput: () => this.iml.feedbackPlacedOutput(),
dislikeGeometric: (heardVec?: Float32Array, lr = 0) => dislikeGeometric: (heardVec?: Float32Array, lr = 0) =>
this.iml.feedbackDislikeGeometric(heardVec, lr), this.iml.feedbackDislikeGeometric(heardVec, lr),
setGeometricConfig: (config) =>
this.iml.feedbackSetGeometricConfig(
config.learningRate,
config.updatesPerSecond,
config.lifetimeMs,
),
advanceGeometric: (dtSeconds) => this.iml.feedbackAdvanceGeometric(dtSeconds),
storePositive: (vec?: Float32Array) => this.iml.feedbackStorePositive(vec), storePositive: (vec?: Float32Array) => this.iml.feedbackStorePositive(vec),
positiveCount: () => this.iml.feedbackPositiveCount(), positiveCount: () => this.iml.feedbackPositiveCount(),
negativeCount: () => this.iml.feedbackNegativeCount(), negativeCount: () => this.iml.feedbackNegativeCount(),

View file

@ -5,12 +5,17 @@
* (everything except EngineProvider/useEngine) imports NO React. * (everything except EngineProvider/useEngine) imports NO React.
*/ */
export { EngineApi, createEngine } from './engine-api'; export {
EngineApi,
createEngine,
DEFAULT_GEOMETRIC_FEEDBACK_CONFIG,
} from './engine-api';
export type { export type {
EngineApiOptions, EngineApiOptions,
EngineAudioApi, EngineAudioApi,
EngineFeedbackApi, EngineFeedbackApi,
EngineExploreApi, EngineExploreApi,
GeometricFeedbackConfig,
} from './engine-api'; } from './engine-api';
export { Spine } from './spine'; export { Spine } from './spine';

View file

@ -114,6 +114,13 @@ export interface NispsModule {
// controller default (1e-3). Returns the FeedbackAction int (14=GeometricPush, // controller default (1e-3). Returns the FeedbackAction int (14=GeometricPush,
// 15=GeometricColdStart when no positives exist yet). // 15=GeometricColdStart when no positives exist yet).
_nisps_ml_feedback_dislike_geometric(ml: number, current_out_ptr: number, lr: number): number; _nisps_ml_feedback_dislike_geometric(ml: number, current_out_ptr: number, lr: number): number;
_nisps_ml_feedback_set_geometric_config(
ml: number,
lr: number,
update_hz: number,
lifetime_ms: number,
): void;
_nisps_ml_feedback_advance_geometric(ml: number, dt_seconds: number): number;
// Store a positive (like) into the replay memory so the k-NN centroid sees it. // Store a positive (like) into the replay memory so the k-NN centroid sees it.
// current_out may be null (live output used). Caller still runs addExample+train. // current_out may be null (live output used). Caller still runs addExample+train.
_nisps_ml_feedback_store_positive(ml: number, current_out_ptr: number): void; _nisps_ml_feedback_store_positive(ml: number, current_out_ptr: number): void;

View file

@ -978,6 +978,24 @@ export class WasmIML {
return action; return action;
} }
feedbackSetGeometricConfig(lr: number, updateHz: number, lifetimeMs: number): void {
this.module._nisps_ml_feedback_set_geometric_config(
this.mlHandle,
lr,
updateHz,
lifetimeMs,
);
}
feedbackAdvanceGeometric(dtSeconds: number): number {
const steps = this.module._nisps_ml_feedback_advance_geometric(this.mlHandle, dtSeconds);
if (steps > 0) {
this.sink.emit('ml.delta_update', { reason: 'geometric-replay' });
this.scheduleSave_();
}
return steps;
}
/** /**
* Feed a positive (like) into the replay memory so the k-NN centroid sees it. * Feed a positive (like) into the replay memory so the k-NN centroid sees it.
* `vec` is the heard output at the liked input (null the live MLP output). * `vec` is the heard output at the liked input (null the live MLP output).

View file

@ -1,7 +1,7 @@
/** /**
* FeedbackController framework-neutral learning-engine behaviour for the two * FeedbackController framework-neutral browser driver for the two feedback
* feedback modes plus solo/arm, prototyped in pure TS on the EXISTING engine * modes plus solo/arm. Weight-affecting behaviour lives in the shared C++ core;
* primitives (NO C++/WASM change). * this layer owns UI scheduling and caller-owned example storage.
* *
* Authoritative design: docs/adr/rl-feedback-design.md (Mode 2 default; * Authoritative design: docs/adr/rl-feedback-design.md (Mode 2 default;
* Mode 1 selectable; SOLO default MaskGradients). Engine primitives audited in * Mode 1 selectable; SOLO default MaskGradients). Engine primitives audited in
@ -14,7 +14,7 @@
* setInput(x,y) / getOutputs() synchronous forward inference (the spine) * setInput(x,y) / getOutputs() synchronous forward inference (the spine)
* process() re-run last input after a weight change * process() re-run last input after a weight change
* addExample([x,y], outVec) append a training example * addExample([x,y], outVec) append a training example
* train() SGD over the dataset * train() supervised training over the dataset
* feedback.{setFocus,thumbsUp,dislikeGeometric,} * feedback.{setFocus,thumbsUp,dislikeGeometric,}
* the SHARED C++ core's RL primitives * the SHARED C++ core's RL primitives
* *
@ -27,6 +27,10 @@
*/ */
import type { FeedbackMode } from '../engine/types'; import type { FeedbackMode } from '../engine/types';
import {
DEFAULT_GEOMETRIC_FEEDBACK_CONFIG,
type GeometricFeedbackConfig,
} from '../engine/engine-api';
/** The two product feedback modes (rl-feedback-design §0). */ /** The two product feedback modes (rl-feedback-design §0). */
export type ProtoFeedbackMode = 'explore-and-place' | 'geometric-dislike'; export type ProtoFeedbackMode = 'explore-and-place' | 'geometric-dislike';
@ -67,6 +71,8 @@ export interface ControllerEngine {
// `heardVec` is the post-pipeline (HEARD) output; returns the FeedbackAction // `heardVec` is the post-pipeline (HEARD) output; returns the FeedbackAction
// int (14=GeometricPush, 15=GeometricColdStart). // int (14=GeometricPush, 15=GeometricColdStart).
dislikeGeometric(heardVec?: Float32Array, lr?: number): number; dislikeGeometric(heardVec?: Float32Array, lr?: number): number;
setGeometricConfig(config: GeometricFeedbackConfig): void;
advanceGeometric(dtSeconds: number): number;
positiveCount(): number; positiveCount(): number;
negativeCount(): number; negativeCount(): number;
// ExploreAndPlace lifecycle — the SHARED C++ core (mode 'explore_and_place'). // ExploreAndPlace lifecycle — the SHARED C++ core (mode 'explore_and_place').
@ -112,12 +118,16 @@ export interface FeedbackControllerOptions {
spread?: number; spread?: number;
/** Nudge perturbation standard deviation (small bounded weight jitter). */ /** Nudge perturbation standard deviation (small bounded weight jitter). */
nudgeStddev?: number; nudgeStddev?: number;
geometricConfig?: GeometricFeedbackConfig;
} }
export class FeedbackController { export class FeedbackController {
private engine: ControllerEngine; private engine: ControllerEngine;
private spread: number; private spread: number;
private nudgeStddev: number; private nudgeStddev: number;
private geometricConfig: GeometricFeedbackConfig;
private geometricTimer: ReturnType<typeof setInterval> | null = null;
private geometricLastTickMs = 0;
private mode: ProtoFeedbackMode = 'explore-and-place'; private mode: ProtoFeedbackMode = 'explore-and-place';
private soloMode: ProtoSoloMode = 'mask-gradients'; private soloMode: ProtoSoloMode = 'mask-gradients';
@ -147,6 +157,10 @@ export class FeedbackController {
this.engine = engine; this.engine = engine;
this.spread = opts.spread ?? 0; this.spread = opts.spread ?? 0;
this.nudgeStddev = opts.nudgeStddev ?? 0.05; this.nudgeStddev = opts.nudgeStddev ?? 0.05;
this.geometricConfig = {
...(opts.geometricConfig ?? DEFAULT_GEOMETRIC_FEEDBACK_CONFIG),
};
this.engine.feedback.setGeometricConfig(this.geometricConfig);
} }
// =================================================================== // ===================================================================
@ -158,6 +172,7 @@ export class FeedbackController {
// Switching mode aborts any active scratchpad session. For explore-and-place // Switching mode aborts any active scratchpad session. For explore-and-place
// the SHARED C++ core owns the scratchpad, so delegate the teardown to it. // the SHARED C++ core owns the scratchpad, so delegate the teardown to it.
if (this.exploringFlag) this.cancel(); if (this.exploringFlag) this.cancel();
if (mode !== 'geometric-dislike') this.stopGeometricReplay();
this.mode = mode; this.mode = mode;
// Keep the C++ core's feedback mode in lockstep so the shared explore-and- // Keep the C++ core's feedback mode in lockstep so the shared explore-and-
// place lifecycle is active when this mode is selected. // place lifecycle is active when this mode is selected.
@ -176,11 +191,29 @@ export class FeedbackController {
this.spread = spread; this.spread = spread;
} }
setGeometricConfig(config: GeometricFeedbackConfig): void {
this.geometricConfig = {
learningRate: Math.max(0.000001, config.learningRate),
updatesPerSecond: Math.max(0, config.updatesPerSecond),
lifetimeMs: Math.max(0, config.lifetimeMs),
};
this.engine.feedback.setGeometricConfig(this.geometricConfig);
if (this.geometricTimer) {
this.stopGeometricReplay();
if (this.engine.feedback.negativeCount() > 0) this.startGeometricReplay();
}
}
getGeometricConfig(): Readonly<GeometricFeedbackConfig> {
return this.geometricConfig;
}
/** /**
* An I/O identity edit resets the core's index-aligned scratch/replay state. * An I/O identity edit resets the core's index-aligned scratch/replay state.
* Mirror that reset locally without issuing another core transition. * Mirror that reset locally without issuing another core transition.
*/ */
resetAfterIoChange(): void { resetAfterIoChange(): void {
this.stopGeometricReplay();
this.exploringFlag = false; this.exploringFlag = false;
this.pickingFlag = false; this.pickingFlag = false;
this.anchors = []; this.anchors = [];
@ -351,14 +384,15 @@ export class FeedbackController {
/** /**
* DISLIKE (thumbs-down in Mode 1). Push the current mapping away from the liked * DISLIKE (thumbs-down in Mode 1). Push the current mapping away from the liked
* centroid the SHARED C++ core's geometric push (nisps/ml/geo_push.hpp + * centroid the SHARED C++ core's geometric push (nisps/ml/geo_push.hpp +
* replay.hpp + mlp.train_targets, ported from upstream InterfaceRL 0a541cc). * replay.hpp + mlp.train_targets, ported from upstream InterfaceRL e291192).
* *
* The core uses the MLP's CURRENT input and the passed HEARD output vector: * The core uses the MLP's CURRENT input and the passed HEARD output vector:
* 1. stores the negative (input, a_neg) in the ReplayStore (dedup within 0.05) * 1. stores the negative (input, a_neg) in the ReplayStore (dedup within 0.05)
* 2. k-NN(k=4) centroid of positives near the input * 2. k-NN(k=4) centroid of positives near the input
* 3. target[j] = clamp(a_neg[j] + dir/||dir|| · pushStep/(1+||dir||), 0, 1) * 3. target[j] = clamp(a_neg[j] + dir/||dir|| · pushStep, 0, 1)
* 4. trains toward that target at lr·negLRRatio * 4. trains all live negatives once immediately, then at the configured
* 5. cold-start fallback (negative-LR) when there are no positives yet. * replay rate until each rejection reaches its wall-clock lifetime
* 5. uses a deterministic random direction when there are no positives yet.
* Soloed/active dims come from the core's focus mask (set via setArmMask). * Soloed/active dims come from the core's focus mask (set via setArmMask).
* *
* @param output the HEARD (post-pipeline) output vector a_neg. MUST be the * @param output the HEARD (post-pipeline) output vector a_neg. MUST be the
@ -369,9 +403,44 @@ export class FeedbackController {
dislike(output: Float32Array): number { dislike(output: Float32Array): number {
const action = this.engine.feedback.dislikeGeometric(output); const action = this.engine.feedback.dislikeGeometric(output);
this.engine.process(); this.engine.process();
this.startGeometricReplay();
return action; return action;
} }
private startGeometricReplay(): void {
this.stopGeometricReplay();
if (
this.mode !== 'geometric-dislike' ||
this.geometricConfig.updatesPerSecond <= 0 ||
this.geometricConfig.lifetimeMs <= 0 ||
this.engine.feedback.negativeCount() <= 0
) {
return;
}
this.geometricLastTickMs = Date.now();
const intervalMs = Math.max(
5,
Math.min(50, 1000 / this.geometricConfig.updatesPerSecond),
);
this.geometricTimer = setInterval(() => {
const now = Date.now();
const dtSeconds = Math.max(0, (now - this.geometricLastTickMs) / 1000);
this.geometricLastTickMs = now;
const steps = this.engine.feedback.advanceGeometric(dtSeconds);
if (steps > 0) this.engine.process();
if (this.engine.feedback.negativeCount() <= 0) this.stopGeometricReplay();
}, intervalMs);
}
private stopGeometricReplay(): void {
if (this.geometricTimer !== null) clearInterval(this.geometricTimer);
this.geometricTimer = null;
}
dispose(): void {
this.stopGeometricReplay();
}
/** /**
* LIKE + train (thumbs-up in Mode 1). Feeds the positive into the C++ core's * LIKE + train (thumbs-up in Mode 1). Feeds the positive into the C++ core's
* k-NN centroid (via the core's thumbsUp auto-store, ADR §2.1) AND stores the * k-NN centroid (via the core's thumbsUp auto-store, ADR §2.1) AND stores the

View file

@ -70,4 +70,36 @@ test.describe('geometric dislike (Mode 1) — core-backed', () => {
expect(countChanged(before, after, 1e-4)).toBeGreaterThan(0); expect(countChanged(before, after, 1e-4)).toBeGreaterThan(0);
expect(allWithin(after, 0, 1)).toBe(true); expect(allWithin(after, 0, 1)).toBe(true);
}); });
test('parameterised replay applies configured dose then expires by wall time', async ({ page }) => {
const result = await page.evaluate((heard) => {
const p = window.__nisps!;
p.setFeedbackMode('avoid');
p.setAvoidStyle(0);
p.setGeometricConfig(0.001, 20, 100);
p.setInputs(0.4, 0.6);
p.dislikeGeometric(heard);
const first = p.advanceGeometric(0.05);
const live = p.feedbackCounts().negative;
const second = p.advanceGeometric(0.05);
const expired = p.feedbackCounts().negative;
return { first, live, second, expired };
}, HEARD);
expect(result).toEqual({ first: 1, live: 1, second: 1, expired: 0 });
});
test('expanded Learning panel exposes the upstream-default experiment controls', async ({ page }) => {
await page.getByTitle('Learning', { exact: true }).click();
await page.getByTitle('Expand', { exact: true }).click();
await expect(page.getByText('push · learning rate', { exact: true })).toBeVisible();
await expect(page.getByText('push · updates / second', { exact: true })).toBeVisible();
await expect(page.getByText('push · lifetime', { exact: true })).toBeVisible();
await expect(page.getByText('0.0010', { exact: true })).toBeVisible();
await expect(page.getByText('200 Hz', { exact: true })).toBeVisible();
await expect(page.getByText('2.5 s', { exact: true })).toBeVisible();
await expect(page.getByRole('button', { name: 'Upstream defaults' })).toBeVisible();
await expect(page.getByText('≈ 500 replay updates + the press')).toBeVisible();
});
}); });

View file

@ -249,6 +249,7 @@ class DynamicFeedbackStorage {
+ replay_cap * n_in // replay inputs + replay_cap * n_in // replay inputs
+ replay_cap * n_out // replay actions + replay_cap * n_out // replay actions
+ replay_cap // replay rewards + replay_cap // replay rewards
+ replay_cap // replay ages (ms)
+ n_out * 2u // centroid + target + n_out * 2u // centroid + target
+ focus_floats; + focus_floats;
arena_ = new (std::nothrow) float[total](); arena_ = new (std::nothrow) float[total]();
@ -260,7 +261,8 @@ class DynamicFeedbackStorage {
off_replay_in_ = off_undo_ + n_weights_ * undo_cap_; off_replay_in_ = off_undo_ + n_weights_ * undo_cap_;
off_replay_a_ = off_replay_in_ + replay_cap_ * n_in_; off_replay_a_ = off_replay_in_ + replay_cap_ * n_in_;
off_replay_r_ = off_replay_a_ + replay_cap_ * n_out_; off_replay_r_ = off_replay_a_ + replay_cap_ * n_out_;
off_centroid_ = off_replay_r_ + replay_cap_; off_replay_age_ = off_replay_r_ + replay_cap_;
off_centroid_ = off_replay_age_ + replay_cap_;
off_target_ = off_centroid_ + n_out_; off_target_ = off_centroid_ + n_out_;
off_focus_ = off_target_ + n_out_; off_focus_ = off_target_ + n_out_;
} }
@ -302,6 +304,7 @@ class DynamicFeedbackStorage {
std::span<float> replay_inputs() noexcept { return {arena_ + off_replay_in_, replay_cap_ * n_in_}; } std::span<float> replay_inputs() noexcept { return {arena_ + off_replay_in_, replay_cap_ * n_in_}; }
std::span<float> replay_actions() noexcept { return {arena_ + off_replay_a_, replay_cap_ * n_out_}; } std::span<float> replay_actions() noexcept { return {arena_ + off_replay_a_, replay_cap_ * n_out_}; }
std::span<float> replay_rewards() noexcept { return {arena_ + off_replay_r_, replay_cap_}; } std::span<float> replay_rewards() noexcept { return {arena_ + off_replay_r_, replay_cap_}; }
std::span<float> replay_ages_ms() noexcept { return {arena_ + off_replay_age_, replay_cap_}; }
std::span<float> centroid_buf() noexcept { return {arena_ + off_centroid_, n_out_}; } std::span<float> centroid_buf() noexcept { return {arena_ + off_centroid_, n_out_}; }
std::span<float> target_buf() noexcept { return {arena_ + off_target_, n_out_}; } std::span<float> target_buf() noexcept { return {arena_ + off_target_, n_out_}; }
std::span<std::uint8_t> focus() noexcept { std::span<std::uint8_t> focus() noexcept {
@ -318,7 +321,8 @@ class DynamicFeedbackStorage {
off_placed_ = o.off_placed_; off_snap_ = o.off_snap_; off_placed_ = o.off_placed_; off_snap_ = o.off_snap_;
off_scratch_ = o.off_scratch_; off_undo_ = o.off_undo_; off_focus_ = o.off_focus_; off_scratch_ = o.off_scratch_; off_undo_ = o.off_undo_; off_focus_ = o.off_focus_;
off_replay_in_ = o.off_replay_in_; off_replay_a_ = o.off_replay_a_; off_replay_in_ = o.off_replay_in_; off_replay_a_ = o.off_replay_a_;
off_replay_r_ = o.off_replay_r_; off_centroid_ = o.off_centroid_; off_replay_r_ = o.off_replay_r_; off_replay_age_ = o.off_replay_age_;
off_centroid_ = o.off_centroid_;
off_target_ = o.off_target_; off_target_ = o.off_target_;
arena_ = o.arena_; arena_ = o.arena_;
o.arena_ = nullptr; o.arena_ = nullptr;
@ -328,7 +332,7 @@ class DynamicFeedbackStorage {
std::size_t off_placed_ = 0u, off_snap_ = 0u, off_scratch_ = 0u, std::size_t off_placed_ = 0u, off_snap_ = 0u, off_scratch_ = 0u,
off_undo_ = 0u, off_focus_ = 0u, off_replay_in_ = 0u, off_undo_ = 0u, off_focus_ = 0u, off_replay_in_ = 0u,
off_replay_a_ = 0u, off_replay_r_ = 0u, off_centroid_ = 0u, off_replay_a_ = 0u, off_replay_r_ = 0u, off_centroid_ = 0u,
off_target_ = 0u; off_replay_age_ = 0u, off_target_ = 0u;
float* arena_ = nullptr; float* arena_ = nullptr;
}; };

View file

@ -18,9 +18,9 @@
// - Geometric (DEFAULT) — the ported firmware k-NN // - Geometric (DEFAULT) — the ported firmware k-NN
// centroid push-away, backed by the controller's // centroid push-away, backed by the controller's
// own ReplayMemory (see dislike_geometric() below). // own ReplayMemory (see dislike_geometric() below).
// Cold-starts with a negative-LR fallback until the // Replays all live negatives for a parameterised
// first positive is stored. The old "geometric push // wall-clock window; cold start uses a deterministic
// not ported" note is stale — it IS ported. // random direction until a positive is stored.
// - Diffuse — the pre-P3 undirected MLP::move_weights // - Diffuse — the pre-P3 undirected MLP::move_weights
// perturb. Deliberately-retained research reserve, // perturb. Deliberately-retained research reserve,
// not the product default (see its definition below // not the product default (see its definition below
@ -138,7 +138,7 @@ enum class FeedbackAction : std::uint8_t {
CancelPlace = 13, // Placing→Exploring; backed out of placing (no store). CancelPlace = 13, // Placing→Exploring; backed out of placing (no store).
// ---- Geometric dislike (append-only) ---- // ---- Geometric dislike (append-only) ----
GeometricPush = 14, // dislike trained toward the computed push-away target. GeometricPush = 14, // dislike trained toward the computed push-away target.
GeometricColdStart = 15, // no positives yet: negative-LR fallback ran; UI shows the cold-start prompt. GeometricColdStart = 15, // no positives yet: random-direction push ran; UI shows the cold-start prompt.
}; };
// --------------------------------------------------------------------------- // ---------------------------------------------------------------------------
@ -180,6 +180,7 @@ class FixedFeedbackStorage {
NISPS_FORCE_INLINE std::span<float> replay_inputs() noexcept { return replay_in_; } NISPS_FORCE_INLINE std::span<float> replay_inputs() noexcept { return replay_in_; }
NISPS_FORCE_INLINE std::span<float> replay_actions() noexcept { return replay_act_; } NISPS_FORCE_INLINE std::span<float> replay_actions() noexcept { return replay_act_; }
NISPS_FORCE_INLINE std::span<float> replay_rewards() noexcept { return replay_rew_; } NISPS_FORCE_INLINE std::span<float> replay_rewards() noexcept { return replay_rew_; }
NISPS_FORCE_INLINE std::span<float> replay_ages_ms() noexcept { return replay_age_ms_; }
// Centroid + push-target scratch (n_out each). // Centroid + push-target scratch (n_out each).
NISPS_FORCE_INLINE std::span<float> centroid_buf() noexcept { return centroid_; } NISPS_FORCE_INLINE std::span<float> centroid_buf() noexcept { return centroid_; }
NISPS_FORCE_INLINE std::span<float> target_buf() noexcept { return target_; } NISPS_FORCE_INLINE std::span<float> target_buf() noexcept { return target_; }
@ -194,6 +195,7 @@ class FixedFeedbackStorage {
std::array<float, ReplayCap * NIn> replay_in_{}; std::array<float, ReplayCap * NIn> replay_in_{};
std::array<float, ReplayCap * NOut> replay_act_{}; std::array<float, ReplayCap * NOut> replay_act_{};
std::array<float, ReplayCap> replay_rew_{}; std::array<float, ReplayCap> replay_rew_{};
std::array<float, ReplayCap> replay_age_ms_{};
std::array<float, NOut> centroid_{}; std::array<float, NOut> centroid_{};
std::array<float, NOut> target_{}; std::array<float, NOut> target_{};
}; };
@ -230,6 +232,17 @@ class FeedbackControllerCore : public FbStorage {
// InterfaceRL default 1e-3, pre-scaling). // InterfaceRL default 1e-3, pre-scaling).
void set_geo_lr(float lr) noexcept { geo_lr_ = lr; } void set_geo_lr(float lr) noexcept { geo_lr_ = lr; }
float geo_lr() const noexcept { return geo_lr_; } float geo_lr() const noexcept { return geo_lr_; }
void set_geo_update_hz(float hz) noexcept {
geo_update_hz_ = (hz > 0.f) ? hz : 0.f;
geo_step_accum_ = 0.f;
if (!(geo_update_hz_ > 0.f)) replay_().remove_all_negatives();
}
float geo_update_hz() const noexcept { return geo_update_hz_; }
void set_geo_lifetime_ms(float ms) noexcept {
geo_lifetime_ms_ = (ms > 0.f) ? ms : 0.f;
if (!(geo_lifetime_ms_ > 0.f)) replay_().remove_all_negatives();
}
float geo_lifetime_ms() const noexcept { return geo_lifetime_ms_; }
// `exploring()` is true whenever a scratchpad net is live and learning is // `exploring()` is true whenever a scratchpad net is live and learning is
// paused — for the legacy RANDOMISE_* modes, AND for ExploreAndPlace in // paused — for the legacy RANDOMISE_* modes, AND for ExploreAndPlace in
@ -393,9 +406,9 @@ class FeedbackControllerCore : public FbStorage {
void seed(std::uint64_t s) noexcept { rng_.seed(s); } void seed(std::uint64_t s) noexcept { rng_.seed(s); }
// ========================================================================= // =========================================================================
// Geometric dislike (rl-feedback-design §2.1) — the press-time half and // Geometric dislike (rl-feedback-design §2.1): press stores the rejection
// the async optimise() half of upstream InterfaceRL collapsed into ONE // and performs one immediate update; advance_geometric() supplies the
// synchronous call (nisps has no background optimise driver). // repeated, wall-clock-bounded optimise dose used by current upstream.
// ========================================================================= // =========================================================================
std::size_t replay_size() const noexcept { return replay_count_; } std::size_t replay_size() const noexcept { return replay_count_; }
@ -416,18 +429,12 @@ class FeedbackControllerCore : public FbStorage {
// Thumbs-down at the MLP's CURRENT input with heard action `current_out` // Thumbs-down at the MLP's CURRENT input with heard action `current_out`
// (empty ⇒ the MLP's live outputs). Runs the full upstream sequence: // (empty ⇒ the MLP's live outputs). Runs the full upstream sequence:
// 1. deepen-or-store the negative (dedup radius 0.05). // 1. deepen-or-store the negative (dedup radius 0.05).
// 2. k-NN(4) positive centroid — or zeros when nothing is liked yet — // 2. immediately optimise all live negatives once. Subsequent updates
// → push-away target → train toward it at lr * negLRRatio, gated by // come from advance_geometric().
// the focus/solo mask. With no positives the push direction is
// random per dim, which is upstream's cold start (there is no
// separate negative-LR branch any more; see the body).
// 3. proportional decay + eviction of expired negatives.
template <typename M> template <typename M>
FeedbackAction dislike_geometric(M& mlp, std::span<const float> current_out, FeedbackAction dislike_geometric(M& mlp, std::span<const float> current_out,
float lr) noexcept { float lr) noexcept {
auto replay = replay_(); auto replay = replay_();
const std::size_t n_out = this->n_out();
std::span<const float> a_neg = current_out.empty() std::span<const float> a_neg = current_out.empty()
? std::span<const float>(mlp.outputs()) ? std::span<const float>(mlp.outputs())
: current_out; : current_out;
@ -436,44 +443,46 @@ class FeedbackControllerCore : public FbStorage {
// 1. store/deepen the negative (InterfaceRL.cpp:42-66). // 1. store/deepen the negative (InterfaceRL.cpp:42-66).
replay.deepen_or_store_negative(x_neg, a_neg); replay.deepen_or_store_negative(x_neg, a_neg);
const std::size_t pos_total = replay.positive_count(); const bool have_positives = replay.positive_count() > 0u;
const std::size_t neg_total = replay.negative_count(); (void)optimise_geometric_once_(mlp, lr);
const float avg_neg = replay.avg_negative_reward(); if (!(geo_update_hz_ > 0.f) || !(geo_lifetime_ms_ > 0.f)) {
replay.remove_all_negatives();
// 2+3. ONE path, as upstream (InterfaceRL.tpp:696-761). With no likes
// stored there is no centroid to push away from, so the centroid is
// zeros and `have_positives=false` sends every dim in a random
// direction — upstream's `useRandom = !havePositives || ...`. We still
// report GeometricColdStart so the caller can show its "like a few
// sounds first" prompt, but the TRAINING is the same push at the same
// LR either way. (Until 2026-07-25 the cold start instead trained AWAY
// from the heard action at a NEGATIVE lr — a port of the older
// `0a541cc` fallback that upstream has since replaced.)
auto mean = this->centroid_buf();
const bool have_positives = (pos_total > 0u);
if (have_positives) {
(void)replay.knn_positive_centroid(x_neg, kCentroidK, mean);
} else {
for (std::size_t j = 0; j < n_out; ++j) mean[j] = 0.f;
} }
return have_positives
auto target = this->target_buf();
compute_push_target(a_neg.subspan(0, (a_neg.size() < n_out) ? a_neg.size() : n_out),
std::span<const float>(mean.data(), n_out),
focus_span_(), geo_push_step(avg_neg), have_positives,
rng_, target);
const float ratio = geo_neg_lr_ratio(neg_total, pos_total);
mlp.train_targets(x_neg, std::span<const float>(target.data(), n_out),
lr * ratio, focus_span_());
const FeedbackAction action = have_positives
? FeedbackAction::GeometricPush ? FeedbackAction::GeometricPush
: FeedbackAction::GeometricColdStart; : FeedbackAction::GeometricColdStart;
}
// 4. decay + evict expired negatives (InterfaceRL.cpp:752-760). // Advance the upstream-style replay optimiser by elapsed wall-clock time.
replay.decay_negatives(); // Returns the number of optimisation cycles applied. A long scheduler gap
// does not create an unbounded catch-up burst: upstream also cannot execute
// missed loop iterations while blocked. Ages still advance by the full dt.
template <typename M>
std::size_t advance_geometric(M& mlp, float dt_seconds) noexcept {
if (mode_ != FeedbackMode::Avoid || avoid_style_ != AvoidStyle::Geometric ||
!(dt_seconds > 0.f)) {
return 0u;
}
auto replay = replay_();
if (replay.negative_count() == 0u) {
geo_step_accum_ = 0.f;
return 0u;
}
return action; constexpr float kMaxDoseDtSeconds = 0.1f;
const float dose_dt = (dt_seconds < kMaxDoseDtSeconds)
? dt_seconds
: kMaxDoseDtSeconds;
geo_step_accum_ += dose_dt * geo_update_hz_;
std::size_t steps = static_cast<std::size_t>(geo_step_accum_);
geo_step_accum_ -= static_cast<float>(steps);
for (std::size_t i = 0; i < steps; ++i) {
if (!optimise_geometric_once_(mlp, geo_lr_)) break;
}
replay.advance_negative_ages(dt_seconds * 1000.f, geo_lifetime_ms_);
if (replay.negative_count() == 0u) geo_step_accum_ = 0.f;
return steps;
} }
// ========================================================================= // =========================================================================
@ -661,8 +670,43 @@ class FeedbackControllerCore : public FbStorage {
// The replay view over the storage-owned buffers. // The replay view over the storage-owned buffers.
ReplayView replay_() noexcept { ReplayView replay_() noexcept {
return ReplayView(this->replay_inputs(), this->replay_actions(), return ReplayView(this->replay_inputs(), this->replay_actions(),
this->replay_rewards(), this->n_in(), this->n_out(), this->replay_rewards(), this->replay_ages_ms(),
this->replay_cap(), replay_count_); this->n_in(), this->n_out(), this->replay_cap(),
replay_count_);
}
// One upstream-style optimise cycle over ALL live negatives. Unlike the
// upstream cursor-coupled implementation, the positive centroid is looked
// up at each negative's own stored input, so moving the cursor cannot
// reinterpret an older rejection.
template <typename M>
bool optimise_geometric_once_(M& mlp, float lr) noexcept {
auto replay = replay_();
const std::size_t pos_total = replay.positive_count();
const std::size_t neg_total = replay.negative_count();
if (neg_total == 0u) return false;
const float avg_neg = replay.avg_negative_reward();
const std::size_t n_out = this->n_out();
const float ratio = geo_neg_lr_ratio(neg_total, pos_total);
for (std::size_t i = 0; i < replay.size(); ++i) {
if (replay.reward(i) > 0.f) continue;
const auto x_neg = replay.input(i);
const auto a_neg = replay.action(i);
auto mean = this->centroid_buf();
const bool have_positives =
replay.knn_positive_centroid(x_neg, kCentroidK, mean) > 0u;
if (!have_positives) {
for (std::size_t j = 0; j < n_out; ++j) mean[j] = 0.f;
}
auto target = this->target_buf();
compute_push_target(a_neg, std::span<const float>(mean.data(), n_out),
focus_span_(), geo_push_step(avg_neg),
have_positives, rng_, target);
mlp.train_targets(x_neg, std::span<const float>(target.data(), n_out),
lr * ratio, focus_span_());
}
return true;
} }
// The focus mask as the geometric active-dims gate (empty ⇒ all active). // The focus mask as the geometric active-dims gate (empty ⇒ all active).
@ -795,6 +839,9 @@ class FeedbackControllerCore : public FbStorage {
// ---- Geometric dislike state --------------------------------------------- // ---- Geometric dislike state ---------------------------------------------
std::size_t replay_count_ = 0u; std::size_t replay_count_ = 0u;
float geo_lr_ = 0.001f; // upstream InterfaceRL.hpp:312 float geo_lr_ = 0.001f; // upstream InterfaceRL.hpp:312
float geo_update_hz_ = 200.f; // upstream default optimise loop
float geo_lifetime_ms_ = 2500.f; // upstream kDislikeLifetimeMs
float geo_step_accum_ = 0.f;
// ---- ExploreAndPlace state ---------------------------------------------- // ---- ExploreAndPlace state ----------------------------------------------
ExploreState ep_state_ = ExploreState::Idle; ExploreState ep_state_ = ExploreState::Idle;

View file

@ -5,7 +5,7 @@
// * `_perform_dislike_action()` (InterfaceRL.cpp:42-66) — nearby-negative // * `_perform_dislike_action()` (InterfaceRL.cpp:42-66) — nearby-negative
// deepening within Euclidean 0.05, else store reward=-1. // deepening within Euclidean 0.05, else store reward=-1.
// * `optimise()` k-NN positive centroid (InterfaceRL.cpp:602-627). // * `optimise()` k-NN positive centroid (InterfaceRL.cpp:602-627).
// * proportional negative decay + eviction (InterfaceRL.cpp:664, :752-760). // * wall-clock negative lifetime + eviction (current upstream e291192).
// //
// STORAGE: the buffers live in the feedback controller's storage policy // STORAGE: the buffers live in the feedback controller's storage policy
// (nisps/ml/feedback.hpp — fixed std::array on firmware, arena slice in the // (nisps/ml/feedback.hpp — fixed std::array on firmware, arena slice in the
@ -29,22 +29,21 @@
namespace nisps::ml { namespace nisps::ml {
// Upstream constants (InterfaceRL.hpp:293-296, .cpp:42-66,664). // Upstream constants (InterfaceRL.hpp / InterfaceRL.tpp @ e291192).
inline constexpr float kReplayDedupRadius = 0.05f; inline constexpr float kReplayDedupRadius = 0.05f;
inline constexpr float kReplayDecayStep = 0.0025f;
inline constexpr float kReplayEvictThreshold = -0.01f;
inline constexpr float kMaxDislikeMagnitude = 16.f; inline constexpr float kMaxDislikeMagnitude = 16.f;
inline constexpr std::size_t kCentroidK = 4u; inline constexpr std::size_t kCentroidK = 4u;
// A non-owning view over the replay buffers (inputs: cap×n_in, actions: // A non-owning view over the replay buffers (inputs: cap×n_in, actions:
// cap×n_out, rewards: cap) plus the live count. All methods deterministic, // cap×n_out, rewards/ages: cap) plus the live count. All methods deterministic,
// allocation-free. // allocation-free.
class ReplayView { class ReplayView {
public: public:
ReplayView(std::span<float> inputs, std::span<float> actions, ReplayView(std::span<float> inputs, std::span<float> actions,
std::span<float> rewards, std::size_t n_in, std::size_t n_out, std::span<float> rewards, std::span<float> ages_ms,
std::size_t cap, std::size_t& count) noexcept std::size_t n_in, std::size_t n_out, std::size_t cap,
: inputs_(inputs), actions_(actions), rewards_(rewards), std::size_t& count) noexcept
: inputs_(inputs), actions_(actions), rewards_(rewards), ages_ms_(ages_ms),
n_in_(n_in), n_out_(n_out), cap_(cap), count_(count) {} n_in_(n_in), n_out_(n_out), cap_(cap), count_(count) {}
std::size_t size() const noexcept { return count_; } std::size_t size() const noexcept { return count_; }
@ -57,6 +56,7 @@ class ReplayView {
return actions_.subspan(i * n_out_, n_out_); return actions_.subspan(i * n_out_, n_out_);
} }
float reward(std::size_t i) const noexcept { return rewards_[i]; } float reward(std::size_t i) const noexcept { return rewards_[i]; }
float age_ms(std::size_t i) const noexcept { return ages_ms_[i]; }
std::size_t positive_count() const noexcept { std::size_t positive_count() const noexcept {
std::size_t n = 0u; std::size_t n = 0u;
@ -113,6 +113,7 @@ class ReplayView {
auto act = actions_.subspan(i * n_out_, n_out_); auto act = actions_.subspan(i * n_out_, n_out_);
const std::size_t n = (a.size() < n_out_) ? a.size() : n_out_; const std::size_t n = (a.size() < n_out_) ? a.size() : n_out_;
for (std::size_t j = 0; j < n; ++j) act[j] = a[j]; for (std::size_t j = 0; j < n; ++j) act[j] = a[j];
ages_ms_[i] = 0.f;
return true; return true;
} }
} }
@ -162,21 +163,21 @@ class ReplayView {
return used; return used;
} }
// Proportional decay of every negative (`reward += 0.0025 * max(|r|, 1)`) // Advance wall-clock age for every negative and remove those which have
// and in-place eviction of items decayed past -0.01. Returns the number // lived their configured full-strength window. Positives do not expire.
// evicted (the caller halves its dislike multiplier per expiry, matching // Current upstream uses a timestamp and kDislikeLifetimeMs=2500; explicit
// upstream InterfaceRL.cpp:752-760). // elapsed time keeps the core deterministic on firmware, native, and WASM.
std::size_t decay_negatives() noexcept { std::size_t advance_negative_ages(float dt_ms, float lifetime_ms) noexcept {
if (!(dt_ms > 0.f) || !(lifetime_ms > 0.f)) return 0u;
std::size_t evicted = 0u; std::size_t evicted = 0u;
std::size_t i = 0u; std::size_t i = 0u;
while (i < count_) { while (i < count_) {
if (rewards_[i] <= 0.f) { if (rewards_[i] <= 0.f) {
const float mag = (rewards_[i] < 0.f) ? -rewards_[i] : rewards_[i]; ages_ms_[i] += dt_ms;
rewards_[i] += kReplayDecayStep * ((mag > 1.f) ? mag : 1.f); if (ages_ms_[i] >= lifetime_ms) {
if (rewards_[i] > kReplayEvictThreshold) {
evict_(i); evict_(i);
++evicted; ++evicted;
continue; // same index now holds the next item continue;
} }
} }
++i; ++i;
@ -184,6 +185,17 @@ class ReplayView {
return evicted; return evicted;
} }
void remove_all_negatives() noexcept {
std::size_t i = 0u;
while (i < count_) {
if (rewards_[i] <= 0.f) {
evict_(i);
continue;
}
++i;
}
}
void clear() noexcept { count_ = 0u; } void clear() noexcept { count_ = 0u; }
private: private:
@ -207,6 +219,7 @@ class ReplayView {
for (std::size_t j = 0; j < n_in_; ++j) in[j] = (j < nx) ? x[j] : 0.f; for (std::size_t j = 0; j < n_in_; ++j) in[j] = (j < nx) ? x[j] : 0.f;
for (std::size_t j = 0; j < n_out_; ++j) act[j] = (j < na) ? a[j] : 0.f; for (std::size_t j = 0; j < n_out_; ++j) act[j] = (j < na) ? a[j] : 0.f;
rewards_[slot] = reward; rewards_[slot] = reward;
ages_ms_[slot] = 0.f;
} }
// Remove item i, shifting everything after it down one slot. // Remove item i, shifting everything after it down one slot.
@ -219,6 +232,7 @@ class ReplayView {
auto src_act = actions_.subspan(m * n_out_, n_out_); auto src_act = actions_.subspan(m * n_out_, n_out_);
for (std::size_t j = 0; j < n_out_; ++j) dst_act[j] = src_act[j]; for (std::size_t j = 0; j < n_out_; ++j) dst_act[j] = src_act[j];
rewards_[m - 1u] = rewards_[m]; rewards_[m - 1u] = rewards_[m];
ages_ms_[m - 1u] = ages_ms_[m];
} }
--count_; --count_;
} }
@ -226,6 +240,7 @@ class ReplayView {
std::span<float> inputs_; std::span<float> inputs_;
std::span<float> actions_; std::span<float> actions_;
std::span<float> rewards_; std::span<float> rewards_;
std::span<float> ages_ms_;
std::size_t n_in_; std::size_t n_in_;
std::size_t n_out_; std::size_t n_out_;
std::size_t cap_; std::size_t cap_;

View file

@ -403,6 +403,9 @@ int nisps_ml_reshape(void* ml, int input_size, int output_size,
const auto feedback_mode = h->feedback.mode(); const auto feedback_mode = h->feedback.mode();
const auto avoid_style = h->feedback.avoid_style(); const auto avoid_style = h->feedback.avoid_style();
const float geo_lr = h->feedback.geo_lr();
const float geo_update_hz = h->feedback.geo_update_hz();
const float geo_lifetime_ms = h->feedback.geo_lifetime_ms();
BrowserMLP fresh(h->seed64, d.n_in, std::span<const std::size_t>(d.hidden, 3u), d.n_out); BrowserMLP fresh(h->seed64, d.n_in, std::span<const std::size_t>(d.hidden, 3u), d.n_out);
if (!fresh.valid()) return 0; if (!fresh.valid()) return 0;
fresh.draw_weights(spread); fresh.draw_weights(spread);
@ -412,6 +415,9 @@ int nisps_ml_reshape(void* ml, int input_size, int output_size,
kFeedbackUndoDepth, d.n_in, kFeedbackReplayCap); kFeedbackUndoDepth, d.n_in, kFeedbackReplayCap);
if (!fb.valid()) return 0; if (!fb.valid()) return 0;
fb.set_avoid_style(avoid_style); fb.set_avoid_style(avoid_style);
fb.set_geo_lr(geo_lr);
fb.set_geo_update_hz(geo_update_hz);
fb.set_geo_lifetime_ms(geo_lifetime_ms);
fb.set_mode(feedback_mode, fresh); fb.set_mode(feedback_mode, fresh);
h->mlp = static_cast<BrowserMLP&&>(fresh); h->mlp = static_cast<BrowserMLP&&>(fresh);
@ -627,10 +633,16 @@ void nisps_ml_feedback_reset(void* ml) {
auto* h = static_cast<MLHandle*>(ml); auto* h = static_cast<MLHandle*>(ml);
const auto feedback_mode = h->feedback.mode(); const auto feedback_mode = h->feedback.mode();
const auto avoid_style = h->feedback.avoid_style(); const auto avoid_style = h->feedback.avoid_style();
const float geo_lr = h->feedback.geo_lr();
const float geo_update_hz = h->feedback.geo_update_hz();
const float geo_lifetime_ms = h->feedback.geo_lifetime_ms();
BrowserFeedback fb(h->seed64 ^ kFeedbackSalt, h->n_out(), h->mlp.weight_count(), BrowserFeedback fb(h->seed64 ^ kFeedbackSalt, h->n_out(), h->mlp.weight_count(),
kFeedbackUndoDepth, h->n_in(), kFeedbackReplayCap); kFeedbackUndoDepth, h->n_in(), kFeedbackReplayCap);
if (!fb.valid()) return; if (!fb.valid()) return;
fb.set_avoid_style(avoid_style); fb.set_avoid_style(avoid_style);
fb.set_geo_lr(geo_lr);
fb.set_geo_update_hz(geo_update_hz);
fb.set_geo_lifetime_ms(geo_lifetime_ms);
fb.set_mode(feedback_mode, h->mlp); fb.set_mode(feedback_mode, h->mlp);
h->feedback = static_cast<BrowserFeedback&&>(fb); h->feedback = static_cast<BrowserFeedback&&>(fb);
h->feedback_static_scratch.assign(h->n_out(), 0.f); h->feedback_static_scratch.assign(h->n_out(), 0.f);
@ -817,6 +829,29 @@ int nisps_ml_feedback_dislike_geometric(void* ml, const float* current_out, floa
return static_cast<int>(h->feedback.dislike_geometric(h->mlp, out, use_lr)); return static_cast<int>(h->feedback.dislike_geometric(h->mlp, out, use_lr));
} }
// Configure the replay dose. Non-negative values are accepted; zero update
// rate or lifetime gives one-shot press behaviour. Defaults are upstream:
// 1e-3, 200 Hz, 2500 ms.
EMSCRIPTEN_KEEPALIVE
void nisps_ml_feedback_set_geometric_config(void* ml, float lr,
float update_hz,
float lifetime_ms) {
if (!ml) return;
auto* h = static_cast<MLHandle*>(ml);
if (lr > 0.f) h->feedback.set_geo_lr(lr);
h->feedback.set_geo_update_hz(update_hz);
h->feedback.set_geo_lifetime_ms(lifetime_ms);
}
// Deterministic elapsed-time seam. The host owns the clock; every weight-
// affecting step remains in the shared C++ core. Returns optimise cycles run.
EMSCRIPTEN_KEEPALIVE
int nisps_ml_feedback_advance_geometric(void* ml, float dt_seconds) {
if (!ml) return 0;
auto* h = static_cast<MLHandle*>(ml);
return static_cast<int>(h->feedback.advance_geometric(h->mlp, dt_seconds));
}
// Store a positive (like) into the replay memory so the k-NN centroid sees // Store a positive (like) into the replay memory so the k-NN centroid sees
// it. current_out may be null (live output used). The caller still runs its // it. current_out may be null (live output used). The caller still runs its
// usual addExample + train. // usual addExample + train.

View file

@ -56,6 +56,7 @@ EXPORTED_FUNCS='[
"_nisps_ml_feedback_undo_depth", "_nisps_ml_feedback_undo_depth",
"_nisps_ml_feedback_placed_output", "_nisps_ml_feedback_placed_output",
"_nisps_ml_feedback_dislike_geometric","_nisps_ml_feedback_store_positive", "_nisps_ml_feedback_dislike_geometric","_nisps_ml_feedback_store_positive",
"_nisps_ml_feedback_set_geometric_config","_nisps_ml_feedback_advance_geometric",
"_nisps_ml_feedback_positive_count","_nisps_ml_feedback_negative_count", "_nisps_ml_feedback_positive_count","_nisps_ml_feedback_negative_count",
"_nisps_ml_feedback_set_avoid_style", "_nisps_ml_feedback_set_avoid_style",
"_nisps_ml_jolt_press","_nisps_ml_jolt_step","_nisps_ml_jolt_release", "_nisps_ml_jolt_press","_nisps_ml_jolt_step","_nisps_ml_jolt_release",

View file

@ -281,10 +281,10 @@ int main(int argc, char** argv) {
// ---- Stage 6: geometric dislike (one-core-engine P3) ---- // ---- Stage 6: geometric dislike (one-core-engine P3) ----
// Scripted feedback session — likes at two corners feed the replay // Scripted feedback session — likes at two corners feed the replay
// positives (via the Avoid+Geometric on_up path), then two dislikes at // positives (via the Avoid+Geometric on_up path), then two dislikes at
// a probed input: the first stores the negative and trains toward the // a probed input. Eight 5ms driver ticks cover the parameterised
// computed push-away target; the second deepens and pushes again. The // upstream-style 200 Hz replay seam over all live negatives. The weight
// weight trajectory must match native↔WASM within 1e-5 (the useRandom // trajectory must match native↔WASM within 1e-5 (the useRandom branch
// branch never fires here; the controller Rng is untouched). // never fires here; the controller Rng is untouched).
fb.set_mode(nisps::ml::FeedbackMode::Avoid, mlp); fb.set_mode(nisps::ml::FeedbackMode::Avoid, mlp);
auto like_at = [&](float x, float y) { auto like_at = [&](float x, float y) {
@ -316,6 +316,9 @@ int main(int argc, char** argv) {
}; };
dislike_at(0.25f, 0.75f); dislike_at(0.25f, 0.75f);
dislike_at(0.26f, 0.74f); // within dedup radius → deepen + push dislike_at(0.26f, 0.74f); // within dedup radius → deepen + push
for (int i = 0; i < 8; ++i) {
fb.advance_geometric(mlp, 0.005f);
}
payload.push_back(static_cast<float>(fb.positive_count())); payload.push_back(static_cast<float>(fb.positive_count()));
payload.push_back(static_cast<float>(fb.negative_count())); payload.push_back(static_cast<float>(fb.negative_count()));

View file

@ -104,6 +104,7 @@ function bind(Module) {
feedbackLike: cwrap('nisps_ml_feedback_like', null, ['number']), feedbackLike: cwrap('nisps_ml_feedback_like', null, ['number']),
feedbackCommitPlace: cwrap('nisps_ml_feedback_commit_place', null, ['number']), feedbackCommitPlace: cwrap('nisps_ml_feedback_commit_place', null, ['number']),
feedbackPlacedOutput: cwrap('nisps_ml_feedback_placed_output', 'number', ['number','number']), feedbackPlacedOutput: cwrap('nisps_ml_feedback_placed_output', 'number', ['number','number']),
feedbackAdvanceGeometric: cwrap('nisps_ml_feedback_advance_geometric', 'number', ['number','number']),
feedbackPositiveCount: cwrap('nisps_ml_feedback_positive_count', 'number', ['number']), feedbackPositiveCount: cwrap('nisps_ml_feedback_positive_count', 'number', ['number']),
feedbackNegativeCount: cwrap('nisps_ml_feedback_negative_count', 'number', ['number']), feedbackNegativeCount: cwrap('nisps_ml_feedback_negative_count', 'number', ['number']),
describe: cwrap('nisps_ml_describe', null, ['number','number']), describe: cwrap('nisps_ml_describe', null, ['number','number']),
@ -333,9 +334,9 @@ async function main() {
// --- Stage 6: geometric dislike (one-core-engine P3) --- // --- Stage 6: geometric dislike (one-core-engine P3) ---
// Mirrors parity_check.cpp stage 6: two likes feed the replay positives via // Mirrors parity_check.cpp stage 6: two likes feed the replay positives via
// the Avoid+Geometric on_up path, then two dislikes (second deepens) train // the Avoid+Geometric on_up path, then two dislikes (second deepens) plus
// toward the computed push-away target. f32 arithmetic for the "heard" // eight 5ms replay ticks train toward the computed push-away target. f32
// vector via Math.fround to match native float ops exactly. // arithmetic for the "heard" vector via Math.fround matches native exactly.
const FB_AVOID = 0; const FB_AVOID = 0;
api.feedbackSetMode(ml, FB_AVOID); api.feedbackSetMode(ml, FB_AVOID);
@ -368,6 +369,7 @@ async function main() {
}; };
dislikeAt(0.25, 0.75); dislikeAt(0.25, 0.75);
dislikeAt(0.26, 0.74); // within dedup radius: deepen + push dislikeAt(0.26, 0.74); // within dedup radius: deepen + push
for (let i = 0; i < 8; i++) api.feedbackAdvanceGeometric(ml, 0.005);
feedbackFloats.push(api.feedbackPositiveCount(ml)); feedbackFloats.push(api.feedbackPositiveCount(ml));
feedbackFloats.push(api.feedbackNegativeCount(ml)); feedbackFloats.push(api.feedbackNegativeCount(ml));

View file

@ -3,9 +3,8 @@
// //
// Covers: replay dedup/deepen at radius 0.05, k-NN centroid selection with // Covers: replay dedup/deepen at radius 0.05, k-NN centroid selection with
// deterministic index tie-break, push direction sign (target moves AWAY from // deterministic index tie-break, push direction sign (target moves AWAY from
// the liked centroid), taper, cold-start posMemCount==0 fallback, decay/ // the liked centroid), cold-start posMemCount==0 fallback, parameterised replay
// eviction + dislike-multiplier bookkeeping, solo/focus gating, and fixed-seed // dose/lifetime, solo/focus gating, and fixed-seed determinism.
// determinism.
#include <array> #include <array>
#include <cmath> #include <cmath>
@ -36,10 +35,11 @@ struct RawReplay {
std::array<float, kCap * kNIn> in{}; std::array<float, kCap * kNIn> in{};
std::array<float, kCap * kNOut> act{}; std::array<float, kCap * kNOut> act{};
std::array<float, kCap> rew{}; std::array<float, kCap> rew{};
std::array<float, kCap> age_ms{};
std::size_t count = 0u; std::size_t count = 0u;
ReplayView view() { ReplayView view() {
return ReplayView(in, act, rew, kNIn, kNOut, kCap, count); return ReplayView(in, act, rew, age_ms, kNIn, kNOut, kCap, count);
} }
}; };
@ -118,24 +118,27 @@ NISPS_TEST(replay_knn_centroid_deterministic_tie_break) {
NISPS_EXPECT(used_after_neg == 3u); NISPS_EXPECT(used_after_neg == 3u);
} }
NISPS_TEST(replay_decay_and_evict) { NISPS_TEST(replay_wall_clock_lifetime_and_evict) {
RawReplay raw; RawReplay raw;
auto r = raw.view(); auto r = raw.view();
const float x[kNIn] = {0.1f, 0.1f}; const float x[kNIn] = {0.1f, 0.1f};
const float a[kNOut] = {}; const float a[kNOut] = {};
// A shallow negative just above the evict threshold decays out in a few r.store(-1.f, std::span<const float>(x), std::span<const float>(a));
// calls; rewards move by +0.0025*max(|r|,1) per call.
r.store(-0.012f, std::span<const float>(x), std::span<const float>(a));
NISPS_ASSERT(r.size() == 1u); NISPS_ASSERT(r.size() == 1u);
std::size_t evicted = r.decay_negatives(); // -0.012 + 0.0025 = -0.0095 > -0.01 → evict std::size_t evicted = r.advance_negative_ages(2499.f, 2500.f);
NISPS_EXPECT(evicted == 0u);
NISPS_EXPECT(r.size() == 1u);
NISPS_EXPECT_NEAR(r.age_ms(0), 2499.f, 1e-6);
evicted = r.advance_negative_ages(1.f, 2500.f);
NISPS_EXPECT(evicted == 1u); NISPS_EXPECT(evicted == 1u);
NISPS_EXPECT(r.size() == 0u); NISPS_EXPECT(r.size() == 0u);
// Positives are never decayed/evicted. // Positives do not age or expire.
r.store(1.f, std::span<const float>(x), std::span<const float>(a)); r.store(1.f, std::span<const float>(x), std::span<const float>(a));
evicted = r.decay_negatives(); evicted = r.advance_negative_ages(5000.f, 2500.f);
NISPS_EXPECT(evicted == 0u); NISPS_EXPECT(evicted == 0u);
NISPS_EXPECT(r.size() == 1u); NISPS_EXPECT(r.size() == 1u);
NISPS_EXPECT(r.age_ms(0) == 0.f);
} }
// -- compute_push_target -------------------------------------------------------- // -- compute_push_target --------------------------------------------------------
@ -302,6 +305,38 @@ NISPS_TEST(geo_dislike_deterministic_under_fixed_seed) {
} }
} }
NISPS_TEST(geo_dislike_replay_rate_and_lifetime_are_parameterised) {
GeoMLP m(88ull);
m.draw_weights(0.5f);
GeoFB fb(99ull);
NISPS_EXPECT(fb.geo_lr() == 0.001f);
NISPS_EXPECT(fb.geo_update_hz() == 200.f);
NISPS_EXPECT(fb.geo_lifetime_ms() == 2500.f);
fb.set_geo_lr(0.002f);
fb.set_geo_update_hz(20.f);
fb.set_geo_lifetime_ms(100.f);
set_inputs(m, 0.4f, 0.6f);
m.process();
fb.on_down(m, {}, 0.1f, 0.5f, {}); // one immediate update
NISPS_ASSERT(fb.negative_count() == 1u);
// 20 Hz gives one replay update per 50 ms. At 100 ms the rejection
// expires after its second scheduled update.
NISPS_EXPECT(fb.advance_geometric(m, 0.025f) == 0u);
NISPS_EXPECT(fb.advance_geometric(m, 0.025f) == 1u);
NISPS_EXPECT(fb.negative_count() == 1u);
NISPS_EXPECT(fb.advance_geometric(m, 0.05f) == 1u);
NISPS_EXPECT(fb.negative_count() == 0u);
NISPS_EXPECT(fb.advance_geometric(m, 0.05f) == 0u);
// Zero rate is an explicit one-shot mode: the press still updates once,
// but no stale negative remains to affect a later press.
fb.set_geo_update_hz(0.f);
fb.on_down(m, {}, 0.1f, 0.5f, {});
NISPS_EXPECT(fb.negative_count() == 0u);
}
NISPS_TEST(geo_dislike_diffuse_style_preserves_legacy_path) { NISPS_TEST(geo_dislike_diffuse_style_preserves_legacy_path) {
GeoMLP m(9ull); GeoMLP m(9ull);
GeoFB fb(9ull); GeoFB fb(9ull);