# Findings — Current RL / Feedback Behaviour (Phase-1 Audit)

*Read-only audit, 2026-06-27. Citations are `file:line`. "VERIFIED" = read in source; "INFER" = deduced.
Produced by the Phase-1 research fan-out (workflow `wf_db146f21`), persisted by the orchestrator because a
hook blocks subagent report-file writes. Adversarially spot-checked against source.*

## Executive summary

1. The operator's mental model ("negative feedback takes the n-D opposite vector and pushes weights the other
   way — geometric push-away") describes the **firmware `InterfaceRL`** path, NOT what is deployed in the
   browser/aimmersive build.
2. The **deployed JS engine** and the **`nisps/` C++ core** both implement thumbs-down as
   `move_weights(speed, spread, pinMask)` = per-weight decay (`w *= 1-0.1·spread`) then additive Gaussian
   noise (`stddev = speed · layer_scale`). There is **no opposite vector, no centroid, no geometric direction**
   in deployed/core code (VERIFIED: grep for `centroid|geometric|meanPositive|neg_action` returns nothing in
   `deployments/meml-aimmersive/js/`).
3. So "push the weights the other way" is, in the deployed system, actually **"jitter the whole network with
   random Gaussian noise"** — undirected diffusion, not directed avoidance.
4. The true geometric push-away (move output away from k-NN centroid of liked examples) exists only in the
   firmware submodule (`memllib` InterfaceRL; "geo push" commits) and is documented in
   `docs/redesign/feedback-modes-port-spec.md` §1.1. **This is the algorithm Mode 1 must match** (workstream B).
5. **Why often inaudible (deployed AVOID):** noise added across all 4 layers of `MLP<2,10,14,18,126>`,
   per-layer scaled `(1-spread)+spread/√fan_in`, capped at `noiseCap=0.12`, spread default 0.6 → tiny
   per-step perturbation that the sigmoid output + output smoothing/slew pipeline absorb. One press moves the
   heard 126-vector very little, in a random direction uncorrelated with what was disliked.
6. **Three FEEDBACK_MODEs already scaffolded** in untracked working files `nisps/ml/feedback.hpp` +
   `tests/cpp/test_mlp_feedback.cpp` (`FeedbackController<MLP_T>`): `Avoid`, `RandomiseOutputs`,
   `RandomiseMlp`. Not yet wired into bindings/TS/UI; no CMake registration.
7. `RandomiseMlp` = the operator's Mode 2 as an audition: snapshot weights → `draw_weights(spread)` → user
   moves joystick to hear → up commits a +1 example, down/switch restores the snapshot. **This is the seed of
   Mode 2** but the updated spec is stricter (scratchpad NEVER trained; explicit place-at-location; warm-start
   real model to interpolate anchors).
8. `RandomiseOutputs` = bypass the MLP entirely, hold a static random output vector (focus-aware per-dim),
   re-roll on each down; up commits the held vector as a +1 example.
9. The operator's "randomise → anchor → train" alternative maps onto `RandomiseMlp` (randomise) + thumbs-up
   `LikeStore` (anchor) + `trainOnCurrent` (train). Missing in deployed build: the randomise-audition-then-anchor
   loop; today thumbs-down only diffuses noise and never anchors.
10. The full port spec (component, WASM C API, TS FFI, ~22 ctest cases, parity stage) already exists at
    `docs/redesign/feedback-modes-port-spec.md` and matches the untracked `feedback.hpp`.
11. **Parity/perf constraints:** no heap / `std::array` only, per-instance deterministic `nisps::Rng`
    (no libc `rand()`), no virtual dispatch, `.f` literals, fixed WASM arch `MLP<2,10,14,18,126>`,
    native↔WASM parity within `1e-5`.
12. **Bottom line:** deployed negative-feedback is undirected Gaussian noise (not geometric push-away), which is
    why it is weak/inaudible; the in-progress `FeedbackController` already implements the seed of the operator's
    randomise-anchor-train idea but is unplumbed past the header.

## 1. Current algorithm, precisely, per layer

### 1.1 DEPLOYED (browser/aimmersive) — NOT geometric
Thumbs-down = `move_weights`. Pure-JS reference `deployments/meml-aimmersive/js/nisps/mlp.js:238-265`
(VERIFIED): per layer/node/weight `j`: `decay = 1-0.1*spread`; `layerScale = (1-spread)+(1/√fanIn)*spread`;
`node.weights[j] *= decay`; `accum = Σ_{n=0..2}(rand*2-1)`; `node.weights[j] += 3*accum*speed*layerScale`.
Output-layer nodes with `outputPinMask[ni]` skipped (`mlp.js:248`). WASM wrapper `nisps-wasm.js:389-406` →
`cwrap('nisps_mlp_move_weights_ex')`. IML `iml.js:166-170` forwards then re-infers — **no dislike-direction
logic, no example stored on dislike**.

**Key finding (VERIFIED, exhaustive grep):** `deployments/meml-aimmersive/js/` has zero matches for
`centroid|meanPositive|geometric|pushStep|dislikeMultiplier|neg_action|RANDOMISE|FEEDBACK_MODE|explore_active`.
Deployed negative feedback is purely undirected Gaussian weight diffusion. The disliked 126-vector is never read.

### 1.2 `nisps/` C++ core — same move_weights, no geometric push
`nisps/ml/rl.hpp:63-100` `move_weights_layer` (VERIFIED): `layer_scale = spread_scale(spread,fan_in) =
(1-spread)+spread/√fan_in` (`init.hpp:31-35`); `noise_stddev = speed*layer_scale`; `decay = 1-0.1*spread`
(`rl.hpp:49-51`). Bias: `biases[node] += rng.next_float_gaussian(noise_stddev)` unless pinned (`rl.hpp:83`).
Each weight: `w = w*decay + rng.next_float_gaussian(noise_stddev)` (`rl.hpp:92-97`); RNG advanced even on skip.
Pin mask only on `is_final_layer` (`rl.hpp:78-79`). `mlp.hpp:312-322` iterates 4 layers, only layer3
final+masked. `draw_weights` (`mlp.hpp:324-332`→`init.hpp:42-54`): `w = rng.next_float_signed()*spread_scale`,
biases zeroed. C API `nisps_ml_move_weights`/`nisps_ml_draw_weights` (`bindings.cpp:419-437`). TS
`wasm-iml.ts:495-508`. Handler `mode-runtime.ts:548-565`: builds pin mask, `moveWeights(cap,spread,pinMask)`,
`growNoise`, re-`setInput`. **No example stored on thumbs-down** (vs thumbs-up `mode-runtime.ts:523-546`).

### 1.3 TRUE geometric push-away (firmware only) — THE MODE-1 TARGET
Per `feedback-modes-port-spec.md:61-79` (sourced from firmware `InterfaceRL.cpp`), applied in `optimise()` not
at press time: `pushStep = clamp(|avgRewardNeg|,0.25,1.0)*0.5`; `dir[j] = neg_action[j] -
meanPositiveAction[j]` (away from k-NN(+) centroid); `target[j] = clamp(neg_action[j] +
(dir/||dir||)[j]*pushStep/(1+||dir||), 0, 1)`; then train toward `target`. Depends on firmware-only
`ReplayMemory`. Lives in `src/memllib` submodule (commits `2429bcc` "geo push", `d301cc7` "rlview and dislike";
upstream `SB2026`; memllib main `e291192`/`abe93ec`). **Workstream A must confirm the newest variant.**

## 2. FEEDBACK_MODEs implemented / scaffolded, and where

| Mode | Firmware memllib | nisps/ core (untracked) | Deployed JS | Playground TS |
|---|---|---|---|---|
| AVOID geometric push | VERIFIED (geo-push commits) | NOT ported | — | — |
| AVOID = move_weights | — | `feedback.hpp:103-105` | `mlp.js`/`nisps-wasm.js` | `mode-runtime.ts:548` |
| RANDOMISE_OUTPUTS | VERIFIED (spec §1.2) | `feedback.hpp:106-112,160-181` | — | — |
| RANDOMISE_MLP | VERIFIED (spec §1.3) | `feedback.hpp:113-119,183-189` | — | — |

**Untracked `nisps/ml/feedback.hpp`** (214 lines, VERIFIED): `nisps::ml::FeedbackController<MLP_T>`.
`enum FeedbackMode {Avoid=0,RandomiseOutputs=1,RandomiseMlp=2}` (`:47-51`); `enum FeedbackAction
{None,AvoidPerturb,LikeStore,EnterExplore,Reroll,CommitStore,Cancel,Restore}` (`:56-65`). State all
`std::array` (no heap), own `Rng` seeded separately (`:203-210`). `on_down`: Avoid→move_weights;
RandOut→enter/roll; RandMlp→enter(snapshot+draw_weights)/cancel. `on_up`: exploring→restore+CommitStore else
LikeStore. `on_drag`: RandMlp+exploring→restore+Restore else LikeStore. `static_output`: bypass hook only when
RandomiseOutputs&&exploring. `roll_static_outputs`: focused→`rng_.next_float_uniform()`, unfocused frozen.
`set_mode` aborts active exploration first.

**Untracked `tests/cpp/test_mlp_feedback.cpp`** (437 lines, ~22 NISPS_TEST, VERIFIED):
`SmallMLP=MLP<2,4,4,4,6,8,32>`; covers all transitions, RNG determinism, focus edge cases, a golden RNG
stream. **Not registered in `nisps/CMakeLists.txt`** (spec §5.1/§7 lists as TODO).

**Not yet wired (gaps):** no `nisps_ml_feedback_*` in `bindings.cpp`; no `FeedbackController` on `MLHandle`;
no TS FFI in `types.ts`; no UI selector. Firmware parent branch `origin/feat/feedback-explore-modes` wires a
TR-8S FEEDBACK_MODE selector (`5c2d4d2`) and adds `docs/dislike_system_{analysis,design_space}.md` (4 meanings
of "dislike"; here/anywhere × point/radius scope matrix — VERIFIED).

## 3. Spread / noise / zoom machinery
- **spread∈[0,1]** (`exploration-store.ts:35,74`, default 0.6): feeds per-layer scale and decay. spread=0:
  uniform weights, big noise, no decay (saturated); spread=1: Xavier, small noise, 10% decay (spread out).
- **noiseCap∈[floor,1]** (default 0.12): the `speed`/`cap` magnitude for thumbs-down and auto-explore.
- **growNoise/decayNoise** driven by pointer `pressureForce` (`mode-runtime.ts:491-499,544,562`).
- **Zoom** scales auto-explore intensity: `scaledIntensity = intensity*(0.3+0.7*zoom)` — an input-pipeline
  gain, not output magnitude.
- **Auto-explore loop** (`mode-runtime.ts:444-482`): periodic `move_weights`+`growNoise`, default 2000ms, off
  by default — same diffusion as thumbs-down.
- **Pin mask** `buildPinMask(...)`: freezes final-layer weights for pinned/overridden params — the only
  directional gating today (protects dims, doesn't push them). *Relevant to the SOLO/arm design (B).*

## 4. Parity/perf constraints any core change must respect
No heap in `nisps/` (`std::array`/`FixedBuffer`; `feedback.hpp` obeys); deterministic per-instance `nisps::Rng`,
no libc `rand()`; no virtual dispatch; `.f` literals, no double promotion, `NISPS_*` mem attrs; fixed WASM arch
`MLP<2,10,14,18,126>` (snapshot = `weight_count()` floats, compile-time fixed); native↔WASM parity within
`1e-5`. RandomiseMlp snapshot/restore must round-trip `get_weights`/`set_weights` byte-identically;
RandomiseOutputs static vectors need same-seed/same-Rng; golden tests pin RNG streams.

## 5. WHY the effect is inaudible — the mechanism
1. **Undirected, not opposite.** Zero-mean Gaussian noise on every weight in all 4 layers; the disliked output
   is never read. Expected output change ≈ 0; only variance moves — wobbles randomly, doesn't move *away*.
2. **Small magnitude, multiply-attenuated.** `speed=0.12, spread=0.6` → output layer `layer_scale≈0.54`,
   `stddev≈0.065` per weight, squashed by a near-saturated sigmoid, then damped by smoothing→slew→freeze.
3. **Decay fights accumulation.** `w *= 0.94` each press pulls back toward the basin → equilibrium by design.
4. **Pinned/override dims frozen** on the final layer → never respond.
5. **No anchoring.** Thumbs-down stores no example and triggers no training, so there's no learning signal.

## 6. Operator's Mode 2 ("explore-and-place") vs what exists
Already half-built as `RandomiseMlp`:
- **Randomise** = `RandomiseMlp::on_down` enter → `snapshot + draw_weights(spread)`: whole net re-rolled →
  audibly different; user auditions via joystick.
- **Anchor** = `on_up` → `restore_after_explore` + `CommitStore`: caller stores +1 example at
  `(current_input, heard_output)`, original net restored.
- **Train** = existing `trainOnCurrent()` trains restored net toward the anchored example.
- **Discard** = `on_down` again → `Cancel`, or mode-switch `abort_explore` restores snapshot.

**Updated-spec deltas to implement (workstream B):** the scratchpad must be NEVER trained (only randomise +
undoable nudge); the place step must let the user CHOOSE the input location (press → pick location → release);
the real model is warm-started to interpolate ALL placed anchors. The existing `RandomiseMlp` is the closest
prototype but commits at the *current* input position, not a chosen one, and lacks the explicit nudge op.

### Key files
- Deployed: `deployments/meml-aimmersive/js/nisps/{mlp.js,nisps-wasm.js,iml.js}`
- Core RL: `nisps/ml/{rl.hpp,init.hpp,mlp.hpp}`
- Core feedback (untracked): `nisps/ml/feedback.hpp`, `tests/cpp/test_mlp_feedback.cpp`
- WASM API: `nisps/wasm/bindings.cpp:419-437`
- TS: `playground/src/ml/wasm-iml.ts:488-508`; runtime `playground/src/modes/mode-runtime.ts:444-565`
- Exploration knobs: `playground/src/stores/exploration-store.ts`
- Port spec: `docs/redesign/feedback-modes-port-spec.md`
- Firmware geo-push: `src/memllib` (commits `2429bcc`, `d301cc7`; upstream `SB2026`; main `e291192`/`abe93ec`);
  design docs on `origin/feat/feedback-explore-modes`: `docs/dislike_system_{analysis,design_space}.md`