w1n5t0n/memlnaut-nisps
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(ml)!: P3 core — geometric dislike in nisps/, jolt/OU + geo ABI

Browse source 
Geometric dislike (rl-feedback-design §2.1/§4; upstream InterfaceRL @
0a541cc ported verbatim, constants included):

- nisps/ml/replay.hpp: ReplayView over storage-owned buffers — deepen-or-
  store negatives (dedup 0.05, clamp -16), k-NN positive centroid with
  deterministic index tie-break + fixed accumulation order, proportional
  decay (0.0025*max(|r|,1)) + eviction, order-preserving compaction.
- nisps/ml/geo_push.hpp: push-away target (pushStep clamp(|avgNeg|,.25,1)
  *0.5, taper /(1+len), useRandom on len<=1e-4 via nisps::Rng — the single
  deliberate divergence from libc rand()), negLRRatio 0.5-0.4*negFraction.
- mlp.hpp: train_targets(input, computed-target, lr, out_mask) — trains
  toward computed targets (negative lr = cold-start train-away); solo/
  focus gating zeroes masked derivs.
- feedback.hpp: AvoidStyle {Geometric (new default), Diffuse (legacy
  move_weights, kept for A/B)}; dislike_geometric() collapses upstream's
  press+optimise into one synchronous call; on_up in geometric Avoid
  feeds the positive centroid; dislike-multiplier bookkeeping. Storage
  gains replay buffers (Fixed: ReplayCap=32 firmware default ≈ +8KB SRAM;
  Dynamic arena: cap 64).
- bindings: nisps_ml_feedback_{dislike_geometric,store_positive,
  positive_count,negative_count,set_avoid_style} + P3.2 jolt/OU ABI:
  nisps_ml_jolt_{press,step,release,active,lr_scale,tick_lr_ramp},
  nisps_ml_explore_{intensity,get_intensity,apply} (OUNoise<4096>
  over-provisioned; same code the firmware ModeBase runs).
- parity v4: Stage 6 scripted geometric session (2 likes → 2 dislikes,
  f32-exact heard vectors via Math.fround) — 961 floats PASS at 2.4e-7.
- tests: test_mlp_geo_dislike.cpp (replay dedup/deepen/clamp, centroid
  tie-break, push direction/taper/mask/clamp, cold-start inertness +
  train-away, determinism, Diffuse legacy); legacy Avoid test pinned to
  Diffuse per the ADR's deliberate-break note.

Firmware: PAFSynth .text/.data unchanged (geometric path not referenced
by current glue). NOTE: discovered pre-existing bug 10c3e55c — the
explore/place wiring is linker-GC'd out of the PAFSynth ELF (predates
this refactor; evidence in the ergo task).
This commit is contained in:
monkey-w1n5t0n 2026-07-14 04:16:21 +02:00
parent 7e957457cd
commit 9490e20a7a
15 changed files with 1136 additions and 30 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
manifold/public/nisps.js
View file

File diff suppressed because one or more lines are too long

BIN
manifold/public/nisps.wasm
View file

Binary file not shown.

1
nisps/CMakeLists.txt
View file

@ -60,6 +60,7 @@ if(NOT EMSCRIPTEN)
${NISPS_TEST_DIR}/test_mlp_jolt.cpp
${NISPS_TEST_DIR}/test_mlp_ou_noise.cpp
${NISPS_TEST_DIR}/test_mlp_feedback.cpp
${NISPS_TEST_DIR}/test_mlp_geo_dislike.cpp
${NISPS_TEST_DIR}/test_mlp_serialize.cpp
)
target_link_libraries(nisps_core_tests PRIVATE nisps_core)

57
nisps/ml/dynamic_storage.hpp
View file

@ -203,23 +203,39 @@ class DynamicFeedbackStorage {
public:
DynamicFeedbackStorage(std::size_t n_out,
std::size_t n_weights,
std::size_t undo_depth = 4u) noexcept
: n_out_(n_out), n_weights_(n_weights), undo_cap_(undo_depth) {
if (n_out == 0u || n_weights == 0u || undo_depth == 0u) return;
// float regions: static_out, placed_out, snapshot, scratch, undo ring
std::size_t undo_depth = 4u,
std::size_t n_in = 2u,
std::size_t replay_cap = 64u) noexcept
: n_out_(n_out), n_weights_(n_weights), undo_cap_(undo_depth),
n_in_(n_in), replay_cap_(replay_cap) {
if (n_out == 0u || n_weights == 0u || undo_depth == 0u ||
n_in == 0u || replay_cap == 0u) {
return;
}
// float regions: static_out, placed_out, snapshot, scratch, undo ring,
// replay (inputs/actions/rewards), centroid + target scratch
// byte region: focus mask (n_out bytes, rounded up to whole floats)
const std::size_t focus_floats = (n_out + sizeof(float) - 1u) / sizeof(float);
const std::size_t total = n_out * 2u // static_out + placed_out
+ n_weights * 2u // snapshot + scratch
+ n_weights * undo_depth // undo ring
+ replay_cap * n_in // replay inputs
+ replay_cap * n_out // replay actions
+ replay_cap // replay rewards
+ n_out * 2u // centroid + target
+ focus_floats;
arena_ = new (std::nothrow) float[total]();
if (!arena_) return;
off_placed_ = n_out_;
off_snap_ = off_placed_ + n_out_;
off_scratch_ = off_snap_ + n_weights_;
off_undo_ = off_scratch_ + n_weights_;
off_focus_ = off_undo_ + n_weights_ * undo_cap_;
off_placed_ = n_out_;
off_snap_ = off_placed_ + n_out_;
off_scratch_ = off_snap_ + n_weights_;
off_undo_ = off_scratch_ + n_weights_;
off_replay_in_ = off_undo_ + n_weights_ * undo_cap_;
off_replay_a_ = off_replay_in_ + replay_cap_ * n_in_;
off_replay_r_ = off_replay_a_ + replay_cap_ * n_out_;
off_centroid_ = off_replay_r_ + replay_cap_;
off_target_ = off_centroid_ + n_out_;
off_focus_ = off_target_ + n_out_;
}
~DynamicFeedbackStorage() { delete[] arena_; }
@ -237,9 +253,11 @@ class DynamicFeedbackStorage {
bool valid() const noexcept { return arena_ != nullptr; }
std::size_t n_out() const noexcept { return n_out_; }
std::size_t n_weights() const noexcept { return n_weights_; }
std::size_t undo_cap() const noexcept { return undo_cap_; }
std::size_t n_out() const noexcept { return n_out_; }
std::size_t n_weights() const noexcept { return n_weights_; }
std::size_t undo_cap() const noexcept { return undo_cap_; }
std::size_t n_in() const noexcept { return n_in_; }
std::size_t replay_cap() const noexcept { return replay_cap_; }
std::span<float> static_out() noexcept { return {arena_, n_out_}; }
std::span<const float> static_out() const noexcept { return {arena_, n_out_}; }
@ -254,6 +272,11 @@ class DynamicFeedbackStorage {
std::span<const float> undo_slot(std::size_t i) const noexcept {
return {arena_ + off_undo_ + i * n_weights_, n_weights_};
}
std::span<float> replay_inputs() noexcept { return {arena_ + off_replay_in_, replay_cap_ * n_in_}; }
std::span<float> replay_actions() noexcept { return {arena_ + off_replay_a_, replay_cap_ * n_out_}; }
std::span<float> replay_rewards() noexcept { return {arena_ + off_replay_r_, replay_cap_}; }
std::span<float> centroid_buf() noexcept { return {arena_ + off_centroid_, n_out_}; }
std::span<float> target_buf() noexcept { return {arena_ + off_target_, n_out_}; }
std::span<std::uint8_t> focus() noexcept {
return {reinterpret_cast<std::uint8_t*>(arena_ + off_focus_), n_out_};
}
@ -264,15 +287,21 @@ class DynamicFeedbackStorage {
private:
void move_from_(DynamicFeedbackStorage& o) noexcept {
n_out_ = o.n_out_; n_weights_ = o.n_weights_; undo_cap_ = o.undo_cap_;
n_in_ = o.n_in_; replay_cap_ = o.replay_cap_;
off_placed_ = o.off_placed_; off_snap_ = o.off_snap_;
off_scratch_ = o.off_scratch_; off_undo_ = o.off_undo_; off_focus_ = o.off_focus_;
off_replay_in_ = o.off_replay_in_; off_replay_a_ = o.off_replay_a_;
off_replay_r_ = o.off_replay_r_; off_centroid_ = o.off_centroid_;
off_target_ = o.off_target_;
arena_ = o.arena_;
o.arena_ = nullptr;
}
std::size_t n_out_ = 0u, n_weights_ = 0u, undo_cap_ = 0u;
std::size_t n_out_ = 0u, n_weights_ = 0u, undo_cap_ = 0u, n_in_ = 0u, replay_cap_ = 0u;
std::size_t off_placed_ = 0u, off_snap_ = 0u, off_scratch_ = 0u,
off_undo_ = 0u, off_focus_ = 0u;
off_undo_ = 0u, off_focus_ = 0u, off_replay_in_ = 0u,
off_replay_a_ = 0u, off_replay_r_ = 0u, off_centroid_ = 0u,
off_target_ = 0u;
float* arena_ = nullptr;
};

176
nisps/ml/feedback.hpp
View file

@ -51,16 +51,27 @@
#include "../core/perf.hpp"
#include "../core/rng.hpp"
#include "geo_push.hpp"
#include "replay.hpp"
namespace nisps::ml {
enum class FeedbackMode : std::uint8_t {
Avoid = 0, // down → move_weights (Gaussian perturb). No internal state.
Avoid = 0, // down → geometric push-away (or legacy Diffuse — see AvoidStyle).
RandomiseOutputs = 1, // down → bypass MLP, hold static random vector; re-roll each down.
RandomiseMlp = 2, // down → snapshot + draw_weights live net; down-again cancels.
ExploreAndPlace = 3, // Idle→Exploring→Placing→Idle scratchpad lifecycle (default product mode).
};
// How the Avoid mode realises a dislike (rl-feedback-design §2.1). Geometric
// is the ported firmware behaviour (replay-backed k-NN centroid push-away);
// Diffuse is the pre-P3 undirected move_weights, kept reachable as a legacy
// sub-mode for A/B comparison.
enum class AvoidStyle : std::uint8_t {
Geometric = 0,
Diffuse = 1,
};
// The explicit lifecycle state for FeedbackMode::ExploreAndPlace. The whole
// mode is a three-state machine; granular methods drive the transitions
// (firmware maps buttons to them directly), while on_down/on_up implement the
@ -97,6 +108,9 @@ enum class FeedbackAction : std::uint8_t {
BeginPlace = 11, // Exploring→Placing; placed_out captured + frozen (no store yet).
CommitPlace = 12, // Placing→Idle; real net restored. CALLER adds +1 (input→placed_output) + trains.
CancelPlace = 13, // Placing→Exploring; backed out of placing (no store).
// ---- Geometric dislike (append-only) ----
GeometricPush = 14, // dislike trained toward the computed push-away target.
GeometricColdStart = 15, // no positives yet: negative-LR fallback ran; UI shows the cold-start prompt.
};
// ---------------------------------------------------------------------------
@ -105,16 +119,21 @@ enum class FeedbackAction : std::uint8_t {
// ExploreAndPlace; each undo slot is NWeights floats. WASM historically used
// depth 4, firmware 2 (per rl-feedback-design §2.2 — SRAM budget).
// ---------------------------------------------------------------------------
template <std::size_t NOut, std::size_t NWeights, std::size_t UndoDepth = 4u>
template <std::size_t NOut, std::size_t NWeights, std::size_t UndoDepth = 4u,
std::size_t NIn = 2u, std::size_t ReplayCap = 32u>
class FixedFeedbackStorage {
public:
static constexpr std::size_t kNOut = NOut;
static constexpr std::size_t kWeights = NWeights;
static constexpr std::size_t kUndoDepth = UndoDepth;
static constexpr std::size_t kNIn = NIn;
static constexpr std::size_t kReplayCap = ReplayCap;
static constexpr std::size_t n_out() noexcept { return NOut; }
static constexpr std::size_t n_weights() noexcept { return NWeights; }
static constexpr std::size_t undo_cap() noexcept { return UndoDepth; }
static constexpr std::size_t n_out() noexcept { return NOut; }
static constexpr std::size_t n_weights() noexcept { return NWeights; }
static constexpr std::size_t undo_cap() noexcept { return UndoDepth; }
static constexpr std::size_t n_in() noexcept { return NIn; }
static constexpr std::size_t replay_cap() noexcept { return ReplayCap; }
NISPS_FORCE_INLINE std::span<float> static_out() noexcept { return static_out_; }
NISPS_FORCE_INLINE std::span<const float> static_out() const noexcept { return static_out_; }
@ -129,6 +148,13 @@ class FixedFeedbackStorage {
NISPS_FORCE_INLINE std::span<const float> undo_slot(std::size_t i) const noexcept {
return undo_ring_[i];
}
// Replay memory buffers (geometric dislike — nisps/ml/replay.hpp).
NISPS_FORCE_INLINE std::span<float> replay_inputs() noexcept { return replay_in_; }
NISPS_FORCE_INLINE std::span<float> replay_actions() noexcept { return replay_act_; }
NISPS_FORCE_INLINE std::span<float> replay_rewards() noexcept { return replay_rew_; }
// Centroid + push-target scratch (n_out each).
NISPS_FORCE_INLINE std::span<float> centroid_buf() noexcept { return centroid_; }
NISPS_FORCE_INLINE std::span<float> target_buf() noexcept { return target_; }
private:
std::array<float, NOut> static_out_{};
@ -137,6 +163,11 @@ class FixedFeedbackStorage {
std::array<float, NOut> placed_out_{};
std::array<std::array<float, NWeights>, UndoDepth> undo_ring_{};
std::array<float, NWeights> scratch_buf_{};
std::array<float, ReplayCap * NIn> replay_in_{};
std::array<float, ReplayCap * NOut> replay_act_{};
std::array<float, ReplayCap> replay_rew_{};
std::array<float, NOut> centroid_{};
std::array<float, NOut> target_{};
};
// ---------------------------------------------------------------------------
@ -162,6 +193,16 @@ class FeedbackControllerCore : public FbStorage {
}
FeedbackMode mode() const noexcept { return mode_; }
// Avoid sub-mode: Geometric (default, the ported firmware behaviour) or
// Diffuse (legacy undirected move_weights — kept for A/B comparison).
void set_avoid_style(AvoidStyle s) noexcept { avoid_style_ = s; }
AvoidStyle avoid_style() const noexcept { return avoid_style_; }
// Base learning rate for the geometric push training (upstream
// InterfaceRL default 1e-3, pre-scaling).
void set_geo_lr(float lr) noexcept { geo_lr_ = lr; }
float geo_lr() const noexcept { return geo_lr_; }
// `exploring()` is true whenever a scratchpad net is live and learning is
// paused — for the legacy RANDOMISE_* modes, AND for ExploreAndPlace in
// either Exploring or Placing (the real net stays snapshotted aside the
@ -209,8 +250,11 @@ class FeedbackControllerCore : public FbStorage {
std::span<const std::uint8_t> pin_mask) noexcept {
switch (mode_) {
case FeedbackMode::Avoid:
mlp.move_weights(speed, spread, pin_mask);
return FeedbackAction::AvoidPerturb;
if (avoid_style_ == AvoidStyle::Diffuse) {
mlp.move_weights(speed, spread, pin_mask);
return FeedbackAction::AvoidPerturb;
}
return dislike_geometric(mlp, current_out, geo_lr_);
case FeedbackMode::RandomiseOutputs:
if (!explore_active_) {
enter_randomise_outputs(current_out);
@ -272,6 +316,11 @@ class FeedbackControllerCore : public FbStorage {
restore_after_explore(mlp);
return FeedbackAction::CommitStore;
}
if (mode_ == FeedbackMode::Avoid && avoid_style_ == AvoidStyle::Geometric) {
// A geometric-mode like also feeds the positive centroid (ADR
// §2.1); the caller still runs addExample + train as usual.
store_positive(mlp);
}
return FeedbackAction::LikeStore;
}
@ -309,6 +358,92 @@ class FeedbackControllerCore : public FbStorage {
void seed(std::uint64_t s) noexcept { rng_.seed(s); }
// =========================================================================
// Geometric dislike (rl-feedback-design §2.1) — the press-time half and
// the async optimise() half of upstream InterfaceRL collapsed into ONE
// synchronous call (nisps has no background optimise driver).
// =========================================================================
std::size_t replay_size() const noexcept { return replay_count_; }
std::size_t positive_count() noexcept { return replay_().positive_count(); }
std::size_t negative_count() noexcept { return replay_().negative_count(); }
std::size_t dislike_multiplier() const noexcept { return dislike_multiplier_; }
// Store a positive (like) into the replay so the k-NN centroid sees it.
// `current_out` may be empty ⇒ the MLP's live output vector is used. The
// input is the MLP's current input vector.
template <typename M>
void store_positive(M& mlp, std::span<const float> current_out = {}) noexcept {
std::span<const float> a = current_out.empty()
? std::span<const float>(mlp.outputs())
: current_out;
replay_().store(1.f, std::span<const float>(mlp.input_buf()), a);
}
// Thumbs-down at the MLP's CURRENT input with heard action `current_out`
// (empty ⇒ the MLP's live outputs). Runs the full upstream sequence:
// 1. deepen-or-store the negative (dedup radius 0.05); double the
// dislike multiplier (max 16).
// 2. cold start (no positives): train AWAY from the heard action at
// lr * 0.1 * avgRewardNeg (negative LR — upstream fallback).
// 3. else: k-NN(4) positive centroid → push-away target → train toward
// it at lr * negLRRatio, gated by the focus/solo mask.
// 4. proportional decay + eviction of expired negatives; halve the
// multiplier per expiry.
template <typename M>
FeedbackAction dislike_geometric(M& mlp, std::span<const float> current_out,
float lr) noexcept {
auto replay = replay_();
const std::size_t n_out = this->n_out();
std::span<const float> a_neg = current_out.empty()
? std::span<const float>(mlp.outputs())
: current_out;
std::span<const float> x_neg(mlp.input_buf());
// 1. store/deepen the negative (InterfaceRL.cpp:42-66).
replay.deepen_or_store_negative(x_neg, a_neg);
dislike_multiplier_ *= 2u;
if (dislike_multiplier_ > 16u) dislike_multiplier_ = 16u;
const std::size_t pos_total = replay.positive_count();
const std::size_t neg_total = replay.negative_count();
const float avg_neg = replay.avg_negative_reward();
FeedbackAction action;
if (pos_total == 0u) {
// 2. cold-start fallback (InterfaceRL.cpp:746): negative-LR
// training away from the heard action; no geometric push. The
// caller shows the "like a few sounds first" prompt.
mlp.train_targets(x_neg, a_neg, lr * 0.1f * avg_neg, focus_span_());
action = FeedbackAction::GeometricColdStart;
} else {
// 3. centroid → target → train (InterfaceRL.cpp:602-743).
auto mean = this->centroid_buf();
const std::size_t used =
replay.knn_positive_centroid(x_neg, kCentroidK, mean);
auto target = this->target_buf();
compute_push_target(a_neg.subspan(0, (a_neg.size() < n_out) ? a_neg.size() : n_out),
std::span<const float>(mean.data(), n_out),
focus_span_(), geo_push_step(avg_neg), rng_, target);
const float ratio = geo_neg_lr_ratio(neg_total, pos_total);
mlp.train_targets(x_neg, std::span<const float>(target.data(), n_out),
lr * ratio, focus_span_());
(void)used;
action = FeedbackAction::GeometricPush;
}
// 4. decay + evict; halve the multiplier per expiry, reset when no
// negatives remain (InterfaceRL.cpp:752-760).
const std::size_t expired = replay.decay_negatives();
for (std::size_t i = 0; i < expired; ++i) {
dislike_multiplier_ = (dislike_multiplier_ > 1u) ? dislike_multiplier_ / 2u : 1u;
}
if (expired > 0u && replay.negative_count() == 0u) dislike_multiplier_ = 1u;
return action;
}
// =========================================================================
// ExploreAndPlace — granular lifecycle methods (firmware maps buttons to
// these directly; on_down/on_up call them for the browser default policy).
@ -491,6 +626,20 @@ class FeedbackControllerCore : public FbStorage {
}
private:
// The replay view over the storage-owned buffers.
ReplayView replay_() noexcept {
return ReplayView(this->replay_inputs(), this->replay_actions(),
this->replay_rewards(), this->n_in(), this->n_out(),
this->replay_cap(), replay_count_);
}
// The focus mask as the geometric active-dims gate (empty ⇒ all active).
std::span<const std::uint8_t> focus_span_() const noexcept {
if (focus_count_ == 0u) return {};
const auto focus = this->focus();
return focus.subspan(0, focus_count_);
}
void capture_placed(std::span<const float> src) noexcept {
auto placed = this->placed_out();
const std::size_t n = (src.size() < placed.size()) ? src.size() : placed.size();
@ -608,10 +757,16 @@ class FeedbackControllerCore : public FbStorage {
}
FeedbackMode mode_ = FeedbackMode::Avoid;
AvoidStyle avoid_style_ = AvoidStyle::Geometric;
bool explore_active_ = false;
bool learning_paused_ = false;
std::size_t focus_count_ = 0; // 0 ⇒ all active
// ---- Geometric dislike state ---------------------------------------------
std::size_t replay_count_ = 0u;
std::size_t dislike_multiplier_ = 1u;
float geo_lr_ = 0.001f; // upstream InterfaceRL.hpp:312
// ---- ExploreAndPlace state ----------------------------------------------
ExploreState ep_state_ = ExploreState::Idle;
bool last_placed_valid_ = false;
@ -624,8 +779,11 @@ class FeedbackControllerCore : public FbStorage {
// The classic fixed-size controller over a compile-time MLP type — the
// firmware model and the default for tests. Sizes derive from the MLP.
template <typename MLP_T, std::size_t UndoDepth = 4u>
// ReplayCap 32 is the firmware SRAM-budget default (rl-feedback-design §4);
// the browser's DynamicFeedbackStorage uses 64.
template <typename MLP_T, std::size_t UndoDepth = 4u, std::size_t ReplayCap = 32u>
using FeedbackController = FeedbackControllerCore<
FixedFeedbackStorage<MLP_T::kOutput, MLP_T::weight_count(), UndoDepth>>;
FixedFeedbackStorage<MLP_T::kOutput, MLP_T::weight_count(), UndoDepth,
MLP_T::kInput, ReplayCap>>;
} // namespace nisps::ml

95
nisps/ml/geo_push.hpp Normal file
View file

@ -0,0 +1,95 @@
// nisps/ml/geo_push.hpp — geometric push-away target computation for the
// dislike gesture (docs/adr/rl-feedback-design.md §2.1/§4).
//
// Verbatim port of the per-negative target computation in upstream
// InterfaceRL.cpp:713-735 (memllib @ 0a541cc):
//
// pushStep = clamp(|avgRewardNeg|, 0.25, 1.0) * kGeometricPushScale
// dir[j] = neg_action[j] - meanPositiveAction[j]
// len = ||dir||
// useRandom = (len <= 1e-4) (disliked ON the centroid)
// effectivePushStep = pushStep / (1 + len) (taper for far items)
// d = useRandom ? random ∈ [-1,1] : dir[j] / len
// target[j] = clamp(neg_action[j] + d * effectivePushStep, 0, 1)
// inactive dims keep neg_action[j]
//
// SINGLE DELIBERATE FIRMWARE DIVERGENCE (recorded in ALIGNMENT.md): the
// upstream `useRandom` branch draws libc `rand() & 0xFF`; we draw from the
// caller's deterministic per-instance `nisps::Rng` so native == WASM parity
// holds. The branch only fires when a disliked action sits exactly on the
// centroid.
//
// Pure free functions over spans — no replay/centroid logic in the MLP
// kernel, no state, no heap (Anchor-First graft, ADR §0).
#pragma once
#include <cmath>
#include <cstddef>
#include <cstdint>
#include <span>
#include "../core/perf.hpp"
#include "../core/rng.hpp"
namespace nisps::ml {
// Upstream InterfaceRL.hpp:293.
inline constexpr float kGeometricPushScale = 0.5f;
// pushStep from the mean negative reward (InterfaceRL.cpp:713).
inline float geo_push_step(float avg_reward_neg) noexcept {
float mag = (avg_reward_neg < 0.f) ? -avg_reward_neg : avg_reward_neg;
if (mag < 0.25f) mag = 0.25f;
if (mag > 1.0f) mag = 1.0f;
return mag * kGeometricPushScale;
}
// Compute the push-away target for ONE disliked action. `active_mask`
// (1 = active) gates which dims move — empty ⇒ all active (this is the solo/
// focus mask; upstream `activeDims_`). Writes n_out floats into `target`.
inline void compute_push_target(std::span<const float> neg_action,
std::span<const float> mean_positive,
std::span<const std::uint8_t> active_mask,
float push_step,
Rng& rng,
std::span<float> target) noexcept {
const std::size_t n = neg_action.size();
float len_sq = 0.f;
for (std::size_t j = 0; j < n; ++j) {
const float d = neg_action[j] - mean_positive[j];
len_sq += d * d;
}
const float len = std::sqrt(len_sq);
const bool use_random = (len <= 1e-4f);
const float effective = push_step / (1.0f + len);
for (std::size_t j = 0; j < n; ++j) {
const bool active =
active_mask.empty() || (j < active_mask.size() && active_mask[j] != 0u);
if (!active) {
target[j] = neg_action[j];
continue;
}
const float d = use_random
? rng.next_float_signed()
: ((neg_action[j] - mean_positive[j]) / len);
float t = neg_action[j] + d * effective;
if (t < 0.f) t = 0.f;
if (t > 1.f) t = 1.f;
target[j] = t;
}
}
// Dynamic LR ratio (InterfaceRL.cpp:742-743): push harder when dislikes are
// rare, gentler when they flood the buffer.
inline float geo_neg_lr_ratio(std::size_t neg_count, std::size_t pos_count) noexcept {
const std::size_t total = neg_count + pos_count;
const float neg_fraction = (total > 0u)
? static_cast<float>(neg_count) / static_cast<float>(total)
: 0.f;
return 0.5f - 0.4f * neg_fraction;
}
} // namespace nisps::ml

36
nisps/ml/mlp.hpp
View file

@ -209,6 +209,42 @@ class MLPCore : public Storage {
return epoch_loss;
}
// Train ONE step toward a COMPUTED target vector (not a stored label) —
// the geometric-dislike hook (docs/adr/rl-feedback-design.md §4). The
// dataset is untouched. A negative `lr` trains AWAY from the target (the
// upstream cold-start fallback). `out_mask` (1 = active) zeroes the
// loss-derivative of inactive output dims before backprop — the solo/
// focus gate; empty ⇒ all active. Returns the sample loss.
float train_targets(std::span<const float> input,
std::span<const float> target,
float lr,
std::span<const std::uint8_t> out_mask = {}) noexcept {
if (!storage_ok_()) return 0.f;
const std::size_t n_out = this->n_out();
if (input.size() < this->n_in() || target.size() < n_out) return 0.f;
forward_(input);
auto deriv = this->template eval_act_l<3u>();
const float loss = mse_per_sample(
target,
std::span<const float>(this->template act_l<3u>()),
deriv);
if (!out_mask.empty()) {
for (std::size_t j = 0; j < n_out; ++j) {
const bool active = (j < out_mask.size() && out_mask[j] != 0u);
if (!active) deriv[j] = 0.f;
}
}
backprop_(input, deriv, 1.f);
apply_grad_<3u>(lr);
apply_grad_<2u>(lr);
apply_grad_<1u>(lr);
apply_grad_<0u>(lr);
return loss;
}
// ---------------------------------------------------------------
// RL ops (concept: move_weights / draw_weights)
// ---------------------------------------------------------------

235
nisps/ml/replay.hpp Normal file
View file

@ -0,0 +1,235 @@
// nisps/ml/replay.hpp — reward-tagged replay memory algorithms for the
// geometric-dislike feedback mode (docs/adr/rl-feedback-design.md §4).
//
// Ported from upstream InterfaceRL (memllib @ 0a541cc "highlighting"):
// * `_perform_dislike_action()` (InterfaceRL.cpp:42-66) — nearby-negative
// deepening within Euclidean 0.05, else store reward=-1.
// * `optimise()` k-NN positive centroid (InterfaceRL.cpp:602-627).
// * proportional negative decay + eviction (InterfaceRL.cpp:664, :752-760).
//
// STORAGE: the buffers live in the feedback controller's storage policy
// (nisps/ml/feedback.hpp — fixed std::array on firmware, arena slice in the
// browser). `ReplayView` binds those spans plus the live item count and
// carries the algorithms — no ownership, no heap, deterministic.
//
// DETERMINISM (the classic float-sum parity traps, per the ADR):
// * k-NN selection uses a fixed-size insertion into k slots with ties
// broken by LOWER index (no std::sort, no heap).
// * centroid accumulation runs in slot order (nearest first) — a fixed
// summation order so native == WASM bitwise.
// * eviction compacts in place preserving insertion order.
#pragma once
#include <cmath>
#include <cstddef>
#include <span>
#include "../core/perf.hpp"
namespace nisps::ml {
// Upstream constants (InterfaceRL.hpp:293-296, .cpp:42-66,664).
inline constexpr float kReplayDedupRadius = 0.05f;
inline constexpr float kReplayDecayStep = 0.0025f;
inline constexpr float kReplayEvictThreshold = -0.01f;
inline constexpr float kMaxDislikeMagnitude = 16.f;
inline constexpr std::size_t kCentroidK = 4u;
// A non-owning view over the replay buffers (inputs: cap×n_in, actions:
// cap×n_out, rewards: cap) plus the live count. All methods deterministic,
// allocation-free.
class ReplayView {
public:
ReplayView(std::span<float> inputs, std::span<float> actions,
std::span<float> rewards, std::size_t n_in, std::size_t n_out,
std::size_t cap, std::size_t& count) noexcept
: inputs_(inputs), actions_(actions), rewards_(rewards),
n_in_(n_in), n_out_(n_out), cap_(cap), count_(count) {}
std::size_t size() const noexcept { return count_; }
std::size_t capacity() const noexcept { return cap_; }
std::span<const float> input(std::size_t i) const noexcept {
return inputs_.subspan(i * n_in_, n_in_);
}
std::span<const float> action(std::size_t i) const noexcept {
return actions_.subspan(i * n_out_, n_out_);
}
float reward(std::size_t i) const noexcept { return rewards_[i]; }
std::size_t positive_count() const noexcept {
std::size_t n = 0u;
for (std::size_t i = 0; i < count_; ++i) {
if (rewards_[i] > 0.f) ++n;
}
return n;
}
std::size_t negative_count() const noexcept {
std::size_t n = 0u;
for (std::size_t i = 0; i < count_; ++i) {
if (rewards_[i] <= 0.f) ++n;
}
return n;
}
// Mean reward across negatives (≤ 0); 0 when there are none. Fixed
// accumulation order (insertion order).
float avg_negative_reward() const noexcept {
float sum = 0.f;
std::size_t n = 0u;
for (std::size_t i = 0; i < count_; ++i) {
if (rewards_[i] <= 0.f) {
sum += rewards_[i];
++n;
}
}
return (n > 0u) ? (sum / static_cast<float>(n)) : 0.f;
}
// Store an item. When full, the OLDEST item is evicted (shift-down —
// deterministic, preserves relative order).
void store(float reward, std::span<const float> x, std::span<const float> a) noexcept {
std::size_t slot;
if (count_ < cap_) {
slot = count_++;
} else {
evict_(0u);
slot = count_++;
}
write_(slot, reward, x, a);
}
// Upstream `_perform_dislike_action` core: a negative within Euclidean
// `radius` of x has its reward deepened (clamped at -kMaxDislikeMagnitude)
// and its ACTION REFRESHED to the latest heard vector; otherwise a new
// reward=-1 item is stored. Returns true when an existing item deepened.
bool deepen_or_store_negative(std::span<const float> x, std::span<const float> a,
float radius = kReplayDedupRadius) noexcept {
for (std::size_t i = 0; i < count_; ++i) {
if (rewards_[i] < 0.f && distance_(i, x) < radius) {
float r = rewards_[i] - 1.f;
if (r < -kMaxDislikeMagnitude) r = -kMaxDislikeMagnitude;
rewards_[i] = r;
auto act = actions_.subspan(i * n_out_, n_out_);
const std::size_t n = (a.size() < n_out_) ? a.size() : n_out_;
for (std::size_t j = 0; j < n; ++j) act[j] = a[j];
return true;
}
}
store(-1.f, x, a);
return false;
}
// k-NN positive centroid (InterfaceRL.cpp:602-627): mean action of the k
// positives nearest to x. Writes into `mean` (n_out floats) and returns
// the number of positives used (0 ⇒ cold start; `mean` untouched).
// Deterministic: fixed k-slot insertion, ties keep the LOWER index;
// accumulation in slot order.
std::size_t knn_positive_centroid(std::span<const float> x, std::size_t k,
std::span<float> mean) const noexcept {
constexpr std::size_t kMaxK = 8u;
if (k > kMaxK) k = kMaxK;
float best_d[kMaxK];
std::size_t best_i[kMaxK];
std::size_t used = 0u;
for (std::size_t i = 0; i < count_; ++i) {
if (rewards_[i] <= 0.f) continue;
const float d = distance_(i, x);
// Insertion: strictly-less displaces, so equal distances keep the
// earlier (lower-index) item.
std::size_t pos = used;
while (pos > 0u && d < best_d[pos - 1u]) --pos;
if (pos >= k) continue;
const std::size_t tail = (used < k) ? used : (k - 1u);
for (std::size_t m = tail; m > pos; --m) {
best_d[m] = best_d[m - 1u];
best_i[m] = best_i[m - 1u];
}
best_d[pos] = d;
best_i[pos] = i;
if (used < k) ++used;
}
if (used == 0u) return 0u;
for (std::size_t j = 0; j < n_out_; ++j) mean[j] = 0.f;
for (std::size_t s = 0; s < used; ++s) {
const auto act = action(best_i[s]);
for (std::size_t j = 0; j < n_out_; ++j) mean[j] += act[j];
}
const float inv = 1.f / static_cast<float>(used);
for (std::size_t j = 0; j < n_out_; ++j) mean[j] *= inv;
return used;
}
// Proportional decay of every negative (`reward += 0.0025 * max(|r|, 1)`)
// and in-place eviction of items decayed past -0.01. Returns the number
// evicted (the caller halves its dislike multiplier per expiry, matching
// upstream InterfaceRL.cpp:752-760).
std::size_t decay_negatives() noexcept {
std::size_t evicted = 0u;
std::size_t i = 0u;
while (i < count_) {
if (rewards_[i] <= 0.f) {
const float mag = (rewards_[i] < 0.f) ? -rewards_[i] : rewards_[i];
rewards_[i] += kReplayDecayStep * ((mag > 1.f) ? mag : 1.f);
if (rewards_[i] > kReplayEvictThreshold) {
evict_(i);
++evicted;
continue; // same index now holds the next item
}
}
++i;
}
return evicted;
}
void clear() noexcept { count_ = 0u; }
private:
float distance_(std::size_t i, std::span<const float> x) const noexcept {
const auto in = input(i);
const std::size_t n = (x.size() < n_in_) ? x.size() : n_in_;
float acc = 0.f;
for (std::size_t j = 0; j < n; ++j) {
const float d = in[j] - x[j];
acc += d * d;
}
return std::sqrt(acc);
}
void write_(std::size_t slot, float reward, std::span<const float> x,
std::span<const float> a) noexcept {
auto in = inputs_.subspan(slot * n_in_, n_in_);
auto act = actions_.subspan(slot * n_out_, n_out_);
const std::size_t nx = (x.size() < n_in_) ? x.size() : n_in_;
const std::size_t na = (a.size() < n_out_) ? a.size() : n_out_;
for (std::size_t j = 0; j < n_in_; ++j) in[j] = (j < nx) ? x[j] : 0.f;
for (std::size_t j = 0; j < n_out_; ++j) act[j] = (j < na) ? a[j] : 0.f;
rewards_[slot] = reward;
}
// Remove item i, shifting everything after it down one slot.
void evict_(std::size_t i) noexcept {
for (std::size_t m = i + 1u; m < count_; ++m) {
auto dst_in = inputs_.subspan((m - 1u) * n_in_, n_in_);
auto src_in = inputs_.subspan(m * n_in_, n_in_);
for (std::size_t j = 0; j < n_in_; ++j) dst_in[j] = src_in[j];
auto dst_act = actions_.subspan((m - 1u) * n_out_, n_out_);
auto src_act = actions_.subspan(m * n_out_, n_out_);
for (std::size_t j = 0; j < n_out_; ++j) dst_act[j] = src_act[j];
rewards_[m - 1u] = rewards_[m];
}
--count_;
}
std::span<float> inputs_;
std::span<float> actions_;
std::span<float> rewards_;
std::size_t n_in_;
std::size_t n_out_;
std::size_t cap_;
std::size_t& count_;
};
} // namespace nisps::ml

154
nisps/wasm/bindings.cpp
View file

@ -65,7 +65,9 @@
#include "../core/types.hpp"
#include "../ml/dynamic_storage.hpp"
#include "../ml/feedback.hpp"
#include "../ml/jolt.hpp"
#include "../ml/mlp.hpp"
#include "../ml/ou_noise.hpp"
#include "../ml/stats.hpp"
#include "../ml/warm_start.hpp"
@ -89,12 +91,18 @@ constexpr std::size_t kDefaultOutputs = 126u;
constexpr std::size_t kMaxDim = 4096u;
constexpr std::uint64_t kFeedbackSalt = 0xFEEDBACC0DEull;
// Distinct salts keep the jolt/OU RNG streams independent of the MLP's and
// the feedback controller's (mirrors the firmware ModeBase seeding).
constexpr std::uint64_t kJoltSalt = 0xB01DFACEull;
constexpr std::uint64_t kOUSalt = 0x0DDBA11ull;
using BrowserMLP = nisps::ml::MLPCore<nisps::ml::DynamicStorage>;
using BrowserFeedback =
nisps::ml::FeedbackControllerCore<nisps::ml::DynamicFeedbackStorage>;
constexpr std::size_t kFeedbackUndoDepth = 4u;
// Browser replay capacity (rl-feedback-design §4: WASM 64, firmware 32).
constexpr std::size_t kFeedbackReplayCap = 64u;
struct MlDims {
std::size_t n_in;
@ -145,15 +153,26 @@ struct MLHandle {
std::array<float, BrowserMLP::kNumLayers * 4u> stats_scratch{};
// Static-output buffer for the RandomiseOutputs bypass path.
std::vector<float> feedback_static_scratch;
// Jolt (held weight morph) + OU exploration noise — the P3 gesture
// engines, same code the firmware ModeBase runs. Jolt operates on the
// flat weight buffer via jolt_scratch; OU state is over-provisioned to
// kMaxDim and applies to the first n_out entries.
nisps::ml::Jolt jolt;
nisps::ml::OUNoise<kMaxDim> ou;
std::vector<float> jolt_scratch;
// infer_batch cap; callers must split larger requests.
static constexpr std::size_t kMaxBatch = 4096u;
MLHandle(std::uint64_t seed, const MlDims& d) noexcept
: seed64(seed),
mlp(seed, d.n_in, std::span<const std::size_t>(d.hidden, 3u), d.n_out),
feedback(seed ^ kFeedbackSalt, d.n_out, mlp.weight_count(), kFeedbackUndoDepth),
feedback(seed ^ kFeedbackSalt, d.n_out, mlp.weight_count(), kFeedbackUndoDepth,
d.n_in, kFeedbackReplayCap),
output_scratch(d.n_out, 0.f),
feedback_static_scratch(d.n_out, 0.f) {}
feedback_static_scratch(d.n_out, 0.f),
jolt(seed ^ kJoltSalt),
ou(seed ^ kOUSalt),
jolt_scratch(mlp.weight_count(), 0.f) {}
bool valid() const noexcept { return mlp.valid() && feedback.valid(); }
std::size_t n_in() const noexcept { return mlp.n_in(); }
@ -372,13 +391,16 @@ int nisps_ml_reshape(void* ml, int input_size, int output_size,
nisps::ml::warm_start_copy(fresh, h->mlp);
BrowserFeedback fb(h->seed64 ^ kFeedbackSalt, d.n_out, fresh.weight_count(),
kFeedbackUndoDepth);
kFeedbackUndoDepth, d.n_in, kFeedbackReplayCap);
if (!fb.valid()) return 0;
h->mlp = static_cast<BrowserMLP&&>(fresh);
h->feedback = static_cast<BrowserFeedback&&>(fb);
h->output_scratch.assign(d.n_out, 0.f);
h->feedback_static_scratch.assign(d.n_out, 0.f);
h->jolt.release();
h->ou.reset();
h->jolt_scratch.assign(h->mlp.weight_count(), 0.f);
return 1;
}
@ -737,6 +759,132 @@ int nisps_ml_feedback_placed_output(void* ml, float* out) {
return 1;
}
// ---------------------------------------------------------------------------
// ML feedback — geometric dislike (one-core-engine P3; rl-feedback-design
// §2.1). The Avoid mode's default realisation. current_out may be null (the
// MLP's live output is used — note the zero-derivative caveat: pass the
// HEARD post-pipeline vector for an audible push). lr <= 0 uses the
// controller default (1e-3, upstream).
// ---------------------------------------------------------------------------
EMSCRIPTEN_KEEPALIVE
int nisps_ml_feedback_dislike_geometric(void* ml, const float* current_out, float lr) {
if (!ml) return 0;
auto* h = static_cast<MLHandle*>(ml);
std::span<const float> out;
if (current_out) out = std::span<const float>(current_out, h->n_out());
const float use_lr = (lr > 0.f) ? lr : h->feedback.geo_lr();
return static_cast<int>(h->feedback.dislike_geometric(h->mlp, out, use_lr));
}
// Store a positive (like) into the replay memory so the k-NN centroid sees
// it. current_out may be null (live output used). The caller still runs its
// usual addExample + train.
EMSCRIPTEN_KEEPALIVE
void nisps_ml_feedback_store_positive(void* ml, const float* current_out) {
if (!ml) return;
auto* h = static_cast<MLHandle*>(ml);
std::span<const float> out;
if (current_out) out = std::span<const float>(current_out, h->n_out());
h->feedback.store_positive(h->mlp, out);
}
EMSCRIPTEN_KEEPALIVE
int nisps_ml_feedback_positive_count(void* ml) {
if (!ml) return 0;
return static_cast<int>(static_cast<MLHandle*>(ml)->feedback.positive_count());
}
EMSCRIPTEN_KEEPALIVE
int nisps_ml_feedback_negative_count(void* ml) {
if (!ml) return 0;
return static_cast<int>(static_cast<MLHandle*>(ml)->feedback.negative_count());
}
// Avoid sub-mode: 0 = Geometric (default), 1 = Diffuse (legacy move_weights,
// kept for A/B comparison).
EMSCRIPTEN_KEEPALIVE
void nisps_ml_feedback_set_avoid_style(void* ml, int style) {
if (!ml) return;
static_cast<MLHandle*>(ml)->feedback.set_avoid_style(
style == 1 ? nisps::ml::AvoidStyle::Diffuse : nisps::ml::AvoidStyle::Geometric);
}
// ---------------------------------------------------------------------------
// Jolt (held weight morph) + OU exploration noise (one-core-engine P3.2) —
// the same nisps/ml/{jolt,ou_noise}.hpp the firmware ModeBase runs.
// ---------------------------------------------------------------------------
EMSCRIPTEN_KEEPALIVE
void nisps_ml_jolt_press(void* ml) {
if (!ml) return;
auto* h = static_cast<MLHandle*>(ml);
h->jolt.press(h->mlp.weight_count());
}
// One ~200 Hz morph tick while held (no-op when inactive): reads the flat
// weights, glides the jolt-selected few, writes them back.
EMSCRIPTEN_KEEPALIVE
void nisps_ml_jolt_step(void* ml) {
if (!ml) return;
auto* h = static_cast<MLHandle*>(ml);
if (!h->jolt.active()) return;
auto w = h->mlp.get_weights();
for (std::size_t i = 0; i < w.size(); ++i) h->jolt_scratch[i] = w[i];
h->jolt.step(std::span<float>(h->jolt_scratch.data(), w.size()));
h->mlp.set_weights(std::span<const float>(h->jolt_scratch.data(), w.size()));
}
EMSCRIPTEN_KEEPALIVE
void nisps_ml_jolt_release(void* ml) {
if (!ml) return;
static_cast<MLHandle*>(ml)->jolt.release();
}
EMSCRIPTEN_KEEPALIVE
int nisps_ml_jolt_active(void* ml) {
if (!ml) return 0;
return static_cast<MLHandle*>(ml)->jolt.active() ? 1 : 0;
}
// Post-release learning-rate ramp: multiply the training LR by this (0 while
// held, ramps back to 1 over ~5 s of ticks).
EMSCRIPTEN_KEEPALIVE
float nisps_ml_jolt_lr_scale(void* ml) {
if (!ml) return 1.f;
return static_cast<MLHandle*>(ml)->jolt.lr_scale();
}
EMSCRIPTEN_KEEPALIVE
void nisps_ml_jolt_tick_lr_ramp(void* ml) {
if (!ml) return;
static_cast<MLHandle*>(ml)->jolt.tick_lr_ramp();
}
// Exploration amount in [0,1]; 0 disables (inert — parity-safe).
EMSCRIPTEN_KEEPALIVE
void nisps_ml_explore_intensity(void* ml, float level) {
if (!ml) return;
static_cast<MLHandle*>(ml)->ou.set_intensity(level);
}
EMSCRIPTEN_KEEPALIVE
float nisps_ml_explore_get_intensity(void* ml) {
if (!ml) return 0.f;
return static_cast<MLHandle*>(ml)->ou.intensity();
}
// Advance the OU walk and add it (clamped to [0,1]) to `inout` (n floats,
// capped at the instance's n_out). No-op at intensity 0.
EMSCRIPTEN_KEEPALIVE
void nisps_ml_explore_apply(void* ml, float* inout, int n) {
if (!ml || !inout || n <= 0) return;
auto* h = static_cast<MLHandle*>(ml);
std::size_t count = static_cast<std::size_t>(n);
if (count > h->n_out()) count = h->n_out();
h->ou.apply(std::span<float>(inout, count));
}
EMSCRIPTEN_KEEPALIVE
void nisps_ml_get_layer_stats(void* ml, float* out_stats) {
if (!ml || !out_stats) return;

6
scripts/build-wasm.sh
View file

@ -47,6 +47,12 @@ EXPORTED_FUNCS='[
"_nisps_ml_feedback_like","_nisps_ml_feedback_commit_place","_nisps_ml_feedback_cancel_place",
"_nisps_ml_feedback_placing","_nisps_ml_feedback_state","_nisps_ml_feedback_undo_depth",
"_nisps_ml_feedback_placed_output",
"_nisps_ml_feedback_dislike_geometric","_nisps_ml_feedback_store_positive",
"_nisps_ml_feedback_positive_count","_nisps_ml_feedback_negative_count",
"_nisps_ml_feedback_set_avoid_style",
"_nisps_ml_jolt_press","_nisps_ml_jolt_step","_nisps_ml_jolt_release",
"_nisps_ml_jolt_active","_nisps_ml_jolt_lr_scale","_nisps_ml_jolt_tick_lr_ramp",
"_nisps_ml_explore_intensity","_nisps_ml_explore_get_intensity","_nisps_ml_explore_apply",
"_nisps_ml_get_layer_stats","_nisps_ml_describe",
"_nisps_engine_create","_nisps_engine_destroy",
"_nisps_engine_set_params","_nisps_engine_process_block"

54
tests/cpp/parity_check.cpp
View file

@ -83,7 +83,7 @@ constexpr std::array<std::size_t, 12u> kProbeIdx = {
};
constexpr std::uint32_t kMagic = 0x5450524Eu; // 'NPRT'
constexpr std::uint32_t kVersion = 3u; // v3 adds stage 5d (ExploreAndPlace lifecycle)
constexpr std::uint32_t kVersion = 4u; // v4 adds stage 6 (geometric dislike)
// Must match the salt in nisps/wasm/bindings.cpp MLHandle so the controller's
// static-output RNG stream is identical native ↔ WASM.
@ -270,6 +270,58 @@ int main(int argc, char** argv) {
const auto v = fb.committed_output();
for (std::size_t i = 0; i < 126u; ++i) payload.push_back(i < v.size() ? v[i] : 0.f);
}
// ---- Stage 6: geometric dislike (one-core-engine P3) ----
// Scripted feedback session — likes at two corners feed the replay
// positives (via the Avoid+Geometric on_up path), then two dislikes at
// a probed input: the first stores the negative and trains toward the
// computed push-away target; the second deepens and pushes again. The
// weight trajectory must match native↔WASM within 1e-5 (the useRandom
// branch never fires here; the controller Rng is untouched).
fb.set_mode(nisps::ml::FeedbackMode::Avoid, mlp);
auto like_at = [&](float x, float y) {
mlp.set_input(0u, x);
mlp.set_input(1u, y);
mlp.process();
fb.on_up(mlp); // Avoid+Geometric: store_positive + LikeStore
};
like_at(0.2f, 0.2f);
like_at(0.8f, 0.8f);
auto dislike_at = [&](float x, float y) {
mlp.set_input(0u, x);
mlp.set_input(1u, y);
mlp.process();
// The "heard" vector deliberately differs from the raw output
// (the browser passes the post-pipeline vector) so the push
// trains meaningfully. f32 arithmetic mirrored in parity_wasm.mjs
// via Math.fround.
std::array<float, 126u> heard{};
const auto outs = mlp.outputs();
for (std::size_t j = 0; j < 126u; ++j) {
float v = outs[j] + (((j & 1u) != 0u) ? -0.15f : 0.15f);
if (v < 0.f) v = 0.f;
if (v > 1.f) v = 1.f;
heard[j] = v;
}
fb.on_down(mlp, heard, 0.1f, 0.5f, no_mask);
};
dislike_at(0.25f, 0.75f);
dislike_at(0.26f, 0.74f); // within dedup radius → deepen + push
payload.push_back(static_cast<float>(fb.positive_count()));
payload.push_back(static_cast<float>(fb.negative_count()));
mlp.set_input(0u, kInputX);
mlp.set_input(1u, kInputY);
mlp.process();
{
const auto outs = mlp.outputs();
push_floats(payload, std::span<const float>(outs.data(), 126u));
const auto w = mlp.get_weights();
for (std::size_t idx : kProbeIdx) payload.push_back(idx < w.size() ? w[idx] : 0.f);
}
}
// ---- Sanity: every value finite ----

2
tests/cpp/parity_diff.mjs
View file

@ -16,7 +16,7 @@
import { readFile } from 'node:fs/promises';
const MAGIC = 0x5450524e;
const VERSION = 3;
const VERSION = 4; // v4 adds stage 6 (geometric dislike)
const DEFAULT_TOL = 1e-5;
// Layout for context in error messages — must match parity_check.cpp / parity_wasm.mjs.

55
tests/cpp/parity_wasm.mjs
View file

@ -38,7 +38,7 @@ const __dirname = dirname(__filename);
const repoRoot = resolve(__dirname, '..', '..');
const MAGIC = 0x5450524e; // 'NPRT'
const VERSION = 3; // v3 adds stage 5d (ExploreAndPlace lifecycle)
const VERSION = 4; // v4 adds stage 6 (geometric dislike)
const SEED = 42 >>> 0;
const INPUT_X = 0.25;
@ -112,6 +112,8 @@ function bind(Module) {
feedbackLike: cwrap('nisps_ml_feedback_like', null, ['number']),
feedbackCommitPlace: cwrap('nisps_ml_feedback_commit_place', null, ['number']),
feedbackPlacedOutput: cwrap('nisps_ml_feedback_placed_output', 'number', ['number','number']),
feedbackPositiveCount: cwrap('nisps_ml_feedback_positive_count', 'number', ['number']),
feedbackNegativeCount: cwrap('nisps_ml_feedback_negative_count', 'number', ['number']),
describe: cwrap('nisps_ml_describe', null, ['number','number']),
engineCreate: cwrap('nisps_engine_create', 'number', ['string','number']),
@ -324,6 +326,57 @@ async function main() {
api.free(committedBuf);
}
// --- Stage 6: geometric dislike (one-core-engine P3) ---
// Mirrors parity_check.cpp stage 6: two likes feed the replay positives via
// the Avoid+Geometric on_up path, then two dislikes (second deepens) train
// toward the computed push-away target. f32 arithmetic for the "heard"
// vector via Math.fround to match native float ops exactly.
const FB_AVOID = 0;
api.feedbackSetMode(ml, FB_AVOID);
const likeAt = (x, y) => {
api.setInput(ml, 0, x);
api.setInput(ml, 1, y);
api.process(ml);
api.feedbackUp(ml); // Avoid+Geometric: store_positive + LikeStore
};
likeAt(0.2, 0.2);
likeAt(0.8, 0.8);
const POS_DELTA = Math.fround(0.15);
const dislikeAt = (x, y) => {
api.setInput(ml, 0, x);
api.setInput(ml, 1, y);
api.process(ml);
const outs = getOutputsCopy(api, ml, N_OUT);
const heard = new Float32Array(N_OUT);
for (let j = 0; j < N_OUT; j++) {
let v = Math.fround(outs[j] + ((j & 1) !== 0 ? -POS_DELTA : POS_DELTA));
if (v < 0) v = 0;
if (v > 1) v = 1;
heard[j] = v;
}
const heardBuf = api.malloc(N_OUT * 4);
new Float32Array(api.HEAPF32.buffer, heardBuf, N_OUT).set(heard);
api.feedbackDown(ml, heardBuf, 0.1, 0.5, 0);
api.free(heardBuf);
};
dislikeAt(0.25, 0.75);
dislikeAt(0.26, 0.74); // within dedup radius: deepen + push
feedbackFloats.push(api.feedbackPositiveCount(ml));
feedbackFloats.push(api.feedbackNegativeCount(ml));
api.setInput(ml, 0, INPUT_X);
api.setInput(ml, 1, INPUT_Y);
api.process(ml);
{
const outs = getOutputsCopy(api, ml, N_OUT);
for (const v of outs) feedbackFloats.push(v);
const w = getWeightsCopy(api, ml);
for (const idx of PROBE_IDX) feedbackFloats.push(idx < w.length ? w[idx] : 0);
}
api.destroy(ml);
// --- Build payload, write blob ---

4
tests/cpp/test_mlp_feedback.cpp
View file

@ -57,6 +57,10 @@ NISPS_TEST(feedback_avoid_routes_to_move_weights) {
SmallMLP m(99ull);
m.draw_weights(0.5f);
FB fb(7ull); // default mode is Avoid
// Since one-core-engine P3 the Avoid default is the GEOMETRIC push
// (rl-feedback-design §2.1); the undirected move_weights survives as the
// legacy Diffuse sub-mode, pinned here.
fb.set_avoid_style(nisps::ml::AvoidStyle::Diffuse);
const auto before = snapshot_weights(m);
const FeedbackAction a = fb.on_down(m, kNoOut, 0.1f, 0.5f, kNoMask);

289
tests/cpp/test_mlp_geo_dislike.cpp Normal file
View file

@ -0,0 +1,289 @@
// tests/cpp/test_mlp_geo_dislike.cpp — geometric dislike (one-core-engine P3;
// docs/adr/rl-feedback-design.md §2.1/§4/§6.1).
//
// Covers: replay dedup/deepen at radius 0.05, k-NN centroid selection with
// deterministic index tie-break, push direction sign (target moves AWAY from
// the liked centroid), taper, cold-start posMemCount==0 fallback, decay/
// eviction + dislike-multiplier bookkeeping, solo/focus gating, and fixed-seed
// determinism.
#include <array>
#include <cmath>
#include <cstdint>
#include <span>
#include "../../nisps/ml/feedback.hpp"
#include "../../nisps/ml/geo_push.hpp"
#include "../../nisps/ml/mlp.hpp"
#include "../../nisps/ml/replay.hpp"
#include "test_helpers.hpp"
namespace {
using nisps::ml::AvoidStyle;
using nisps::ml::FeedbackAction;
using nisps::ml::FeedbackMode;
using nisps::ml::ReplayView;
using GeoMLP = nisps::ml::MLP<2u, 4u, 4u, 4u, 6u, 8u, 32u>;
using GeoFB = nisps::ml::FeedbackController<GeoMLP, 2u, 16u>;
constexpr std::size_t kNIn = 2u;
constexpr std::size_t kNOut = 6u;
constexpr std::size_t kCap = 16u;
struct RawReplay {
std::array<float, kCap * kNIn> in{};
std::array<float, kCap * kNOut> act{};
std::array<float, kCap> rew{};
std::size_t count = 0u;
ReplayView view() {
return ReplayView(in, act, rew, kNIn, kNOut, kCap, count);
}
};
void set_inputs(GeoMLP& m, float x, float y) {
m.set_input(0u, x);
m.set_input(1u, y);
}
} // namespace
// -- ReplayView primitives ----------------------------------------------------
NISPS_TEST(replay_deepen_within_radius_else_store) {
RawReplay raw;
auto r = raw.view();
const float x1[kNIn] = {0.5f, 0.5f};
const float x2[kNIn] = {0.52f, 0.52f}; // within 0.05 of x1
const float x3[kNIn] = {0.9f, 0.9f}; // far away
const float a[kNOut] = {0.1f, 0.2f, 0.3f, 0.4f, 0.5f, 0.6f};
const float a2[kNOut] = {0.9f, 0.8f, 0.7f, 0.6f, 0.5f, 0.4f};
NISPS_EXPECT(!r.deepen_or_store_negative(std::span<const float>(x1), std::span<const float>(a)));
NISPS_ASSERT(r.size() == 1u);
NISPS_EXPECT(r.reward(0) == -1.f);
// Nearby dislike deepens (reward -2) and refreshes the action.
NISPS_EXPECT(r.deepen_or_store_negative(std::span<const float>(x2), std::span<const float>(a2)));
NISPS_ASSERT(r.size() == 1u);
NISPS_EXPECT(r.reward(0) == -2.f);
NISPS_EXPECT(r.action(0)[0] == 0.9f);
// Far dislike stores a new item.
NISPS_EXPECT(!r.deepen_or_store_negative(std::span<const float>(x3), std::span<const float>(a)));
NISPS_ASSERT(r.size() == 2u);
// Deepening clamps at -16.
for (int i = 0; i < 40; ++i) {
r.deepen_or_store_negative(std::span<const float>(x1), std::span<const float>(a));
}
NISPS_EXPECT(r.reward(0) == -16.f);
}
NISPS_TEST(replay_knn_centroid_deterministic_tie_break) {
RawReplay raw;
auto r = raw.view();
const float probe[kNIn] = {0.5f, 0.5f};
// Two positives EQUIDISTANT from the probe, distinct actions; k=1 must
// pick the LOWER index deterministically.
const float pa[kNIn] = {0.4f, 0.5f};
const float pb[kNIn] = {0.6f, 0.5f};
float aa[kNOut]; for (std::size_t j = 0; j < kNOut; ++j) aa[j] = 0.2f;
float ab[kNOut]; for (std::size_t j = 0; j < kNOut; ++j) ab[j] = 0.8f;
r.store(1.f, std::span<const float>(pa), std::span<const float>(aa));
r.store(1.f, std::span<const float>(pb), std::span<const float>(ab));
std::array<float, kNOut> mean{};
const std::size_t used = r.knn_positive_centroid(std::span<const float>(probe), 1u, mean);
NISPS_ASSERT(used == 1u);
NISPS_EXPECT(mean[0] == 0.2f); // index 0 wins the tie
// k=4 with 3 positives → uses all 3; centroid is their mean.
const float pc[kNIn] = {0.5f, 0.6f};
float ac[kNOut]; for (std::size_t j = 0; j < kNOut; ++j) ac[j] = 0.5f;
r.store(1.f, std::span<const float>(pc), std::span<const float>(ac));
const std::size_t used4 = r.knn_positive_centroid(std::span<const float>(probe), 4u, mean);
NISPS_ASSERT(used4 == 3u);
NISPS_EXPECT_NEAR(mean[0], (0.2f + 0.8f + 0.5f) / 3.f, 1e-6);
// Negatives never contribute.
const float nx[kNIn] = {0.5f, 0.5f};
float na[kNOut]; for (std::size_t j = 0; j < kNOut; ++j) na[j] = 0.0f;
r.store(-1.f, std::span<const float>(nx), std::span<const float>(na));
const std::size_t used_after_neg =
r.knn_positive_centroid(std::span<const float>(probe), 4u, mean);
NISPS_EXPECT(used_after_neg == 3u);
}
NISPS_TEST(replay_decay_and_evict) {
RawReplay raw;
auto r = raw.view();
const float x[kNIn] = {0.1f, 0.1f};
const float a[kNOut] = {};
// A shallow negative just above the evict threshold decays out in a few
// calls; rewards move by +0.0025*max(|r|,1) per call.
r.store(-0.012f, std::span<const float>(x), std::span<const float>(a));
NISPS_ASSERT(r.size() == 1u);
std::size_t evicted = r.decay_negatives(); // -0.012 + 0.0025 = -0.0095 > -0.01 → evict
NISPS_EXPECT(evicted == 1u);
NISPS_EXPECT(r.size() == 0u);
// Positives are never decayed/evicted.
r.store(1.f, std::span<const float>(x), std::span<const float>(a));
evicted = r.decay_negatives();
NISPS_EXPECT(evicted == 0u);
NISPS_EXPECT(r.size() == 1u);
}
// -- compute_push_target --------------------------------------------------------
NISPS_TEST(geo_push_target_moves_away_from_centroid) {
nisps::Rng rng(7ull);
std::array<float, kNOut> neg{};
std::array<float, kNOut> mean{};
std::array<float, kNOut> target{};
for (std::size_t j = 0; j < kNOut; ++j) {
neg[j] = 0.6f;
mean[j] = 0.4f; // dir = +0.2 per dim → push increases values
}
const float step = nisps::ml::geo_push_step(-1.f); // clamp(1,0.25,1)*0.5 = 0.5
NISPS_EXPECT_NEAR(step, 0.5f, 1e-7);
nisps::ml::compute_push_target(neg, mean, {}, step, rng, target);
for (std::size_t j = 0; j < kNOut; ++j) {
NISPS_EXPECT(target[j] > neg[j]); // strictly away from the centroid
NISPS_EXPECT(target[j] <= 1.f);
}
// Taper: a far-away negative moves LESS than a near one for the same step.
std::array<float, kNOut> mean_far{};
std::array<float, kNOut> target_far{};
for (std::size_t j = 0; j < kNOut; ++j) mean_far[j] = 0.0f; // larger len
nisps::ml::compute_push_target(neg, mean_far, {}, step, rng, target_far);
NISPS_EXPECT((target_far[0] - neg[0]) < (target[0] - neg[0]));
}
NISPS_TEST(geo_push_respects_active_mask_and_clamps) {
nisps::Rng rng(7ull);
std::array<float, kNOut> neg{};
std::array<float, kNOut> mean{};
std::array<float, kNOut> target{};
for (std::size_t j = 0; j < kNOut; ++j) {
neg[j] = 0.99f;
mean[j] = 0.01f;
}
std::array<std::uint8_t, kNOut> mask{1u, 0u, 1u, 0u, 1u, 0u};
nisps::ml::compute_push_target(neg, mean, mask, 0.5f, rng, target);
for (std::size_t j = 0; j < kNOut; ++j) {
if (mask[j]) {
NISPS_EXPECT(target[j] >= neg[j]); // pushed (and clamped at 1)
NISPS_EXPECT(target[j] <= 1.f);
} else {
NISPS_EXPECT(target[j] == neg[j]); // frozen dim untouched
}
}
}
// -- controller end-to-end ------------------------------------------------------
NISPS_TEST(geo_dislike_cold_start_then_push) {
GeoMLP m(42ull);
m.draw_weights(0.5f);
GeoFB fb(42ull ^ 0xFEEDBACC0DEull);
NISPS_ASSERT(fb.avoid_style() == AvoidStyle::Geometric); // the P3 default
set_inputs(m, 0.25f, 0.75f);
m.process();
// Cold start: no positives yet → negative-LR fallback.
auto before = m.get_weights();
std::array<float, GeoMLP::weight_count()> snap{};
for (std::size_t i = 0; i < snap.size(); ++i) snap[i] = before[i];
// With the heard action == the net's own output the MSE derivative is
// zero, so the fallback is INERT — the conservative cold start the ADR
// mandates (never destabilise before any positives exist).
const FeedbackAction a1 = fb.on_down(m, {}, 0.1f, 0.5f, {});
NISPS_EXPECT(a1 == FeedbackAction::GeometricColdStart);
NISPS_EXPECT(fb.negative_count() == 1u);
NISPS_EXPECT(fb.dislike_multiplier() == 2u);
{
auto after = m.get_weights();
for (std::size_t i = 0; i < snap.size(); ++i) {
NISPS_ASSERT(after[i] == snap[i]);
}
}
// When the HEARD action differs from the net's raw output (the real
// browser/firmware case — the user hears the post-pipeline vector), the
// negative-LR fallback trains AWAY from it: weights move.
std::array<float, kNOut> heard{};
{
auto outs = m.outputs();
for (std::size_t j = 0; j < kNOut; ++j) {
heard[j] = (outs[j] < 0.5f) ? outs[j] + 0.2f : outs[j] - 0.2f;
}
}
const FeedbackAction a1b = fb.on_down(m, heard, 0.1f, 0.5f, {});
NISPS_EXPECT(a1b == FeedbackAction::GeometricColdStart);
bool moved = false;
{
auto after = m.get_weights();
for (std::size_t i = 0; i < snap.size(); ++i) {
if (after[i] != snap[i]) { moved = true; break; }
}
}
NISPS_EXPECT(moved);
// Feed positives via the like path, then dislike → geometric push.
set_inputs(m, 0.2f, 0.2f);
m.process();
NISPS_EXPECT(fb.on_up(m) == FeedbackAction::LikeStore);
set_inputs(m, 0.8f, 0.8f);
m.process();
NISPS_EXPECT(fb.on_up(m) == FeedbackAction::LikeStore);
NISPS_EXPECT(fb.positive_count() == 2u);
set_inputs(m, 0.25f, 0.75f);
m.process();
const FeedbackAction a2 = fb.on_down(m, {}, 0.1f, 0.5f, {});
NISPS_EXPECT(a2 == FeedbackAction::GeometricPush);
NISPS_EXPECT(!fb.exploring());
NISPS_EXPECT(!fb.learning_paused());
}
NISPS_TEST(geo_dislike_deterministic_under_fixed_seed) {
auto run = [](std::span<float> out_weights) {
GeoMLP m(123ull);
m.draw_weights(0.6f);
GeoFB fb(456ull);
set_inputs(m, 0.3f, 0.3f);
m.process();
fb.on_up(m); // positive
set_inputs(m, 0.31f, 0.31f);
m.process();
fb.on_down(m, {}, 0.1f, 0.5f, {}); // geometric push
fb.on_down(m, {}, 0.1f, 0.5f, {}); // deepen + push again
auto w = m.get_weights();
for (std::size_t i = 0; i < w.size(); ++i) out_weights[i] = w[i];
};
std::array<float, GeoMLP::weight_count()> w1{}, w2{};
run(w1);
run(w2);
for (std::size_t i = 0; i < w1.size(); ++i) {
NISPS_ASSERT(w1[i] == w2[i]);
}
}
NISPS_TEST(geo_dislike_diffuse_style_preserves_legacy_path) {
GeoMLP m(9ull);
GeoFB fb(9ull);
fb.set_avoid_style(AvoidStyle::Diffuse);
set_inputs(m, 0.5f, 0.5f);
m.process();
const FeedbackAction a = fb.on_down(m, {}, 0.1f, 0.5f, {});
NISPS_EXPECT(a == FeedbackAction::AvoidPerturb);
NISPS_EXPECT(fb.replay_size() == 0u); // diffuse touches no replay
}