825ed6ad33
Stream 2 of the clean-slate rewrite: nisps/ml/ replaces src/memlp/ with a header-only, heap-free MLP that satisfies nisps::core::MLEngine. Files (nisps/ml/): - activations.hpp — ReLU (leaky 0.01 for parity), sigmoid, tanh - loss.hpp — MSE per-sample (fixes meml-ues double-scaling: returns the sample's MSE without an extra 1/N multiplication; the training loop averages explicitly) - init.hpp — uniform/Xavier/spread-aware weight init - training.hpp — gradient clip helper (±10.0 matches legacy) - rl.hpp — move_weights with per-layer Xavier scaling, weight decay (10% * spread), gaussian noise via the deterministic Rng (matches the legacy JS sum-of-three-uniforms shape); draw_weights also spread-aware - stats.hpp — per-layer mean/max/dead/saturating diagnostics - mlp.hpp — 4-layer (3 hidden + sigmoid output) MLP class with std::array-backed weights, biases, gradient accumulators, dataset ring buffer (default 128 examples), loss history (default 4096 iters). Bias is a separate per-layer parameter — no input-vector mutation. Flat get_weights/set_weights layout: weights all layers (row-major, layer order), then biases all layers. Tests (tests/cpp/, all 50 passing under -Wall -Wextra -Werror -Wpedantic): - test_mlp_init.cpp — deterministic seeding, spread regimes, static_assert MLEngine concept satisfied - test_mlp_inference.cpp — golden hand-computed forward pass match, sigmoid output range, set_input bounds - test_mlp_training.cpp — XOR convergence (loss < 0.01 in <2k iters), ring-buffer eviction - test_mlp_loss.cpp — meml-ues regression test: reported loss equals hand-computed average MSE without extra 1/N scaling; sample weights honoured - test_mlp_rl.cpp — move_weights respects output_pin_mask (final-layer rows + biases preserved); spread regimes; grad clear after draw_weights - test_mlp_serialize.cpp — get_weights/set_weights round-trip preserves inference exactly; eval_loss is non-mutating; infer_batch matches individual inference Verification: - Clean build, no warnings - 50 tests pass (22 prior + 28 new) - No std::vector / new / malloc in nisps/ml/ - All float literals .f-suffixed in code (comments excepted)
112 lines
4.9 KiB
C++
112 lines
4.9 KiB
C++
// nisps/ml/rl.hpp — reinforcement-style weight perturbation primitives.
|
|
//
|
|
// `move_weights` and `draw_weights` are the two playground RL operations
|
|
// applied to network weights:
|
|
// - `draw_weights(spread)`: re-randomize all weights using the spread-
|
|
// aware scale (see init.hpp). This is the "thumbs-down really hard"
|
|
// button.
|
|
// - `move_weights(speed, spread, output_pin_mask)`: per-call weight decay
|
|
// followed by gaussian noise injection. This is the "thumbs-down a
|
|
// little" feedback. The pin mask, if supplied, freezes weights feeding
|
|
// specific OUTPUT nodes — only on the FINAL layer.
|
|
//
|
|
// PARITY CONTRACT WITH LEGACY JS PLAYGROUND
|
|
// These match `playground/_archive/js/nisps/mlp.js::moveWeights` and
|
|
// `drawWeights` semantically. Specifically:
|
|
// * Per-layer scale: layer_scale = (1 - spread) + spread / sqrt(fan_in)
|
|
// * Weight decay: each weight gets multiplied by (1 - 0.1 * spread)
|
|
// BEFORE noise is added. spread=0 ⇒ no decay,
|
|
// spread=1 ⇒ ~10% decay per call.
|
|
// * Noise: gaussian via rng.next_float_gaussian(speed *
|
|
// layer_scale). The legacy code uses a sum-of-three-
|
|
// uniforms gaussian, and our `Rng` matches that
|
|
// shape (see core/rng.hpp).
|
|
// * Output pin mask: 1 byte per output node. If mask[i]==1, all weights
|
|
// AND the bias feeding output node i in the final
|
|
// layer are skipped. Hidden layers are unaffected by
|
|
// the mask.
|
|
// * Biases: included in the noise injection, unaffected by
|
|
// decay (matching the legacy JS behavior — decay was
|
|
// applied to weights only there).
|
|
//
|
|
// The functions are layer-scoped; the MLP class (mlp.hpp) iterates through
|
|
// its layers, calling these for each.
|
|
|
|
#pragma once
|
|
|
|
#include <cmath>
|
|
#include <cstddef>
|
|
#include <cstdint>
|
|
#include <span>
|
|
|
|
#include "../core/rng.hpp"
|
|
#include "init.hpp"
|
|
|
|
namespace nisps::ml {
|
|
|
|
// Per-call weight-decay coefficient. spread=0 ⇒ 1.0 (no decay), spread=1 ⇒
|
|
// 0.9 (10% decay per call). See architecture spread §3 in CLAUDE.md.
|
|
inline float weight_decay_factor(float spread) noexcept {
|
|
return 1.f - 0.1f * spread;
|
|
}
|
|
|
|
// Apply RL perturbation to one layer's weights+biases. `is_final_layer`
|
|
// gates the application of `output_pin_mask`. `output_pin_mask`, if
|
|
// non-empty, must be sized fan_out.
|
|
//
|
|
// Effects, applied in order per weight w:
|
|
// w *= (1 - 0.1 * spread)
|
|
// w += gaussian(speed * layer_scale)
|
|
// Biases:
|
|
// b += gaussian(speed * layer_scale)
|
|
// Skip rule on final layer if mask[node]==1: leave w/b untouched.
|
|
inline void move_weights_layer(std::span<float> weights,
|
|
std::span<float> biases,
|
|
std::size_t fan_in,
|
|
float speed,
|
|
float spread,
|
|
bool is_final_layer,
|
|
std::span<const std::uint8_t> output_pin_mask,
|
|
Rng& rng) noexcept {
|
|
const float layer_scale = spread_scale(spread, fan_in);
|
|
const float noise_stddev = speed * layer_scale;
|
|
const float decay = weight_decay_factor(spread);
|
|
|
|
const std::size_t fan_out = biases.size();
|
|
|
|
for (std::size_t node = 0; node < fan_out; ++node) {
|
|
const bool skip =
|
|
is_final_layer && !output_pin_mask.empty() && output_pin_mask[node] != 0u;
|
|
|
|
// Bias perturbation.
|
|
if (!skip) {
|
|
biases[node] += rng.next_float_gaussian(noise_stddev);
|
|
}
|
|
|
|
// Per-weight decay+noise. We always advance the RNG even on skip
|
|
// so that the random stream is independent of pin-mask state — this
|
|
// makes parity tests deterministic regardless of which outputs are
|
|
// pinned. Otherwise toggling a pin would shift the entire
|
|
// downstream noise sequence, which would be surprising.
|
|
const std::size_t row_off = node * fan_in;
|
|
for (std::size_t j = 0; j < fan_in; ++j) {
|
|
const float noise = rng.next_float_gaussian(noise_stddev);
|
|
if (!skip) {
|
|
float& w = weights[row_off + j];
|
|
w = w * decay + noise;
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
// Re-randomize one layer's weights using the spread-aware scale. Biases are
|
|
// reset to zero, matching `init.hpp::spread_init`.
|
|
inline void draw_weights_layer(std::span<float> weights,
|
|
std::span<float> biases,
|
|
std::size_t fan_in,
|
|
float spread,
|
|
Rng& rng) noexcept {
|
|
spread_init(weights, biases, fan_in, spread, rng);
|
|
}
|
|
|
|
} // namespace nisps::ml
|