w1n5t0n/memlnaut-nisps
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(vcv): reunify module onto core MLP — thin iml.hpp adapter (P6)

Browse source 
Replace the vendored runtime MLP in vcv/src/iml.hpp (DetRng + 3D-weight-store
MLP + Dataset + IML) with a THIN, Rack-free adapter over the shared core:
nisps::ml::MLPCore<nisps::ml::DynamicStorage> (8->[16,24,16]->16, the P2 dynamic
case), nisps::Rng, and the core MLP's own FIFO dataset. Behaviour changes from
the vendored approximation to core-exact firmware/WASM semantics.

- MEMLNaut.cpp: staged/pending weight buffers and patch JSON now use the core's
  flat [weights..][biases..] vector (nisps::IML<float>::Weights); patch version
  bumped to 3. Double-buffer / single-writer threading discipline unchanged.
- New ctest tests/cpp/test_vcv_iml_parity.cpp: seeded train/infer/move_weights
  session through the adapter is memcmp-equal to a bare MLPCore<DynamicStorage>.
- Docs: vcv-module.md delta #5 marked CLOSED (2026-07-18); MAP.md vcv/ updated.

Closes vcv-module.md delta #5.
This commit is contained in:
monkey-w1n5t0n 2026-07-18 13:01:28 +02:00
parent e1713ae41f
commit 1b69254de2
7 changed files with 350 additions and 400 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
MAP.md
View file

@ -86,11 +86,13 @@ anchor + locked decisions) and the `docs/specs/*-spec.md` set.
`tests/pipeline-golden.test.ts` (in `bun run test`). `manifold/osc-bridge/` — Deno WS↔UDP-OSC bridge.
### `vcv/` — VCV Rack 2 plugin (MEMLNaut module, WIP)
Native C++ Rack module: ML CV-mapper with RL feedback + a browser bridge. Currently 2→12 (being evolved to
**8 inputs × 16 outputs + per-output LED rings**, palette from the frontend tokens, WS↔OSC browser bridge — see
the "BUILD DELTAS" block at the top of `docs/specs/vcv-module.md`). `src/MEMLNaut.cpp` (module), `src/osc_server.hpp` (bridge),
`src/plugin.{hpp,cpp}`, `res/*.svg` (panels), `Makefile` (needs `RACK_DIR`). Was built against the retired
`nisps-core`; the core include path is being repointed.
Native C++ Rack module: ML CV-mapper with RL feedback + a browser bridge. **8 inputs × 16 outputs + per-output
LED rings**, palette from the frontend tokens, WS↔OSC browser bridge (see `docs/specs/vcv-module.md`).
`src/MEMLNaut.cpp` (module, 8→[16,24,16]→16), `src/iml.hpp` (**thin adapter over `nisps::ml::MLPCore<DynamicStorage>`
+ core `nisps::Rng`** — P6 reunification 2026-07-18, closes vcv-module.md delta #5; behaviour is now core-exact,
pinned by `tests/cpp/test_vcv_iml_parity.cpp`), `src/osc_server.hpp` (bridge, transport-only), `src/plugin.{hpp,cpp}`,
`res/*.svg` (panels), `Makefile` (needs `RACK_DIR`). Builds against the current `../nisps/` core via relative
includes; no `nisps-core`.
### `schemas/` — JSON parameter contracts (firmware/browser source of truth)
- `schemas/schema.json` — Draft 2020-12 meta-schema validating mode files.

16
docs/specs/vcv-module.md
View file

@ -174,9 +174,19 @@ The architecture is **fixed at compile time**; no runtime reconfiguration of inp
### Core Library Integration
The MLP uses a **runtime-shaped IML** (not the fixed-size WASM template). Point the build at the current `../nisps/` (not the retired `nisps-core`) and either:
1. Reuse `nisps/ml/mlp.hpp` and compile with `MLP<8, 24, 32, 16, 16>` type, or
2. Vendor a minimal self-contained 8→16 IML in `vcv/src/`, ensuring it shares the firmware/browser training semantics (spread-aware `DrawWeights`/`MoveWeights`, deterministic RNG). If templated-API constraints block option 1, option 2 is acceptable with a note of alignment as a follow-up.
**Delta #5 CLOSED (2026-07-18, one-core-engine-refactor P6).** The module no longer
vendors its own MLP. `vcv/src/iml.hpp` is now a THIN, Rack-free adapter over the shared
core: `nisps::ml::MLPCore<nisps::ml::DynamicStorage>` (the runtime-shaped branch of the one
core MLP — fixed 4-layer ReLU×3 + Sigmoid topology, three runtime hidden sizes) with the
module's real `[16, 24, 16]` shape, `nisps::Rng` (nisps/core/rng.hpp) replacing the vendored
`DetRng`, and the core MLP's own FIFO dataset replacing the vendored `Dataset`. The adapter
includes only nisps headers + the standard library (no Rack includes) so the host ctest can
compile it. Behaviour therefore **changed** from the vendored *approximation* of firmware/
browser training semantics to **core-exact**: weight init, RL `move_weights`, SGD training,
activations and RNG are now bit-identical to the firmware/WASM engine — pinned by
`tests/cpp/test_vcv_iml_parity.cpp` (adapter == bare `MLPCore<DynamicStorage>`, memcmp-equal).
Persisted weights are now the core's FLAT `[weights…][biases…]` vector (patch `version` = 3);
old 3D vendored weight blobs will not load.
### Spread Parameter

1
nisps/CMakeLists.txt
View file

@ -63,6 +63,7 @@ if(NOT EMSCRIPTEN)
${NISPS_TEST_DIR}/test_mlp_geo_dislike.cpp
${NISPS_TEST_DIR}/test_mlp_serialize.cpp
${NISPS_TEST_DIR}/test_pipeline.cpp
${NISPS_TEST_DIR}/test_vcv_iml_parity.cpp
)
target_link_libraries(nisps_core_tests PRIVATE nisps_core)

123
tests/cpp/test_vcv_iml_parity.cpp Normal file
View file

@ -0,0 +1,123 @@
// tests/cpp/test_vcv_iml_parity.cpp — the VCV module's IML adapter is a THIN
// wrapper over the shared core (one-core-engine-refactor P6 gate; closes
// vcv-module.md delta #5).
//
// A seeded train/infer session driven through the adapter
// (`nisps::IML<float>`, vcv/src/iml.hpp) must be BIT-IDENTICAL to driving a
// bare `MLPCore<DynamicStorage>` of the same shape/seed with the same examples
// and ops. Not 1e-5-near — memcmp-equal. This is what proves the module now
// runs core-exact semantics rather than the retired vendored approximation.
#include <cstdint>
#include <cstring>
#include <span>
#include <vector>
#include "../../nisps/ml/dynamic_storage.hpp"
#include "../../nisps/ml/mlp.hpp"
#include "../../vcv/src/iml.hpp"
#include "test_helpers.hpp"
namespace {
// The module's real runtime shape: 8 inputs → [16, 24, 16] → 16 outputs.
constexpr std::size_t kIn = 8u;
constexpr std::size_t kH1 = 16u, kH2 = 24u, kH3 = 16u;
constexpr std::size_t kOut = 16u;
constexpr std::uint64_t kSeed = 0xC0FFEEu;
using DynamicMLP = nisps::ml::MLPCore<nisps::ml::DynamicStorage>;
bool bit_equal(std::span<const float> a, std::span<const float> b) {
if (a.size() != b.size()) return false;
if (a.empty()) return true;
return std::memcmp(a.data(), b.data(), a.size() * sizeof(float)) == 0;
}
} // namespace
NISPS_TEST(vcv_iml_adapter_matches_core_bitexact) {
nisps::IML<float> adapter(kIn, kOut, {kH1, kH2, kH3},
/*max_iterations=*/200u,
/*learning_rate=*/0.1f,
/*convergence_threshold=*/0.00001f,
kSeed);
// The bare core the adapter is supposed to be a thin skin over: same seed,
// same dims, same capacities (kMaxExamples / max_iter_train) the adapter
// hands its own MLPCore at construction.
const std::size_t hidden[3] = {kH1, kH2, kH3};
DynamicMLP ref(kSeed, kIn, std::span<const std::size_t>(hidden), kOut,
nisps::IML<float>::kMaxExamples, adapter.train_max_iter());
NISPS_ASSERT(ref.valid());
// Construction alone (MLPCore ctor draws weights(1.f) from the seed).
{
auto aw = adapter.get_weights();
auto rw = ref.get_weights();
NISPS_ASSERT(aw.size() == rw.size());
NISPS_EXPECT(bit_equal(std::span<const float>(aw.data(), aw.size()), rw));
}
// Draw at a fixed interior spread.
adapter.randomise_weights(0.6f);
ref.draw_weights(0.6f);
{
auto aw = adapter.get_weights();
NISPS_EXPECT(bit_equal(std::span<const float>(aw.data(), aw.size()), ref.get_weights()));
}
// Add a fixed set of examples through both paths.
for (std::size_t e = 0; e < 6u; ++e) {
float feat[kIn];
float lab[kOut];
for (std::size_t i = 0; i < kIn; ++i)
feat[i] = 0.1f * static_cast<float>((e + i) % 10u);
for (std::size_t i = 0; i < kOut; ++i)
lab[i] = 0.05f * static_cast<float>((e * 3u + i) % 20u);
adapter.add_example(feat, kIn, lab, kOut);
ref.add_example(std::span<const float>(feat), std::span<const float>(lab));
}
NISPS_ASSERT(adapter.get_example_count() == ref.example_count());
// Train a fixed number of iterations. The adapter trains via the module's
// real Training→Inference transition; the bare core uses the identical
// (lr, max_iter, min_err) the adapter would.
adapter.set_mode(nisps::IML<float>::Mode::Training);
adapter.set_mode(nisps::IML<float>::Mode::Inference);
ref.train(adapter.train_lr(), adapter.train_max_iter(), adapter.train_min_err());
{
auto aw = adapter.get_weights();
NISPS_EXPECT(bit_equal(std::span<const float>(aw.data(), aw.size()), ref.get_weights()));
}
// Inference outputs at a fixed probe input.
const float probe[kIn] = {0.1f, 0.9f, 0.25f, 0.75f, 0.5f, 0.33f, 0.66f, 0.42f};
for (std::size_t i = 0; i < kIn; ++i) {
adapter.set_input(i, probe[i]);
ref.set_input(i, probe[i]);
}
adapter.process();
ref.process();
NISPS_EXPECT(bit_equal(std::span<const float>(adapter.get_outputs(), kOut), ref.outputs()));
// RL move_weights with an output pin mask (thumbs-down perturbation path).
std::uint8_t mask[kOut] = {};
mask[2] = 1u;
mask[5] = 1u;
adapter.move_weights(0.3f, 0.4f, std::span<const std::uint8_t>(mask));
ref.move_weights(0.3f, 0.4f, std::span<const std::uint8_t>(mask));
{
auto aw = adapter.get_weights();
NISPS_EXPECT(bit_equal(std::span<const float>(aw.data(), aw.size()), ref.get_weights()));
}
// And outputs stay identical after the perturbation.
for (std::size_t i = 0; i < kIn; ++i) {
adapter.set_input(i, probe[i]);
ref.set_input(i, probe[i]);
}
adapter.process();
ref.process();
NISPS_EXPECT(bit_equal(std::span<const float>(adapter.get_outputs(), kOut), ref.outputs()));
}

5
vcv/Makefile
View file

@ -11,8 +11,9 @@ endif
FLAGS += -std=c++20
FLAGS += -I$(RACK_DIR)/include -I$(RACK_DIR)/dep/include
# The retired nisps-core header tree is gone; the runtime IML/MLP is vendored
# self-contained in src/iml.hpp (see that file's header for the rationale).
# src/iml.hpp is a THIN adapter over the shared C++ core: it pulls in
# nisps/ml/{mlp,dynamic_storage}.hpp + nisps/core/rng.hpp via relative includes
# (../../nisps/…), so no extra -I for the core is needed here (P6, 2026-07-18).
FLAGS += -Isrc
SOURCES += src/plugin.cpp

43
vcv/src/MEMLNaut.cpp
View file

@ -82,12 +82,12 @@ struct MEMLNaut : Module {
nisps::IML<float> iml{NUM_ML_INPUTS, NUM_ML_OUTPUTS, {16, 24, 16}};
nisps::IML<float> imlShadow{NUM_ML_INPUTS, NUM_ML_OUTPUTS, {16, 24, 16}};
// Worker → Audio: staged weights ready for swap
nisps::MLP<float>::mlp_weights pendingWeights;
// Worker → Audio: staged weights ready for swap (core-exact flat layout)
nisps::IML<float>::Weights pendingWeights;
std::atomic<bool> weightsPending{false};
// Audio → Worker: staged weight snapshot for the worker to start from
nisps::MLP<float>::mlp_weights stagedWeightsForWorker;
nisps::IML<float>::Weights stagedWeightsForWorker;
std::vector<std::vector<float>> stagedFeatures;
std::vector<std::vector<float>> stagedLabels;
std::mutex stagingMutex;
@ -624,7 +624,7 @@ struct MEMLNaut : Module {
// ── Serialization ─────────────────────────────────────────────────
json_t* dataToJson() override {
json_t* root = json_object();
json_object_set_new(root, "version", json_integer(2));
json_object_set_new(root, "version", json_integer(3));
json_object_set_new(root, "inputCount", json_integer(NUM_ML_INPUTS));
json_object_set_new(root, "outputCount", json_integer(NUM_ML_OUTPUTS));
json_object_set_new(root, "noiseLevel", json_real(noiseLevel));
@ -643,17 +643,10 @@ struct MEMLNaut : Module {
json_array_append_new(inRanges, json_boolean(inputRangeUnipolar[i]));
json_object_set_new(root, "inputRangeUnipolar", inRanges);
// Core-exact FLAT weight vector: [layer0_w … layer3_w][layer0_b … layer3_b].
auto weights = iml.get_weights();
json_t* jWeights = json_array();
for (auto& layer : weights) {
json_t* jLayer = json_array();
for (auto& node : layer) {
json_t* jNode = json_array();
for (float w : node) json_array_append_new(jNode, json_real(w));
json_array_append_new(jLayer, jNode);
}
json_array_append_new(jWeights, jLayer);
}
for (float w : weights) json_array_append_new(jWeights, json_real(w));
json_object_set_new(root, "weights", jWeights);
auto features = iml.get_example_features();
@ -675,9 +668,11 @@ struct MEMLNaut : Module {
json_object_set_new(jExamples, "labels", jLabels);
json_object_set_new(root, "examples", jExamples);
// Core topology: explicit biases (no trailing bias node), so the layer
// node counts are [n_in, 16, 24, 16, n_out].
json_t* jConfig = json_object();
json_t* jLayers = json_array();
json_array_append_new(jLayers, json_integer(NUM_ML_INPUTS + 1)); // + bias
json_array_append_new(jLayers, json_integer(NUM_ML_INPUTS));
for (int h : {16, 24, 16}) json_array_append_new(jLayers, json_integer(h));
json_array_append_new(jLayers, json_integer(NUM_ML_OUTPUTS));
json_object_set_new(jConfig, "layers", jLayers);
@ -707,21 +702,15 @@ struct MEMLNaut : Module {
for (int i = 0; i < MAX_ML_INPUTS && i < (int)json_array_size(inRanges); i++)
inputRangeUnipolar[i] = json_boolean_value(json_array_get(inRanges, i));
// Core-exact FLAT weight vector (patch version ≥ 3). Old 3D weight
// blobs from the vendored model are a different shape and are ignored
// by set_weights (size guard) — see iml.hpp header.
json_t* jWeights = json_object_get(root, "weights");
if (jWeights && json_is_array(jWeights)) {
nisps::MLP<float>::mlp_weights weights;
for (size_t li = 0; li < json_array_size(jWeights); li++) {
json_t* jLayer = json_array_get(jWeights, li);
std::vector<std::vector<float>> layer;
for (size_t ni = 0; ni < json_array_size(jLayer); ni++) {
json_t* jNode = json_array_get(jLayer, ni);
std::vector<float> node;
for (size_t wi = 0; wi < json_array_size(jNode); wi++)
node.push_back(json_real_value(json_array_get(jNode, wi)));
layer.push_back(node);
}
weights.push_back(layer);
}
nisps::IML<float>::Weights weights;
weights.reserve(json_array_size(jWeights));
for (size_t wi = 0; wi < json_array_size(jWeights); wi++)
weights.push_back(json_real_value(json_array_get(jWeights, wi)));
iml.set_weights(weights);
}

550
vcv/src/iml.hpp
View file

@ -1,409 +1,233 @@
// iml.hpp — Self-contained runtime IML/MLP for the MEMLNaut VCV module.
// iml.hpp — THIN adapter over the real NISPS core for the MEMLNaut VCV module.
//
// This is a VENDORED, runtime-shaped re-implementation of the nisps core
// `nisps::IML<float>` / `nisps::MLP<float>` surface the VCV module relies on.
// The retired `nisps-core` header tree (`-I../nisps-core/include`) is gone, and
// the templated firmware/WASM `nisps/ml` core is fixed-size — neither is a clean
// fit for a runtime 8→16 module. So we ship a small native MLP here that matches
// the firmware/browser TRAINING SEMANTICS as closely as a runtime form allows:
// P6 REUNIFICATION (one-core-engine-refactor, 2026-07-18): this file USED to
// be a self-contained, vendored runtime MLP (a `DetRng` + 3D-weight-store MLP
// with a trailing-bias-node model + its own Dataset). That vendored copy has
// been DELETED. The module now consumes the shared engine directly:
//
// • ReLU hidden layers, sigmoid output (sigmoid maps to [0,1]).
// • A trailing bias node (1.0) appended to the input vector.
// • spread-aware weight init: uniform [-1,1] (spread=0) → Xavier 1/√fan_in
// (spread=1), interpolated per layer.
// • spread-aware perturbation (RL "move weights"): flat noise (spread=0) →
// per-layer Xavier-scaled noise + 10%·spread weight decay (spread=1).
// • Plain SGD / MSE training over the example dataset.
// • DETERMINISTIC per-instance RNG (seeded), so behaviour is reproducible and
// the threading double-buffer stays race-free (each MLP owns its own RNG).
// • `nisps::ml::MLPCore<nisps::ml::DynamicStorage>` — the runtime-shaped
// branch of the ONE core MLP (fixed 4-layer topology: ReLU×3 + Sigmoid,
// three runtime hidden sizes). This is exactly the P2 dynamic case.
// • `nisps::Rng` (nisps/core/rng.hpp, xoshiro256+) — owned by MLPCore,
// replacing the vendored `DetRng` (xorshift128).
// • The core MLP's own FIFO dataset (add_example / train / clear_examples),
// replacing the vendored `Dataset`.
//
// It is NOT bit-identical to the firmware core (different optimiser internals),
// and that divergence is an accepted follow-up (see docs/specs/vcv-module.md delta #5). The
// public method names mirror the core so `MEMLNaut.cpp` is unchanged in spirit.
// BEHAVIOUR CHANGED (this is the point of the phase): outputs are no longer the
// vendored *approximation* of firmware/browser training semantics — they are
// now CORE-EXACT. Weight init, RL `move_weights`, SGD training, activations and
// RNG are bit-identical to what the firmware and the WASM/browser build run
// (see tests/cpp/test_vcv_iml_parity.cpp, which pins adapter == direct
// MLPCore<DynamicStorage> bit-for-bit). Old `.nisps`/patch weight blobs written
// by the vendored 3D model will NOT load — the persisted weight vector is now
// the core's FLAT [weights…][biases…] layout (patch `version` bumped to 3).
//
// This adapter includes ONLY nisps headers + the standard library (no Rack
// includes) so the host ctest can compile it standalone. It keeps the public
// surface as close to the old vendored `IML` as practical so `MEMLNaut.cpp`
// stays mechanical; the one deliberate surface change is `get_weights` /
// `set_weights`, which now speak the core's flat `std::vector<float>` instead
// of the old 3D `mlp_weights` node tree.
//
// MPL-2.0 in spirit with the rest of nisps; wrapper code under the VCV module's
// licence. British spelling in comments where it reads naturally.
#pragma once
#include <vector>
#include <cstddef>
#include <algorithm>
#include <array>
#include <cmath>
#include <cstddef>
#include <cstdint>
#include <limits>
#include <algorithm>
#include <span>
#include <vector>
#include "../../nisps/core/rng.hpp"
#include "../../nisps/ml/dynamic_storage.hpp"
#include "../../nisps/ml/mlp.hpp"
namespace nisps {
// ── Tiny deterministic RNG (xorshift128) ──────────────────────────────
// Per-instance state; seeded in the constructor. No std::random_device, no
// shared global generator — this is what keeps the audio/worker MLP pair free
// of data races and makes parity reproducible.
class DetRng {
public:
explicit DetRng(uint32_t seed = 0x1234567u) { reseed(seed); }
void reseed(uint32_t seed) {
s_[0] = seed ? seed : 0xA5A5A5A5u;
s_[1] = s_[0] ^ 0x9E3779B9u;
s_[2] = s_[0] * 0x85EBCA6Bu + 1u;
s_[3] = s_[0] * 0xC2B2AE35u + 0x27D4EB2Fu;
}
uint32_t next_u32() {
uint32_t t = s_[3];
uint32_t const u = s_[0];
s_[3] = s_[2]; s_[2] = s_[1]; s_[1] = u;
t ^= t << 11;
t ^= t >> 8;
s_[0] = t ^ u ^ (u >> 19);
return s_[0];
}
// Uniform in [0,1)
float uniform01() { return (next_u32() >> 8) * (1.0f / 16777216.0f); }
// Uniform in [-1,1)
float uniform_pm1() { return uniform01() * 2.0f - 1.0f; }
// Approx standard-normal: sum of 3 uniforms (matches the core's gen_randn shape)
float gaussian() {
return (uniform_pm1() + uniform_pm1() + uniform_pm1()) * 0.5773502692f; // /√3 → unit-ish variance
}
private:
uint32_t s_[4];
};
// ── MLP ───────────────────────────────────────────────────────────────
template<typename T = float>
class MLP {
public:
// 3D weight store: [layer][node][weight] where the final weight per node is
// the bias (the previous layer's activations get a trailing 1.0).
using mlp_weights = std::vector<std::vector<std::vector<T>>>;
// layers_nodes: full sizes including input (with bias) and output, e.g.
// {n_in + 1, h0, h1, h2, n_out}
MLP(const std::vector<size_t>& layers_nodes, uint32_t seed)
: layers_nodes_(layers_nodes), rng_(seed) {
build_();
draw_weights_spread_(static_cast<T>(0));
}
size_t num_layers() const { return weights_.size(); }
// Forward pass. `input_with_bias` has the trailing 1.0 already appended.
void forward(const std::vector<T>& input_with_bias, std::vector<T>& out) const {
std::vector<T> act = input_with_bias;
for (size_t l = 0; l < weights_.size(); ++l) {
const auto& layer = weights_[l];
const bool is_output = (l + 1 == weights_.size());
std::vector<T> next(layer.size());
for (size_t n = 0; n < layer.size(); ++n) {
const auto& w = layer[n];
T sum = 0;
// w has act.size()+1 entries? No: w spans the *current* input
// size (which already includes the bias slot of `act`).
const size_t lim = std::min(w.size(), act.size());
for (size_t k = 0; k < lim; ++k) sum += w[k] * act[k];
next[n] = is_output ? sigmoid_(sum) : relu_(sum);
}
// For hidden layers, append a bias term for the next layer's input.
if (!is_output) next.push_back(static_cast<T>(1));
act = std::move(next);
}
out = std::move(act);
}
// ── spread-aware weight init ──────────────────────────────────────
void draw_weights_spread(T spread) { draw_weights_spread_(spread); }
// ── spread-aware perturbation (RL move_weights) ───────────────────
void move_weights_spread(T speed, T spread) {
const T decay = static_cast<T>(1) - static_cast<T>(0.1) * spread;
for (size_t l = 0; l < weights_.size(); ++l) {
const T fan_in = static_cast<T>(input_size_of_layer_(l));
const T xavier = (fan_in > 0) ? static_cast<T>(1) / std::sqrt(fan_in) : static_cast<T>(1);
// spread=0 → flat noise (scale 1); spread=1 → per-layer Xavier scale
const T noiseScale = (static_cast<T>(1) - spread) + spread * xavier;
for (auto& node : weights_[l]) {
for (auto& w : node) {
if (spread > 0) w *= decay; // weight decay only when spread>0
w += rng_.gaussian() * speed * noiseScale;
}
}
}
}
// ── plain SGD / MSE training ──────────────────────────────────────
// features: each row is input WITHOUT bias; labels: target outputs in [0,1].
void train(const std::vector<std::vector<T>>& features,
const std::vector<std::vector<T>>& labels,
int max_iterations, T learning_rate, T convergence) {
const size_t n = std::min(features.size(), labels.size());
if (n == 0) return;
for (int iter = 0; iter < max_iterations; ++iter) {
T epoch_loss = 0;
for (size_t s = 0; s < n; ++s) {
std::vector<T> in = features[s];
in.push_back(static_cast<T>(1)); // bias
epoch_loss += backprop_(in, labels[s], learning_rate);
}
epoch_loss /= static_cast<T>(n);
if (epoch_loss < convergence) break;
}
}
mlp_weights get_weights() const { return weights_; }
void set_weights(const mlp_weights& w) {
// Only adopt if the topology matches; otherwise ignore (keeps the audio
// path safe against malformed snapshots from the bridge / patch files).
if (w.size() != weights_.size()) return;
for (size_t l = 0; l < w.size(); ++l) {
if (w[l].size() != weights_[l].size()) return;
}
weights_ = w;
}
private:
static T relu_(T x) { return x > 0 ? x : 0; }
static T sigmoid_(T x) { return static_cast<T>(1) / (static_cast<T>(1) + std::exp(-x)); }
static T dsigmoid_from_out_(T y) { return y * (static_cast<T>(1) - y); }
size_t input_size_of_layer_(size_t l) const {
// The number of weights per node in layer l (incl. bias slot).
return weights_[l].empty() ? 0 : weights_[l][0].size();
}
void build_() {
weights_.clear();
// layers_nodes_[0] is the input layer WITH bias already counted.
for (size_t l = 1; l < layers_nodes_.size(); ++l) {
const size_t in_sz = layers_nodes_[l - 1]; // includes bias slot
const size_t out_sz = layers_nodes_[l];
std::vector<std::vector<T>> layer(out_sz, std::vector<T>(in_sz, 0));
weights_.push_back(std::move(layer));
}
}
void draw_weights_spread_(T spread) {
for (size_t l = 0; l < weights_.size(); ++l) {
const T fan_in = static_cast<T>(layers_nodes_[l]); // incl. bias
const T xavier = (fan_in > 0) ? static_cast<T>(1) / std::sqrt(fan_in) : static_cast<T>(1);
const T scale = (static_cast<T>(1) - spread) + spread * xavier;
for (auto& node : weights_[l]) {
for (size_t k = 0; k < node.size(); ++k) {
// bias (last weight) initialised to 0, like the core
const bool is_bias = (k + 1 == node.size());
node[k] = is_bias ? static_cast<T>(0) : rng_.uniform_pm1() * scale;
}
}
}
}
// One SGD step on a single example; returns the MSE for this example.
T backprop_(const std::vector<T>& in_with_bias, const std::vector<T>& target,
T lr) {
// Forward, caching activations per layer.
std::vector<std::vector<T>> acts;
acts.reserve(weights_.size() + 1);
acts.push_back(in_with_bias);
std::vector<T> act = in_with_bias;
for (size_t l = 0; l < weights_.size(); ++l) {
const auto& layer = weights_[l];
const bool is_output = (l + 1 == weights_.size());
std::vector<T> next(layer.size());
for (size_t nidx = 0; nidx < layer.size(); ++nidx) {
const auto& w = layer[nidx];
T sum = 0;
const size_t lim = std::min(w.size(), act.size());
for (size_t k = 0; k < lim; ++k) sum += w[k] * act[k];
next[nidx] = is_output ? sigmoid_(sum) : relu_(sum);
}
if (!is_output) next.push_back(static_cast<T>(1));
acts.push_back(next);
act = next;
}
// Output error.
const size_t L = weights_.size();
std::vector<T>& out = acts[L];
T loss = 0;
std::vector<T> delta(out.size());
for (size_t o = 0; o < out.size(); ++o) {
const T t = (o < target.size()) ? target[o] : static_cast<T>(0);
const T e = out[o] - t;
loss += e * e;
delta[o] = e * dsigmoid_from_out_(out[o]); // MSE × sigmoid'
}
loss /= static_cast<T>(out.size() ? out.size() : 1);
// Backprop through layers L-1 .. 0.
std::vector<T> nextDelta;
for (size_t li = L; li-- > 0;) {
const auto& prevAct = acts[li]; // input activations to layer li
auto& layer = weights_[li];
const bool is_output = (li + 1 == L);
// Compute delta to propagate to the previous layer (excludes bias node).
const size_t prevSize = prevAct.size(); // includes bias slot
std::vector<T> propagate(prevSize, 0);
for (size_t nidx = 0; nidx < layer.size(); ++nidx) {
const T d = delta[nidx];
auto& w = layer[nidx];
const size_t lim = std::min(w.size(), prevSize);
for (size_t k = 0; k < lim; ++k) {
propagate[k] += d * w[k];
w[k] -= lr * d * prevAct[k]; // gradient step
}
}
// Turn `propagate` into next-layer delta via ReLU' (skip for input).
if (li > 0) {
const auto& actPrev = acts[li]; // activations of layer li-1's output
nextDelta.assign(actPrev.size(), 0);
for (size_t k = 0; k < actPrev.size(); ++k) {
const T relud = actPrev[k] > 0 ? static_cast<T>(1) : static_cast<T>(0);
nextDelta[k] = propagate[k] * relud;
}
// Drop the trailing bias slot's delta (it has no upstream weights).
if (!nextDelta.empty()) nextDelta.pop_back();
delta = nextDelta;
}
(void)is_output;
}
return loss;
}
std::vector<size_t> layers_nodes_;
mlp_weights weights_;
mutable DetRng rng_;
};
// ── Dataset (FIFO ring, max 100 examples) ─────────────────────────────
template<typename T = float>
class Dataset {
public:
static constexpr size_t kMax_examples = 100;
void add(const std::vector<T>& feat, const std::vector<T>& label) {
if (features_.size() >= kMax_examples) {
features_.erase(features_.begin());
labels_.erase(labels_.begin());
}
features_.push_back(feat);
labels_.push_back(label);
}
void clear() { features_.clear(); labels_.clear(); }
size_t count() const { return features_.size(); }
const std::vector<std::vector<T>>& features() const { return features_; }
const std::vector<std::vector<T>>& labels() const { return labels_; }
void load(const std::vector<std::vector<T>>& f, const std::vector<std::vector<T>>& l) {
clear();
const size_t n = std::min(f.size(), l.size());
for (size_t i = 0; i < n; ++i) add(f[i], l[i]);
}
private:
std::vector<std::vector<T>> features_;
std::vector<std::vector<T>> labels_;
};
// ── IML ───────────────────────────────────────────────────────────────
template<typename Float = float>
// ── IML — runtime-shaped interactive-ML adapter ───────────────────────
//
// One instance owns one `MLPCore<DynamicStorage>` (its own weights, RNG and
// dataset — nothing is shared). The MEMLNaut module keeps two instances (an
// audio-thread `iml` and a worker-thread `imlShadow`) and hands flat weight
// snapshots between them; because each instance is fully self-contained the
// core's lack of internal locking is fine — the module's single-writer
// double-buffer discipline is preserved around this adapter unchanged.
template <typename Float = float>
class IML {
public:
static_assert(std::is_same<Float, float>::value,
"the shared MLP core is float-only");
public:
// Flat weight vector: the core's [layer0_w … layer3_w][layer0_b … layer3_b]
// layout. Cheap to copy between the audio and worker threads.
using Weights = std::vector<float>;
using Examples = std::vector<std::vector<float>>;
enum class Mode { Inference, Training };
IML(size_t n_inputs, size_t n_outputs,
std::vector<size_t> hidden_layers = {16, 24, 16},
size_t max_iterations = 200,
Float learning_rate = static_cast<Float>(0.1),
Float convergence_threshold = static_cast<Float>(0.00001),
uint32_t seed = 0xC0FFEEu)
: n_inputs_(n_inputs), n_outputs_(n_outputs),
max_iterations_(max_iterations), learning_rate_(learning_rate),
convergence_threshold_(convergence_threshold) {
std::vector<size_t> sizes;
sizes.push_back(n_inputs_ + 1); // + bias
for (size_t h : hidden_layers) sizes.push_back(h);
sizes.push_back(n_outputs_);
mlp_ = std::make_unique<MLP<Float>>(sizes, seed);
input_state_.assign(n_inputs_, static_cast<Float>(0.5));
output_state_.assign(n_outputs_, static_cast<Float>(0));
}
// Matches the old vendored capacity so `get_max_examples()` and the FIFO
// eviction threshold are unchanged for the UI / persistence.
static constexpr std::size_t kMaxExamples = 100u;
size_t num_inputs() const { return n_inputs_; }
size_t num_outputs() const { return n_outputs_; }
// Signature mirrors the old vendored IML so the module's construction sites
// are unchanged. `hidden` MUST carry exactly three sizes (the core topology
// is fixed at three hidden layers); fewer are padded from the default,
// extras ignored.
explicit IML(std::size_t n_inputs, std::size_t n_outputs,
std::vector<std::size_t> hidden = {16u, 24u, 16u},
std::size_t max_iterations = 200u,
Float learning_rate = static_cast<Float>(0.1),
Float convergence_threshold = static_cast<Float>(0.00001),
std::uint64_t seed = 0xC0FFEEu)
: n_inputs_(n_inputs),
n_outputs_(n_outputs),
max_iterations_(max_iterations),
learning_rate_(learning_rate),
convergence_threshold_(convergence_threshold),
hidden_(to_hidden3_(hidden)),
core_(seed, n_inputs, std::span<const std::size_t>(hidden_.data(), hidden_.size()),
n_outputs, kMaxExamples, max_iterations) {}
void set_input(size_t i, Float v) {
if (i >= n_inputs_) return;
input_state_[i] = std::clamp(v, static_cast<Float>(0), static_cast<Float>(1));
input_updated_ = true;
}
std::size_t num_inputs() const { return n_inputs_; }
std::size_t num_outputs() const { return n_outputs_; }
const Float* get_outputs() const { return output_state_.data(); }
void process() {
if (!input_updated_) return;
std::vector<Float> in = input_state_;
in.push_back(static_cast<Float>(1));
mlp_->forward(in, output_state_);
if (output_state_.size() < n_outputs_) output_state_.resize(n_outputs_, 0);
input_updated_ = false;
}
// ── Inference ─────────────────────────────────────────────────────
void set_input(std::size_t i, Float v) { core_.set_input(i, v); }
void process() { core_.process(); }
const Float* get_outputs() const { return core_.outputs().data(); }
// ── Mode / training ───────────────────────────────────────────────
// Preserves the vendored two-step semantics the module relies on: a
// Training→Inference transition trains the network over the current
// dataset. (The module drives this on both the audio `iml` — for immediate
// local feedback — and the worker `imlShadow`.)
void set_mode(Mode m) {
if (m == Mode::Inference && mode_ == Mode::Training) train_();
mode_ = m;
}
Mode get_mode() const { return mode_; }
void add_example(const Float* inputs, size_t n_in, const Float* outputs, size_t n_out) {
std::vector<Float> in(inputs, inputs + std::min(n_in, n_inputs_));
in.resize(n_inputs_, static_cast<Float>(0));
std::vector<Float> out(outputs, outputs + std::min(n_out, n_outputs_));
out.resize(n_outputs_, static_cast<Float>(0));
dataset_.add(in, out);
// Direct training entry (used by the parity test); returns final epoch loss.
float train() {
if (features_.empty()) return 0.f;
return core_.train(learning_rate_, max_iterations_, convergence_threshold_);
}
void clear_dataset() { dataset_.clear(); }
void randomise_weights(Float spread) { mlp_->draw_weights_spread(spread); refresh_(); }
void move_weights(Float speed, Float spread) { mlp_->move_weights_spread(speed, spread); refresh_(); }
// ── Dataset ───────────────────────────────────────────────────────
void add_example(const Float* inputs, std::size_t n_in,
const Float* outputs, std::size_t n_out) {
std::vector<float> in(n_inputs_, 0.f);
for (std::size_t i = 0; i < n_inputs_ && i < n_in; ++i) in[i] = inputs[i];
std::vector<float> out(n_outputs_, 0.f);
for (std::size_t i = 0; i < n_outputs_ && i < n_out; ++i) out[i] = outputs[i];
add_example_(in, out);
}
void clear_dataset() {
features_.clear();
labels_.clear();
core_.clear_examples();
}
std::size_t get_example_count() const { return core_.example_count(); }
std::size_t get_max_examples() const { return kMaxExamples; }
Examples get_example_features() const { return features_; }
Examples get_example_labels() const { return labels_; }
void load_examples(const Examples& f, const Examples& l) {
clear_dataset();
const std::size_t n = std::min(f.size(), l.size());
for (std::size_t i = 0; i < n; ++i) {
std::vector<float> in(n_inputs_, 0.f);
for (std::size_t k = 0; k < n_inputs_ && k < f[i].size(); ++k) in[k] = f[i][k];
std::vector<float> out(n_outputs_, 0.f);
for (std::size_t k = 0; k < n_outputs_ && k < l[i].size(); ++k) out[k] = l[i][k];
add_example_(in, out);
}
}
typename MLP<Float>::mlp_weights get_weights() const { return mlp_->get_weights(); }
void set_weights(typename MLP<Float>::mlp_weights& w) { mlp_->set_weights(w); }
// ── RL / weight ops ───────────────────────────────────────────────
void randomise_weights(Float spread) { core_.draw_weights(spread); }
void move_weights(Float speed, Float spread,
std::span<const std::uint8_t> pin_mask = {}) {
core_.move_weights(speed, spread, pin_mask);
}
size_t get_example_count() const { return dataset_.count(); }
size_t get_max_examples() const { return Dataset<Float>::kMax_examples; }
std::vector<std::vector<Float>> get_example_features() const { return dataset_.features(); }
std::vector<std::vector<Float>> get_example_labels() const { return dataset_.labels(); }
void load_examples(const std::vector<std::vector<Float>>& f,
const std::vector<std::vector<Float>>& l) { dataset_.load(f, l); }
// ── Flat weight get/set (core-exact layout) ───────────────────────
// Non-const: the core regenerates a scratch copy on each `get_weights`.
Weights get_weights() {
auto s = core_.get_weights();
return Weights(s.begin(), s.end());
}
void set_weights(const Weights& w) {
core_.set_weights(std::span<const float>(w.data(), w.size()));
}
Float nearest_example_distance(const Float* input, size_t n_in) const {
const auto& feats = dataset_.features();
if (feats.empty()) return static_cast<Float>(-1);
// Novelty helper (used for the display / derived outputs) — nearest
// Euclidean distance from `input` to any stored example, or -1 when empty.
Float nearest_example_distance(const Float* input, std::size_t n_in) const {
if (features_.empty()) return static_cast<Float>(-1);
Float best = std::numeric_limits<Float>::max();
const size_t dims = std::min(n_in, n_inputs_);
for (const auto& f : feats) {
const std::size_t dims = std::min(n_in, n_inputs_);
for (const auto& f : features_) {
Float d = 0;
for (size_t k = 0; k < dims && k < f.size(); ++k) {
for (std::size_t k = 0; k < dims && k < f.size(); ++k) {
const Float diff = f[k] - input[k];
d += diff * diff;
}
best = std::min(best, std::sqrt(d));
best = std::min(best, static_cast<Float>(std::sqrt(d)));
}
return best;
}
private:
void refresh_() { input_updated_ = true; process(); }
void train_() {
if (dataset_.count() == 0) return;
mlp_->train(dataset_.features(), dataset_.labels(),
static_cast<int>(max_iterations_), learning_rate_,
convergence_threshold_);
refresh_();
// Training parameters (exposed so a parity harness can drive a bare
// MLPCore with the same values the adapter's `train()` uses).
float train_lr() const { return learning_rate_; }
std::size_t train_max_iter() const { return max_iterations_; }
float train_min_err() const { return convergence_threshold_; }
private:
static std::array<std::size_t, 3> to_hidden3_(const std::vector<std::size_t>& h) {
std::array<std::size_t, 3> a{16u, 24u, 16u};
for (std::size_t i = 0; i < 3u && i < h.size(); ++i) a[i] = h[i];
return a;
}
size_t n_inputs_, n_outputs_, max_iterations_;
Float learning_rate_, convergence_threshold_;
// Adds to the core dataset AND to the insertion-order mirror. The mirror
// exists because MLPCore's FIFO ring keeps its head index private and its
// slot order scrambles once the buffer fills — but the module needs the
// examples enumerable *in insertion order* for JSON persistence and for
// cross-thread staging (get_example_features/labels → load_examples). The
// core stays the single source of truth for TRAINING; the mirror is purely
// a serialisation/enumeration view and never feeds the network. Both evict
// at kMaxExamples so counts stay equal.
void add_example_(const std::vector<float>& in, const std::vector<float>& out) {
if (features_.size() >= kMaxExamples) {
features_.erase(features_.begin());
labels_.erase(labels_.begin());
}
features_.push_back(in);
labels_.push_back(out);
core_.add_example(std::span<const float>(in.data(), in.size()),
std::span<const float>(out.data(), out.size()));
}
void train_() {
if (features_.empty()) return;
core_.train(learning_rate_, max_iterations_, convergence_threshold_);
}
std::size_t n_inputs_;
std::size_t n_outputs_;
std::size_t max_iterations_;
Float learning_rate_;
Float convergence_threshold_;
std::array<std::size_t, 3> hidden_;
nisps::ml::MLPCore<nisps::ml::DynamicStorage> core_;
Mode mode_ = Mode::Inference;
bool input_updated_ = false;
std::vector<Float> input_state_, output_state_;
Dataset<Float> dataset_;
std::unique_ptr<MLP<Float>> mlp_;
Examples features_;
Examples labels_;
};
} // namespace nisps
} // namespace nisps