feat(vcv): evolve module to 8x16 + LED rings + token palette + WS-OSC bridge
8 inputs x 16 outputs; per-output LED ring widget (drawLayer+nvgArc); palette from frontend tokens; OSC bridge verbs for bidirectional browser training; vendored self-contained iml.hpp (retired nisps-core); compiles against Rack SDK 2.6.4. See SPEC.md BUILD DELTAS.
This commit is contained in:
parent
0d8179d6d5
commit
fbc68ebfab
8 changed files with 1005 additions and 370 deletions
|
|
@ -2,7 +2,9 @@ RACK_DIR ?= $(HOME)/.local/share/Rack2/Rack-SDK
|
||||||
|
|
||||||
FLAGS += -std=c++20
|
FLAGS += -std=c++20
|
||||||
FLAGS += -I$(RACK_DIR)/include -I$(RACK_DIR)/dep/include
|
FLAGS += -I$(RACK_DIR)/include -I$(RACK_DIR)/dep/include
|
||||||
FLAGS += -I../nisps-core/include
|
# The retired nisps-core header tree is gone; the runtime IML/MLP is vendored
|
||||||
|
# self-contained in src/iml.hpp (see that file's header for the rationale).
|
||||||
|
FLAGS += -Isrc
|
||||||
|
|
||||||
SOURCES += src/plugin.cpp
|
SOURCES += src/plugin.cpp
|
||||||
SOURCES += src/MEMLNaut.cpp
|
SOURCES += src/MEMLNaut.cpp
|
||||||
|
|
|
||||||
46
vcv/SPEC.md
46
vcv/SPEC.md
|
|
@ -1,5 +1,51 @@
|
||||||
# MEMLNaut VCV Rack Module — Specification
|
# MEMLNaut VCV Rack Module — Specification
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ⚠️ BUILD DELTAS (2026-06-28) — AUTHORITATIVE OVERRIDES
|
||||||
|
|
||||||
|
These supersede any conflicting detail below. They reflect the Manifold mission + the locked decisions in
|
||||||
|
`docs/redesign/BUILD-PLAN.md` and `docs/redesign/backends-spec.md`. Build to THESE.
|
||||||
|
|
||||||
|
1. **I/O = 8 inputs × 16 outputs** (was 2→12). `NUM_ML_INPUTS = 8`, `NUM_ML_OUTPUTS = 16`. The IML is sized
|
||||||
|
8→16 (a runtime-shaped native MLP is fine here — the module is C++, not the fixed WASM target). The 8 CV
|
||||||
|
inputs feed the model's input dims; the 16 CV outputs are the model's inference outputs (the modular N×M
|
||||||
|
envelope). Keep the control inputs (Spread CV, Learn gate, + / − triggers).
|
||||||
|
2. **LED RING around EACH of the 16 outputs** — a custom ring widget encircling each output jack whose arc
|
||||||
|
fills in proportion to that output's value (0..1 → 0..2π). Draw on `drawLayer()` layer 1 with `nvgArc` for
|
||||||
|
the proportional fill + a dim track ring. Each ring is COLOURED from a palette that MATCHES the frontend
|
||||||
|
design tokens.
|
||||||
|
3. **Palette from the frontend tokens** — generate `vcv/src/palette.hpp` from
|
||||||
|
`docs/redesign/manifold-export/tokens/colors.css`: `--accent #ff6a00` (orange), `--accent-2 #00ccff` (cyan),
|
||||||
|
and the group colours (formant→accent, pitch→accent-2, amp→`--good #6bc26b`, filter→`--warn #f5c45e`,
|
||||||
|
fx→`--info #5b9eef`, mod→`--accent-3 #ffa860`). Assign the 16 rings across these group colours (or a clean
|
||||||
|
16-step ramp between orange and cyan) so the module reads as the same instrument as the browser. A tiny
|
||||||
|
hand-written `palette.hpp` is acceptable (no build-time codegen needed).
|
||||||
|
4. **Browser ↔ VCV bridge = WS↔OSC** (locked transport). The module runs its OSC server (`src/osc_server.hpp`
|
||||||
|
already exists — evolve it). The browser's OSC backend (`manifold/src/backends/osc-backend.ts`) sends over a
|
||||||
|
WebSocket to the Deno bridge (`manifold/osc-bridge/`), which relays UDP-OSC to the module. **Bidirectional
|
||||||
|
training**: drive + train the module FROM the browser (the verdict loop + example-placing over the bridge)
|
||||||
|
AND from the module's own panel (+/− buttons, Learn gate, triggers). OSC verbs to support both directions:
|
||||||
|
`/nisps/input` (drive), `/nisps/output` (module→browser viz), `/nisps/feedback` (thumbs up/down + place),
|
||||||
|
`/nisps/weights`, `/nisps/examples`, `/nisps/state`. Pick a fixed default UDP port (e.g. 7001) + per-instance
|
||||||
|
offset; the Deno bridge maps `ws://localhost:8765` ↔ that UDP port.
|
||||||
|
5. **Core include path** — the Makefile's `-I../nisps-core/include` points at the RETIRED `nisps-core`. Repoint
|
||||||
|
to the current core (`../nisps/`) OR vendor a minimal runtime IML inside `vcv/src/`. Goal: get it COMPILING
|
||||||
|
with an 8→16 runtime MLP that shares the firmware/browser training semantics (spread-aware draw/move_weights,
|
||||||
|
deterministic RNG) as closely as the native runtime-shaped form allows. If full nisps/ml reuse is blocked by
|
||||||
|
the templated fixed-size API, keep a self-contained IML in the module and note the alignment as a follow-up.
|
||||||
|
6. **Derived outputs** (Mean/Std/Delta/Novelty/Confidence) → move to the optional EXPANDER or a context-menu
|
||||||
|
toggle; the headline is 16 raw outputs + their LED rings. Do not let them crowd the 16-jack panel.
|
||||||
|
7. **Build** needs the VCV Rack 2 SDK (`RACK_DIR`, default `$HOME/.local/share/Rack2/Rack-SDK`, NOT installed).
|
||||||
|
The build step must fetch the Linux Rack-SDK zip from vcvrack.com, set `RACK_DIR`, and `make` to verify the
|
||||||
|
plugin compiles. Panel SVGs in `vcv/res/` exist (MEMLNaut.svg / -wide / -expander) — widen/relayout for 16
|
||||||
|
outputs + rings as needed.
|
||||||
|
|
||||||
|
The rest of this document is the prior (2→12) design — useful for threading, persistence, RL workflow, and
|
||||||
|
panel/build mechanics, but the I/O counts, LED rings, palette, bridge, and core path above WIN.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Overview
|
## Overview
|
||||||
|
|
||||||
A VCV Rack module that embeds the NISPS interactive ML engine (nisps-core C++ library) as a CV-to-CV mapper. Users explore high-dimensional parameter spaces via reinforcement learning feedback, producing 12 raw CV outputs and 5 derived meta-signals from configurable CV inputs.
|
A VCV Rack module that embeds the NISPS interactive ML engine (nisps-core C++ library) as a CV-to-CV mapper. Users explore high-dimensional parameter spaces via reinforcement learning feedback, producing 12 raw CV outputs and 5 derived meta-signals from configurable CV inputs.
|
||||||
|
|
|
||||||
|
|
@ -1,7 +1,7 @@
|
||||||
{
|
{
|
||||||
"slug": "MEMLNaut",
|
"slug": "MEMLNaut",
|
||||||
"name": "MEMLNaut",
|
"name": "MEMLNaut",
|
||||||
"version": "0.1.0",
|
"version": "0.2.0",
|
||||||
"license": "proprietary",
|
"license": "proprietary",
|
||||||
"brand": "MEMLNaut",
|
"brand": "MEMLNaut",
|
||||||
"author": "MEML",
|
"author": "MEML",
|
||||||
|
|
|
||||||
75
vcv/src/LedRing.hpp
Normal file
75
vcv/src/LedRing.hpp
Normal file
|
|
@ -0,0 +1,75 @@
|
||||||
|
// LedRing.hpp — Custom LED-ring widget encircling each output jack.
|
||||||
|
//
|
||||||
|
// The arc fills clockwise from 12 o'clock in proportion to the output's value
|
||||||
|
// (0..1 → 0..2π) and is drawn on drawLayer() layer 1 (so it stays bright when
|
||||||
|
// room brightness is lowered, per the VCV custom-light convention). A dim track
|
||||||
|
// ring sits underneath. Colour comes from the frontend design tokens
|
||||||
|
// (palette.hpp) so the module matches the browser.
|
||||||
|
//
|
||||||
|
// Templated on the module type so it can read the per-output value without a
|
||||||
|
// hard include cycle; MEMLNaut.cpp instantiates LedRingWidget<MEMLNaut>.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <rack.hpp>
|
||||||
|
#include "palette.hpp"
|
||||||
|
|
||||||
|
using namespace rack;
|
||||||
|
|
||||||
|
template<typename TModule>
|
||||||
|
struct LedRingWidget : Widget {
|
||||||
|
TModule* module = nullptr;
|
||||||
|
int outIdx = 0;
|
||||||
|
NVGcolor ringColor = memlnaut::palette::accent();
|
||||||
|
float radius = 9.f; // px around a PJ301M jack
|
||||||
|
|
||||||
|
LedRingWidget() {
|
||||||
|
// Box large enough to host the ring around a ~22px jack.
|
||||||
|
box.size = Vec(radius * 2.f + 6.f, radius * 2.f + 6.f);
|
||||||
|
}
|
||||||
|
|
||||||
|
float valueOf() const {
|
||||||
|
if (!module) return 0.f;
|
||||||
|
return clamp(module->ringValue(outIdx), 0.f, 1.f);
|
||||||
|
}
|
||||||
|
|
||||||
|
void drawLayer(const DrawArgs& args, int layer) override {
|
||||||
|
if (layer != 1) {
|
||||||
|
Widget::drawLayer(args, layer);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
float v = valueOf();
|
||||||
|
Vec c = box.size.div(2.f);
|
||||||
|
const float start = -M_PI / 2.f; // 12 o'clock
|
||||||
|
const float end = start + 2.f * M_PI; // full circle
|
||||||
|
|
||||||
|
// Dim track ring (full circle).
|
||||||
|
nvgBeginPath(args.vg);
|
||||||
|
nvgArc(args.vg, c.x, c.y, radius, start, end, NVG_CW);
|
||||||
|
nvgStrokeColor(args.vg, nvgRGBA((unsigned char)(ringColor.r * 255),
|
||||||
|
(unsigned char)(ringColor.g * 255),
|
||||||
|
(unsigned char)(ringColor.b * 255), 40));
|
||||||
|
nvgStrokeWidth(args.vg, 1.4f);
|
||||||
|
nvgStroke(args.vg);
|
||||||
|
|
||||||
|
// Proportional value arc.
|
||||||
|
if (v > 0.001f) {
|
||||||
|
nvgBeginPath(args.vg);
|
||||||
|
nvgArc(args.vg, c.x, c.y, radius, start, start + v * 2.f * M_PI, NVG_CW);
|
||||||
|
nvgStrokeColor(args.vg, ringColor);
|
||||||
|
nvgStrokeWidth(args.vg, 1.9f);
|
||||||
|
nvgLineCap(args.vg, NVG_ROUND);
|
||||||
|
nvgStroke(args.vg);
|
||||||
|
|
||||||
|
// Soft glow halo — the frontend "glow not shadow" signature.
|
||||||
|
nvgGlobalCompositeOperation(args.vg, NVG_LIGHTER);
|
||||||
|
nvgBeginPath(args.vg);
|
||||||
|
nvgArc(args.vg, c.x, c.y, radius, start, start + v * 2.f * M_PI, NVG_CW);
|
||||||
|
nvgStrokeColor(args.vg, nvgRGBAf(ringColor.r, ringColor.g, ringColor.b, 0.25f * v));
|
||||||
|
nvgStrokeWidth(args.vg, 4.0f);
|
||||||
|
nvgStroke(args.vg);
|
||||||
|
nvgGlobalCompositeOperation(args.vg, NVG_SOURCE_OVER);
|
||||||
|
}
|
||||||
|
|
||||||
|
Widget::drawLayer(args, layer);
|
||||||
|
}
|
||||||
|
};
|
||||||
File diff suppressed because it is too large
Load diff
409
vcv/src/iml.hpp
Normal file
409
vcv/src/iml.hpp
Normal file
|
|
@ -0,0 +1,409 @@
|
||||||
|
// iml.hpp — Self-contained runtime IML/MLP for the MEMLNaut VCV module.
|
||||||
|
//
|
||||||
|
// This is a VENDORED, runtime-shaped re-implementation of the nisps core
|
||||||
|
// `nisps::IML<float>` / `nisps::MLP<float>` surface the VCV module relies on.
|
||||||
|
// The retired `nisps-core` header tree (`-I../nisps-core/include`) is gone, and
|
||||||
|
// the templated firmware/WASM `nisps/ml` core is fixed-size — neither is a clean
|
||||||
|
// fit for a runtime 8→16 module. So we ship a small native MLP here that matches
|
||||||
|
// the firmware/browser TRAINING SEMANTICS as closely as a runtime form allows:
|
||||||
|
//
|
||||||
|
// • ReLU hidden layers, sigmoid output (sigmoid maps to [0,1]).
|
||||||
|
// • A trailing bias node (1.0) appended to the input vector.
|
||||||
|
// • spread-aware weight init: uniform [-1,1] (spread=0) → Xavier 1/√fan_in
|
||||||
|
// (spread=1), interpolated per layer.
|
||||||
|
// • spread-aware perturbation (RL "move weights"): flat noise (spread=0) →
|
||||||
|
// per-layer Xavier-scaled noise + 10%·spread weight decay (spread=1).
|
||||||
|
// • Plain SGD / MSE training over the example dataset.
|
||||||
|
// • DETERMINISTIC per-instance RNG (seeded), so behaviour is reproducible and
|
||||||
|
// the threading double-buffer stays race-free (each MLP owns its own RNG).
|
||||||
|
//
|
||||||
|
// It is NOT bit-identical to the firmware core (different optimiser internals),
|
||||||
|
// and that divergence is an accepted follow-up (see vcv/SPEC.md delta #5). The
|
||||||
|
// public method names mirror the core so `MEMLNaut.cpp` is unchanged in spirit.
|
||||||
|
//
|
||||||
|
// MPL-2.0 in spirit with the rest of nisps; wrapper code under the VCV module's
|
||||||
|
// licence. British spelling in comments where it reads naturally.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <vector>
|
||||||
|
#include <cstddef>
|
||||||
|
#include <cmath>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <limits>
|
||||||
|
#include <algorithm>
|
||||||
|
|
||||||
|
namespace nisps {
|
||||||
|
|
||||||
|
// ── Tiny deterministic RNG (xorshift128) ──────────────────────────────
|
||||||
|
// Per-instance state; seeded in the constructor. No std::random_device, no
|
||||||
|
// shared global generator — this is what keeps the audio/worker MLP pair free
|
||||||
|
// of data races and makes parity reproducible.
|
||||||
|
class DetRng {
|
||||||
|
public:
|
||||||
|
explicit DetRng(uint32_t seed = 0x1234567u) { reseed(seed); }
|
||||||
|
void reseed(uint32_t seed) {
|
||||||
|
s_[0] = seed ? seed : 0xA5A5A5A5u;
|
||||||
|
s_[1] = s_[0] ^ 0x9E3779B9u;
|
||||||
|
s_[2] = s_[0] * 0x85EBCA6Bu + 1u;
|
||||||
|
s_[3] = s_[0] * 0xC2B2AE35u + 0x27D4EB2Fu;
|
||||||
|
}
|
||||||
|
uint32_t next_u32() {
|
||||||
|
uint32_t t = s_[3];
|
||||||
|
uint32_t const u = s_[0];
|
||||||
|
s_[3] = s_[2]; s_[2] = s_[1]; s_[1] = u;
|
||||||
|
t ^= t << 11;
|
||||||
|
t ^= t >> 8;
|
||||||
|
s_[0] = t ^ u ^ (u >> 19);
|
||||||
|
return s_[0];
|
||||||
|
}
|
||||||
|
// Uniform in [0,1)
|
||||||
|
float uniform01() { return (next_u32() >> 8) * (1.0f / 16777216.0f); }
|
||||||
|
// Uniform in [-1,1)
|
||||||
|
float uniform_pm1() { return uniform01() * 2.0f - 1.0f; }
|
||||||
|
// Approx standard-normal: sum of 3 uniforms (matches the core's gen_randn shape)
|
||||||
|
float gaussian() {
|
||||||
|
return (uniform_pm1() + uniform_pm1() + uniform_pm1()) * 0.5773502692f; // /√3 → unit-ish variance
|
||||||
|
}
|
||||||
|
private:
|
||||||
|
uint32_t s_[4];
|
||||||
|
};
|
||||||
|
|
||||||
|
// ── MLP ───────────────────────────────────────────────────────────────
|
||||||
|
template<typename T = float>
|
||||||
|
class MLP {
|
||||||
|
public:
|
||||||
|
// 3D weight store: [layer][node][weight] where the final weight per node is
|
||||||
|
// the bias (the previous layer's activations get a trailing 1.0).
|
||||||
|
using mlp_weights = std::vector<std::vector<std::vector<T>>>;
|
||||||
|
|
||||||
|
// layers_nodes: full sizes including input (with bias) and output, e.g.
|
||||||
|
// {n_in + 1, h0, h1, h2, n_out}
|
||||||
|
MLP(const std::vector<size_t>& layers_nodes, uint32_t seed)
|
||||||
|
: layers_nodes_(layers_nodes), rng_(seed) {
|
||||||
|
build_();
|
||||||
|
draw_weights_spread_(static_cast<T>(0));
|
||||||
|
}
|
||||||
|
|
||||||
|
size_t num_layers() const { return weights_.size(); }
|
||||||
|
|
||||||
|
// Forward pass. `input_with_bias` has the trailing 1.0 already appended.
|
||||||
|
void forward(const std::vector<T>& input_with_bias, std::vector<T>& out) const {
|
||||||
|
std::vector<T> act = input_with_bias;
|
||||||
|
for (size_t l = 0; l < weights_.size(); ++l) {
|
||||||
|
const auto& layer = weights_[l];
|
||||||
|
const bool is_output = (l + 1 == weights_.size());
|
||||||
|
std::vector<T> next(layer.size());
|
||||||
|
for (size_t n = 0; n < layer.size(); ++n) {
|
||||||
|
const auto& w = layer[n];
|
||||||
|
T sum = 0;
|
||||||
|
// w has act.size()+1 entries? No: w spans the *current* input
|
||||||
|
// size (which already includes the bias slot of `act`).
|
||||||
|
const size_t lim = std::min(w.size(), act.size());
|
||||||
|
for (size_t k = 0; k < lim; ++k) sum += w[k] * act[k];
|
||||||
|
next[n] = is_output ? sigmoid_(sum) : relu_(sum);
|
||||||
|
}
|
||||||
|
// For hidden layers, append a bias term for the next layer's input.
|
||||||
|
if (!is_output) next.push_back(static_cast<T>(1));
|
||||||
|
act = std::move(next);
|
||||||
|
}
|
||||||
|
out = std::move(act);
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── spread-aware weight init ──────────────────────────────────────
|
||||||
|
void draw_weights_spread(T spread) { draw_weights_spread_(spread); }
|
||||||
|
|
||||||
|
// ── spread-aware perturbation (RL move_weights) ───────────────────
|
||||||
|
void move_weights_spread(T speed, T spread) {
|
||||||
|
const T decay = static_cast<T>(1) - static_cast<T>(0.1) * spread;
|
||||||
|
for (size_t l = 0; l < weights_.size(); ++l) {
|
||||||
|
const T fan_in = static_cast<T>(input_size_of_layer_(l));
|
||||||
|
const T xavier = (fan_in > 0) ? static_cast<T>(1) / std::sqrt(fan_in) : static_cast<T>(1);
|
||||||
|
// spread=0 → flat noise (scale 1); spread=1 → per-layer Xavier scale
|
||||||
|
const T noiseScale = (static_cast<T>(1) - spread) + spread * xavier;
|
||||||
|
for (auto& node : weights_[l]) {
|
||||||
|
for (auto& w : node) {
|
||||||
|
if (spread > 0) w *= decay; // weight decay only when spread>0
|
||||||
|
w += rng_.gaussian() * speed * noiseScale;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── plain SGD / MSE training ──────────────────────────────────────
|
||||||
|
// features: each row is input WITHOUT bias; labels: target outputs in [0,1].
|
||||||
|
void train(const std::vector<std::vector<T>>& features,
|
||||||
|
const std::vector<std::vector<T>>& labels,
|
||||||
|
int max_iterations, T learning_rate, T convergence) {
|
||||||
|
const size_t n = std::min(features.size(), labels.size());
|
||||||
|
if (n == 0) return;
|
||||||
|
for (int iter = 0; iter < max_iterations; ++iter) {
|
||||||
|
T epoch_loss = 0;
|
||||||
|
for (size_t s = 0; s < n; ++s) {
|
||||||
|
std::vector<T> in = features[s];
|
||||||
|
in.push_back(static_cast<T>(1)); // bias
|
||||||
|
epoch_loss += backprop_(in, labels[s], learning_rate);
|
||||||
|
}
|
||||||
|
epoch_loss /= static_cast<T>(n);
|
||||||
|
if (epoch_loss < convergence) break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
mlp_weights get_weights() const { return weights_; }
|
||||||
|
void set_weights(const mlp_weights& w) {
|
||||||
|
// Only adopt if the topology matches; otherwise ignore (keeps the audio
|
||||||
|
// path safe against malformed snapshots from the bridge / patch files).
|
||||||
|
if (w.size() != weights_.size()) return;
|
||||||
|
for (size_t l = 0; l < w.size(); ++l) {
|
||||||
|
if (w[l].size() != weights_[l].size()) return;
|
||||||
|
}
|
||||||
|
weights_ = w;
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
static T relu_(T x) { return x > 0 ? x : 0; }
|
||||||
|
static T sigmoid_(T x) { return static_cast<T>(1) / (static_cast<T>(1) + std::exp(-x)); }
|
||||||
|
static T dsigmoid_from_out_(T y) { return y * (static_cast<T>(1) - y); }
|
||||||
|
|
||||||
|
size_t input_size_of_layer_(size_t l) const {
|
||||||
|
// The number of weights per node in layer l (incl. bias slot).
|
||||||
|
return weights_[l].empty() ? 0 : weights_[l][0].size();
|
||||||
|
}
|
||||||
|
|
||||||
|
void build_() {
|
||||||
|
weights_.clear();
|
||||||
|
// layers_nodes_[0] is the input layer WITH bias already counted.
|
||||||
|
for (size_t l = 1; l < layers_nodes_.size(); ++l) {
|
||||||
|
const size_t in_sz = layers_nodes_[l - 1]; // includes bias slot
|
||||||
|
const size_t out_sz = layers_nodes_[l];
|
||||||
|
std::vector<std::vector<T>> layer(out_sz, std::vector<T>(in_sz, 0));
|
||||||
|
weights_.push_back(std::move(layer));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
void draw_weights_spread_(T spread) {
|
||||||
|
for (size_t l = 0; l < weights_.size(); ++l) {
|
||||||
|
const T fan_in = static_cast<T>(layers_nodes_[l]); // incl. bias
|
||||||
|
const T xavier = (fan_in > 0) ? static_cast<T>(1) / std::sqrt(fan_in) : static_cast<T>(1);
|
||||||
|
const T scale = (static_cast<T>(1) - spread) + spread * xavier;
|
||||||
|
for (auto& node : weights_[l]) {
|
||||||
|
for (size_t k = 0; k < node.size(); ++k) {
|
||||||
|
// bias (last weight) initialised to 0, like the core
|
||||||
|
const bool is_bias = (k + 1 == node.size());
|
||||||
|
node[k] = is_bias ? static_cast<T>(0) : rng_.uniform_pm1() * scale;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// One SGD step on a single example; returns the MSE for this example.
|
||||||
|
T backprop_(const std::vector<T>& in_with_bias, const std::vector<T>& target,
|
||||||
|
T lr) {
|
||||||
|
// Forward, caching activations per layer.
|
||||||
|
std::vector<std::vector<T>> acts;
|
||||||
|
acts.reserve(weights_.size() + 1);
|
||||||
|
acts.push_back(in_with_bias);
|
||||||
|
std::vector<T> act = in_with_bias;
|
||||||
|
for (size_t l = 0; l < weights_.size(); ++l) {
|
||||||
|
const auto& layer = weights_[l];
|
||||||
|
const bool is_output = (l + 1 == weights_.size());
|
||||||
|
std::vector<T> next(layer.size());
|
||||||
|
for (size_t nidx = 0; nidx < layer.size(); ++nidx) {
|
||||||
|
const auto& w = layer[nidx];
|
||||||
|
T sum = 0;
|
||||||
|
const size_t lim = std::min(w.size(), act.size());
|
||||||
|
for (size_t k = 0; k < lim; ++k) sum += w[k] * act[k];
|
||||||
|
next[nidx] = is_output ? sigmoid_(sum) : relu_(sum);
|
||||||
|
}
|
||||||
|
if (!is_output) next.push_back(static_cast<T>(1));
|
||||||
|
acts.push_back(next);
|
||||||
|
act = next;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Output error.
|
||||||
|
const size_t L = weights_.size();
|
||||||
|
std::vector<T>& out = acts[L];
|
||||||
|
T loss = 0;
|
||||||
|
std::vector<T> delta(out.size());
|
||||||
|
for (size_t o = 0; o < out.size(); ++o) {
|
||||||
|
const T t = (o < target.size()) ? target[o] : static_cast<T>(0);
|
||||||
|
const T e = out[o] - t;
|
||||||
|
loss += e * e;
|
||||||
|
delta[o] = e * dsigmoid_from_out_(out[o]); // MSE × sigmoid'
|
||||||
|
}
|
||||||
|
loss /= static_cast<T>(out.size() ? out.size() : 1);
|
||||||
|
|
||||||
|
// Backprop through layers L-1 .. 0.
|
||||||
|
std::vector<T> nextDelta;
|
||||||
|
for (size_t li = L; li-- > 0;) {
|
||||||
|
const auto& prevAct = acts[li]; // input activations to layer li
|
||||||
|
auto& layer = weights_[li];
|
||||||
|
const bool is_output = (li + 1 == L);
|
||||||
|
// Compute delta to propagate to the previous layer (excludes bias node).
|
||||||
|
const size_t prevSize = prevAct.size(); // includes bias slot
|
||||||
|
std::vector<T> propagate(prevSize, 0);
|
||||||
|
for (size_t nidx = 0; nidx < layer.size(); ++nidx) {
|
||||||
|
const T d = delta[nidx];
|
||||||
|
auto& w = layer[nidx];
|
||||||
|
const size_t lim = std::min(w.size(), prevSize);
|
||||||
|
for (size_t k = 0; k < lim; ++k) {
|
||||||
|
propagate[k] += d * w[k];
|
||||||
|
w[k] -= lr * d * prevAct[k]; // gradient step
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Turn `propagate` into next-layer delta via ReLU' (skip for input).
|
||||||
|
if (li > 0) {
|
||||||
|
const auto& actPrev = acts[li]; // activations of layer li-1's output
|
||||||
|
nextDelta.assign(actPrev.size(), 0);
|
||||||
|
for (size_t k = 0; k < actPrev.size(); ++k) {
|
||||||
|
const T relud = actPrev[k] > 0 ? static_cast<T>(1) : static_cast<T>(0);
|
||||||
|
nextDelta[k] = propagate[k] * relud;
|
||||||
|
}
|
||||||
|
// Drop the trailing bias slot's delta (it has no upstream weights).
|
||||||
|
if (!nextDelta.empty()) nextDelta.pop_back();
|
||||||
|
delta = nextDelta;
|
||||||
|
}
|
||||||
|
(void)is_output;
|
||||||
|
}
|
||||||
|
return loss;
|
||||||
|
}
|
||||||
|
|
||||||
|
std::vector<size_t> layers_nodes_;
|
||||||
|
mlp_weights weights_;
|
||||||
|
mutable DetRng rng_;
|
||||||
|
};
|
||||||
|
|
||||||
|
// ── Dataset (FIFO ring, max 100 examples) ─────────────────────────────
|
||||||
|
template<typename T = float>
|
||||||
|
class Dataset {
|
||||||
|
public:
|
||||||
|
static constexpr size_t kMax_examples = 100;
|
||||||
|
void add(const std::vector<T>& feat, const std::vector<T>& label) {
|
||||||
|
if (features_.size() >= kMax_examples) {
|
||||||
|
features_.erase(features_.begin());
|
||||||
|
labels_.erase(labels_.begin());
|
||||||
|
}
|
||||||
|
features_.push_back(feat);
|
||||||
|
labels_.push_back(label);
|
||||||
|
}
|
||||||
|
void clear() { features_.clear(); labels_.clear(); }
|
||||||
|
size_t count() const { return features_.size(); }
|
||||||
|
const std::vector<std::vector<T>>& features() const { return features_; }
|
||||||
|
const std::vector<std::vector<T>>& labels() const { return labels_; }
|
||||||
|
void load(const std::vector<std::vector<T>>& f, const std::vector<std::vector<T>>& l) {
|
||||||
|
clear();
|
||||||
|
const size_t n = std::min(f.size(), l.size());
|
||||||
|
for (size_t i = 0; i < n; ++i) add(f[i], l[i]);
|
||||||
|
}
|
||||||
|
private:
|
||||||
|
std::vector<std::vector<T>> features_;
|
||||||
|
std::vector<std::vector<T>> labels_;
|
||||||
|
};
|
||||||
|
|
||||||
|
// ── IML ───────────────────────────────────────────────────────────────
|
||||||
|
template<typename Float = float>
|
||||||
|
class IML {
|
||||||
|
public:
|
||||||
|
enum class Mode { Inference, Training };
|
||||||
|
|
||||||
|
IML(size_t n_inputs, size_t n_outputs,
|
||||||
|
std::vector<size_t> hidden_layers = {16, 24, 16},
|
||||||
|
size_t max_iterations = 200,
|
||||||
|
Float learning_rate = static_cast<Float>(0.1),
|
||||||
|
Float convergence_threshold = static_cast<Float>(0.00001),
|
||||||
|
uint32_t seed = 0xC0FFEEu)
|
||||||
|
: n_inputs_(n_inputs), n_outputs_(n_outputs),
|
||||||
|
max_iterations_(max_iterations), learning_rate_(learning_rate),
|
||||||
|
convergence_threshold_(convergence_threshold) {
|
||||||
|
std::vector<size_t> sizes;
|
||||||
|
sizes.push_back(n_inputs_ + 1); // + bias
|
||||||
|
for (size_t h : hidden_layers) sizes.push_back(h);
|
||||||
|
sizes.push_back(n_outputs_);
|
||||||
|
mlp_ = std::make_unique<MLP<Float>>(sizes, seed);
|
||||||
|
input_state_.assign(n_inputs_, static_cast<Float>(0.5));
|
||||||
|
output_state_.assign(n_outputs_, static_cast<Float>(0));
|
||||||
|
}
|
||||||
|
|
||||||
|
size_t num_inputs() const { return n_inputs_; }
|
||||||
|
size_t num_outputs() const { return n_outputs_; }
|
||||||
|
|
||||||
|
void set_input(size_t i, Float v) {
|
||||||
|
if (i >= n_inputs_) return;
|
||||||
|
input_state_[i] = std::clamp(v, static_cast<Float>(0), static_cast<Float>(1));
|
||||||
|
input_updated_ = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
const Float* get_outputs() const { return output_state_.data(); }
|
||||||
|
|
||||||
|
void process() {
|
||||||
|
if (!input_updated_) return;
|
||||||
|
std::vector<Float> in = input_state_;
|
||||||
|
in.push_back(static_cast<Float>(1));
|
||||||
|
mlp_->forward(in, output_state_);
|
||||||
|
if (output_state_.size() < n_outputs_) output_state_.resize(n_outputs_, 0);
|
||||||
|
input_updated_ = false;
|
||||||
|
}
|
||||||
|
|
||||||
|
void set_mode(Mode m) {
|
||||||
|
if (m == Mode::Inference && mode_ == Mode::Training) train_();
|
||||||
|
mode_ = m;
|
||||||
|
}
|
||||||
|
Mode get_mode() const { return mode_; }
|
||||||
|
|
||||||
|
void add_example(const Float* inputs, size_t n_in, const Float* outputs, size_t n_out) {
|
||||||
|
std::vector<Float> in(inputs, inputs + std::min(n_in, n_inputs_));
|
||||||
|
in.resize(n_inputs_, static_cast<Float>(0));
|
||||||
|
std::vector<Float> out(outputs, outputs + std::min(n_out, n_outputs_));
|
||||||
|
out.resize(n_outputs_, static_cast<Float>(0));
|
||||||
|
dataset_.add(in, out);
|
||||||
|
}
|
||||||
|
void clear_dataset() { dataset_.clear(); }
|
||||||
|
|
||||||
|
void randomise_weights(Float spread) { mlp_->draw_weights_spread(spread); refresh_(); }
|
||||||
|
void move_weights(Float speed, Float spread) { mlp_->move_weights_spread(speed, spread); refresh_(); }
|
||||||
|
|
||||||
|
typename MLP<Float>::mlp_weights get_weights() const { return mlp_->get_weights(); }
|
||||||
|
void set_weights(typename MLP<Float>::mlp_weights& w) { mlp_->set_weights(w); }
|
||||||
|
|
||||||
|
size_t get_example_count() const { return dataset_.count(); }
|
||||||
|
size_t get_max_examples() const { return Dataset<Float>::kMax_examples; }
|
||||||
|
std::vector<std::vector<Float>> get_example_features() const { return dataset_.features(); }
|
||||||
|
std::vector<std::vector<Float>> get_example_labels() const { return dataset_.labels(); }
|
||||||
|
void load_examples(const std::vector<std::vector<Float>>& f,
|
||||||
|
const std::vector<std::vector<Float>>& l) { dataset_.load(f, l); }
|
||||||
|
|
||||||
|
Float nearest_example_distance(const Float* input, size_t n_in) const {
|
||||||
|
const auto& feats = dataset_.features();
|
||||||
|
if (feats.empty()) return static_cast<Float>(-1);
|
||||||
|
Float best = std::numeric_limits<Float>::max();
|
||||||
|
const size_t dims = std::min(n_in, n_inputs_);
|
||||||
|
for (const auto& f : feats) {
|
||||||
|
Float d = 0;
|
||||||
|
for (size_t k = 0; k < dims && k < f.size(); ++k) {
|
||||||
|
const Float diff = f[k] - input[k];
|
||||||
|
d += diff * diff;
|
||||||
|
}
|
||||||
|
best = std::min(best, std::sqrt(d));
|
||||||
|
}
|
||||||
|
return best;
|
||||||
|
}
|
||||||
|
|
||||||
|
private:
|
||||||
|
void refresh_() { input_updated_ = true; process(); }
|
||||||
|
void train_() {
|
||||||
|
if (dataset_.count() == 0) return;
|
||||||
|
mlp_->train(dataset_.features(), dataset_.labels(),
|
||||||
|
static_cast<int>(max_iterations_), learning_rate_,
|
||||||
|
convergence_threshold_);
|
||||||
|
refresh_();
|
||||||
|
}
|
||||||
|
|
||||||
|
size_t n_inputs_, n_outputs_, max_iterations_;
|
||||||
|
Float learning_rate_, convergence_threshold_;
|
||||||
|
Mode mode_ = Mode::Inference;
|
||||||
|
bool input_updated_ = false;
|
||||||
|
std::vector<Float> input_state_, output_state_;
|
||||||
|
Dataset<Float> dataset_;
|
||||||
|
std::unique_ptr<MLP<Float>> mlp_;
|
||||||
|
};
|
||||||
|
|
||||||
|
} // namespace nisps
|
||||||
|
|
@ -136,6 +136,7 @@ inline std::vector<uint8_t> messageString(const std::string& address,
|
||||||
class OscServer {
|
class OscServer {
|
||||||
public:
|
public:
|
||||||
using StringCallback = std::function<void(const std::string&)>;
|
using StringCallback = std::function<void(const std::string&)>;
|
||||||
|
using FloatVecCallback = std::function<void(const std::vector<float>&)>;
|
||||||
|
|
||||||
OscServer() = default;
|
OscServer() = default;
|
||||||
~OscServer() { stop(); }
|
~OscServer() { stop(); }
|
||||||
|
|
@ -144,9 +145,15 @@ public:
|
||||||
OscServer(const OscServer&) = delete;
|
OscServer(const OscServer&) = delete;
|
||||||
OscServer& operator=(const OscServer&) = delete;
|
OscServer& operator=(const OscServer&) = delete;
|
||||||
|
|
||||||
// Register handlers before starting
|
// Register handlers before starting.
|
||||||
|
// onState — full JSON state snapshot (/nisps/state <s>)
|
||||||
|
// onWeights — weights-only JSON (/nisps/weights <s>)
|
||||||
|
// onInput — live input vector (browser drives the model) (/nisps/input <f…f>)
|
||||||
|
// onFeedback— verdict op JSON (thumbs/place/rand/clear) (/nisps/feedback <s>)
|
||||||
void onState(StringCallback cb) { stateCallback_ = std::move(cb); }
|
void onState(StringCallback cb) { stateCallback_ = std::move(cb); }
|
||||||
void onWeights(StringCallback cb) { weightsCallback_ = std::move(cb); }
|
void onWeights(StringCallback cb) { weightsCallback_ = std::move(cb); }
|
||||||
|
void onInput(FloatVecCallback cb) { inputCallback_ = std::move(cb); }
|
||||||
|
void onFeedback(StringCallback cb) { feedbackCallback_ = std::move(cb); }
|
||||||
|
|
||||||
// Set the target address for sending (where the webapp bridge listens).
|
// Set the target address for sending (where the webapp bridge listens).
|
||||||
// Default: 127.0.0.1:9001
|
// Default: 127.0.0.1:9001
|
||||||
|
|
@ -240,12 +247,24 @@ public:
|
||||||
sendPacket(msg);
|
sendPacket(msg);
|
||||||
}
|
}
|
||||||
|
|
||||||
// Send current input values (2 floats)
|
// Send current input values (N floats)
|
||||||
void sendInputs(const float* values, size_t count) {
|
void sendInputs(const float* values, size_t count) {
|
||||||
auto msg = osc::messageFloats("/nisps/input", values, count);
|
auto msg = osc::messageFloats("/nisps/input", values, count);
|
||||||
sendPacket(msg);
|
sendPacket(msg);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Send a full JSON state snapshot (module → browser).
|
||||||
|
void sendState(const std::string& json) {
|
||||||
|
auto msg = osc::messageString("/nisps/state", json);
|
||||||
|
sendPacket(msg);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Send weights-only JSON (module → browser).
|
||||||
|
void sendWeights(const std::string& json) {
|
||||||
|
auto msg = osc::messageString("/nisps/weights", json);
|
||||||
|
sendPacket(msg);
|
||||||
|
}
|
||||||
|
|
||||||
private:
|
private:
|
||||||
void recvLoop() {
|
void recvLoop() {
|
||||||
uint8_t buf[65536];
|
uint8_t buf[65536];
|
||||||
|
|
@ -291,6 +310,21 @@ private:
|
||||||
std::string payload = osc::readString(buf, len, offset);
|
std::string payload = osc::readString(buf, len, offset);
|
||||||
if (weightsCallback_) weightsCallback_(payload);
|
if (weightsCallback_) weightsCallback_(payload);
|
||||||
}
|
}
|
||||||
|
} else if (address == "/nisps/feedback") {
|
||||||
|
// Verdict op as a JSON string:
|
||||||
|
// {"op":"up|down|rand|clear","spread":f,"input":[…],"output":[…]}
|
||||||
|
if (tags.size() >= 2 && tags[1] == 's') {
|
||||||
|
std::string payload = osc::readString(buf, len, offset);
|
||||||
|
if (feedbackCallback_) feedbackCallback_(payload);
|
||||||
|
}
|
||||||
|
} else if (address == "/nisps/input") {
|
||||||
|
// Live input vector from the browser → drive the model inputs.
|
||||||
|
std::vector<float> values;
|
||||||
|
for (size_t i = 1; i < tags.size(); ++i) {
|
||||||
|
if (tags[i] == 'f') values.push_back(osc::readFloat(buf, len, offset));
|
||||||
|
else break;
|
||||||
|
}
|
||||||
|
if (!values.empty() && inputCallback_) inputCallback_(values);
|
||||||
}
|
}
|
||||||
// Unknown addresses are silently ignored
|
// Unknown addresses are silently ignored
|
||||||
}
|
}
|
||||||
|
|
@ -337,6 +371,8 @@ private:
|
||||||
// Callbacks
|
// Callbacks
|
||||||
StringCallback stateCallback_;
|
StringCallback stateCallback_;
|
||||||
StringCallback weightsCallback_;
|
StringCallback weightsCallback_;
|
||||||
|
StringCallback feedbackCallback_;
|
||||||
|
FloatVecCallback inputCallback_;
|
||||||
|
|
||||||
// Send target
|
// Send target
|
||||||
std::mutex sendMutex_;
|
std::mutex sendMutex_;
|
||||||
|
|
|
||||||
52
vcv/src/palette.hpp
Normal file
52
vcv/src/palette.hpp
Normal file
|
|
@ -0,0 +1,52 @@
|
||||||
|
// palette.hpp — MEMLNaut ring colours, hand-synced from the frontend tokens.
|
||||||
|
//
|
||||||
|
// Source of truth: docs/redesign/manifold-export/tokens/colors.css. These are
|
||||||
|
// the exact hex values from that file, so the VCV module's LED rings read as the
|
||||||
|
// same instrument as the Manifold browser front-end. If a token changes there,
|
||||||
|
// update the matching constant here (small hand-sync; see SPEC delta #3).
|
||||||
|
//
|
||||||
|
// The 16 output rings are assigned across the design-token accents + group/pin
|
||||||
|
// colours as a clean orange→cyan-anchored ramp, so outputs in the same mode
|
||||||
|
// group glow the same colour as the browser heatmap/Console grouping.
|
||||||
|
#pragma once
|
||||||
|
|
||||||
|
#include <rack.hpp>
|
||||||
|
|
||||||
|
namespace memlnaut {
|
||||||
|
namespace palette {
|
||||||
|
|
||||||
|
// ── Design tokens (colors.css) ────────────────────────────────────────
|
||||||
|
inline NVGcolor accent() { return nvgRGB(0xff, 0x6a, 0x00); } // --accent warm primary
|
||||||
|
inline NVGcolor accent2() { return nvgRGB(0x00, 0xcc, 0xff); } // --accent-2 cool secondary
|
||||||
|
inline NVGcolor accent3() { return nvgRGB(0xff, 0xa8, 0x60); } // --accent-3 warm hover/tint
|
||||||
|
inline NVGcolor good() { return nvgRGB(0x6b, 0xc2, 0x6b); } // --good
|
||||||
|
inline NVGcolor warn() { return nvgRGB(0xf5, 0xc4, 0x5e); } // --warn
|
||||||
|
inline NVGcolor info() { return nvgRGB(0x5b, 0x9e, 0xef); } // --info
|
||||||
|
inline NVGcolor pin3() { return nvgRGB(0xb4, 0x64, 0xff); } // --pin-3 base (violet)
|
||||||
|
inline NVGcolor danger() { return nvgRGB(0xff, 0x44, 0x66); } // --danger (bipolar / perturbed)
|
||||||
|
inline NVGcolor bgTrack() { return nvgRGB(0x24, 0x24, 0x24); } // --bg-3 (ring track)
|
||||||
|
|
||||||
|
// ── 16-ring palette ───────────────────────────────────────────────────
|
||||||
|
// Groups cycle through the token accents/group colours. Outputs 0..15 read as a
|
||||||
|
// coherent orange→cyan family with the semantic accents woven in.
|
||||||
|
inline NVGcolor ring(int outIdx) {
|
||||||
|
static const NVGcolor kRing[16] = {
|
||||||
|
// group 0 — formant/primary (orange family)
|
||||||
|
nvgRGB(0xff, 0x6a, 0x00), nvgRGB(0xff, 0x82, 0x2a), nvgRGB(0xff, 0xa8, 0x60), nvgRGB(0xff, 0xc4, 0x90),
|
||||||
|
// group 1 — amp (green)
|
||||||
|
nvgRGB(0x6b, 0xc2, 0x6b), nvgRGB(0x86, 0xcf, 0x86),
|
||||||
|
// group 2 — filter (amber/warn)
|
||||||
|
nvgRGB(0xf5, 0xc4, 0x5e), nvgRGB(0xf8, 0xd4, 0x84),
|
||||||
|
// group 3 — mod (violet/pin-3)
|
||||||
|
nvgRGB(0xb4, 0x64, 0xff), nvgRGB(0xc6, 0x86, 0xff),
|
||||||
|
// group 4 — fx (blue/info)
|
||||||
|
nvgRGB(0x5b, 0x9e, 0xef), nvgRGB(0x82, 0xb6, 0xf3),
|
||||||
|
// group 5 — pitch/data (cyan family → accent-2)
|
||||||
|
nvgRGB(0x3a, 0xd0, 0xf0), nvgRGB(0x1d, 0xce, 0xf7), nvgRGB(0x00, 0xcc, 0xff), nvgRGB(0x55, 0xdd, 0xff),
|
||||||
|
};
|
||||||
|
if (outIdx < 0) outIdx = 0;
|
||||||
|
return kRing[outIdx % 16];
|
||||||
|
}
|
||||||
|
|
||||||
|
} // namespace palette
|
||||||
|
} // namespace memlnaut
|
||||||
Loading…
Reference in a new issue