feat(vcv): evolve module to 8x16 + LED rings + token palette + WS-OSC bridge
8 inputs x 16 outputs; per-output LED ring widget (drawLayer+nvgArc); palette from frontend tokens; OSC bridge verbs for bidirectional browser training; vendored self-contained iml.hpp (retired nisps-core); compiles against Rack SDK 2.6.4. See SPEC.md BUILD DELTAS.
This commit is contained in:
monkey-w1n5t0n
parent
0d8179d6d5
commit
fbc68ebfab
8 changed files with 1005 additions and 370 deletions
|
|
@ -2,7 +2,9 @@ RACK_DIR ?= $(HOME)/.local/share/Rack2/Rack-SDK
|
|||
|
||||
FLAGS += -std=c++20
|
||||
FLAGS += -I$(RACK_DIR)/include -I$(RACK_DIR)/dep/include
|
||||
FLAGS += -I../nisps-core/include
|
||||
# The retired nisps-core header tree is gone; the runtime IML/MLP is vendored
|
||||
# self-contained in src/iml.hpp (see that file's header for the rationale).
|
||||
FLAGS += -Isrc
|
||||
|
||||
SOURCES += src/plugin.cpp
|
||||
SOURCES += src/MEMLNaut.cpp
|
||||
|
|
|
|||
46
vcv/SPEC.md
46
vcv/SPEC.md
|
|
@ -1,5 +1,51 @@
|
|||
# MEMLNaut VCV Rack Module — Specification
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ BUILD DELTAS (2026-06-28) — AUTHORITATIVE OVERRIDES
|
||||
|
||||
These supersede any conflicting detail below. They reflect the Manifold mission + the locked decisions in
|
||||
`docs/redesign/BUILD-PLAN.md` and `docs/redesign/backends-spec.md`. Build to THESE.
|
||||
|
||||
1. **I/O = 8 inputs × 16 outputs** (was 2→12). `NUM_ML_INPUTS = 8`, `NUM_ML_OUTPUTS = 16`. The IML is sized
|
||||
8→16 (a runtime-shaped native MLP is fine here — the module is C++, not the fixed WASM target). The 8 CV
|
||||
inputs feed the model's input dims; the 16 CV outputs are the model's inference outputs (the modular N×M
|
||||
envelope). Keep the control inputs (Spread CV, Learn gate, + / − triggers).
|
||||
2. **LED RING around EACH of the 16 outputs** — a custom ring widget encircling each output jack whose arc
|
||||
fills in proportion to that output's value (0..1 → 0..2π). Draw on `drawLayer()` layer 1 with `nvgArc` for
|
||||
the proportional fill + a dim track ring. Each ring is COLOURED from a palette that MATCHES the frontend
|
||||
design tokens.
|
||||
3. **Palette from the frontend tokens** — generate `vcv/src/palette.hpp` from
|
||||
`docs/redesign/manifold-export/tokens/colors.css`: `--accent #ff6a00` (orange), `--accent-2 #00ccff` (cyan),
|
||||
and the group colours (formant→accent, pitch→accent-2, amp→`--good #6bc26b`, filter→`--warn #f5c45e`,
|
||||
fx→`--info #5b9eef`, mod→`--accent-3 #ffa860`). Assign the 16 rings across these group colours (or a clean
|
||||
16-step ramp between orange and cyan) so the module reads as the same instrument as the browser. A tiny
|
||||
hand-written `palette.hpp` is acceptable (no build-time codegen needed).
|
||||
4. **Browser ↔ VCV bridge = WS↔OSC** (locked transport). The module runs its OSC server (`src/osc_server.hpp`
|
||||
already exists — evolve it). The browser's OSC backend (`manifold/src/backends/osc-backend.ts`) sends over a
|
||||
WebSocket to the Deno bridge (`manifold/osc-bridge/`), which relays UDP-OSC to the module. **Bidirectional
|
||||
training**: drive + train the module FROM the browser (the verdict loop + example-placing over the bridge)
|
||||
AND from the module's own panel (+/− buttons, Learn gate, triggers). OSC verbs to support both directions:
|
||||
`/nisps/input` (drive), `/nisps/output` (module→browser viz), `/nisps/feedback` (thumbs up/down + place),
|
||||
`/nisps/weights`, `/nisps/examples`, `/nisps/state`. Pick a fixed default UDP port (e.g. 7001) + per-instance
|
||||
offset; the Deno bridge maps `ws://localhost:8765` ↔ that UDP port.
|
||||
5. **Core include path** — the Makefile's `-I../nisps-core/include` points at the RETIRED `nisps-core`. Repoint
|
||||
to the current core (`../nisps/`) OR vendor a minimal runtime IML inside `vcv/src/`. Goal: get it COMPILING
|
||||
with an 8→16 runtime MLP that shares the firmware/browser training semantics (spread-aware draw/move_weights,
|
||||
deterministic RNG) as closely as the native runtime-shaped form allows. If full nisps/ml reuse is blocked by
|
||||
the templated fixed-size API, keep a self-contained IML in the module and note the alignment as a follow-up.
|
||||
6. **Derived outputs** (Mean/Std/Delta/Novelty/Confidence) → move to the optional EXPANDER or a context-menu
|
||||
toggle; the headline is 16 raw outputs + their LED rings. Do not let them crowd the 16-jack panel.
|
||||
7. **Build** needs the VCV Rack 2 SDK (`RACK_DIR`, default `$HOME/.local/share/Rack2/Rack-SDK`, NOT installed).
|
||||
The build step must fetch the Linux Rack-SDK zip from vcvrack.com, set `RACK_DIR`, and `make` to verify the
|
||||
plugin compiles. Panel SVGs in `vcv/res/` exist (MEMLNaut.svg / -wide / -expander) — widen/relayout for 16
|
||||
outputs + rings as needed.
|
||||
|
||||
The rest of this document is the prior (2→12) design — useful for threading, persistence, RL workflow, and
|
||||
panel/build mechanics, but the I/O counts, LED rings, palette, bridge, and core path above WIN.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
A VCV Rack module that embeds the NISPS interactive ML engine (nisps-core C++ library) as a CV-to-CV mapper. Users explore high-dimensional parameter spaces via reinforcement learning feedback, producing 12 raw CV outputs and 5 derived meta-signals from configurable CV inputs.
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
{
|
||||
"slug": "MEMLNaut",
|
||||
"name": "MEMLNaut",
|
||||
"version": "0.1.0",
|
||||
"version": "0.2.0",
|
||||
"license": "proprietary",
|
||||
"brand": "MEMLNaut",
|
||||
"author": "MEML",
|
||||
|
|
|
|||
75
vcv/src/LedRing.hpp
Normal file
75
vcv/src/LedRing.hpp
Normal file
|
|
@ -0,0 +1,75 @@
|
|||
// LedRing.hpp — Custom LED-ring widget encircling each output jack.
|
||||
//
|
||||
// The arc fills clockwise from 12 o'clock in proportion to the output's value
|
||||
// (0..1 → 0..2π) and is drawn on drawLayer() layer 1 (so it stays bright when
|
||||
// room brightness is lowered, per the VCV custom-light convention). A dim track
|
||||
// ring sits underneath. Colour comes from the frontend design tokens
|
||||
// (palette.hpp) so the module matches the browser.
|
||||
//
|
||||
// Templated on the module type so it can read the per-output value without a
|
||||
// hard include cycle; MEMLNaut.cpp instantiates LedRingWidget<MEMLNaut>.
|
||||
#pragma once
|
||||
|
||||
#include <rack.hpp>
|
||||
#include "palette.hpp"
|
||||
|
||||
using namespace rack;
|
||||
|
||||
template<typename TModule>
|
||||
struct LedRingWidget : Widget {
|
||||
TModule* module = nullptr;
|
||||
int outIdx = 0;
|
||||
NVGcolor ringColor = memlnaut::palette::accent();
|
||||
float radius = 9.f; // px around a PJ301M jack
|
||||
|
||||
LedRingWidget() {
|
||||
// Box large enough to host the ring around a ~22px jack.
|
||||
box.size = Vec(radius * 2.f + 6.f, radius * 2.f + 6.f);
|
||||
}
|
||||
|
||||
float valueOf() const {
|
||||
if (!module) return 0.f;
|
||||
return clamp(module->ringValue(outIdx), 0.f, 1.f);
|
||||
}
|
||||
|
||||
void drawLayer(const DrawArgs& args, int layer) override {
|
||||
if (layer != 1) {
|
||||
Widget::drawLayer(args, layer);
|
||||
return;
|
||||
}
|
||||
float v = valueOf();
|
||||
Vec c = box.size.div(2.f);
|
||||
const float start = -M_PI / 2.f; // 12 o'clock
|
||||
const float end = start + 2.f * M_PI; // full circle
|
||||
|
||||
// Dim track ring (full circle).
|
||||
nvgBeginPath(args.vg);
|
||||
nvgArc(args.vg, c.x, c.y, radius, start, end, NVG_CW);
|
||||
nvgStrokeColor(args.vg, nvgRGBA((unsigned char)(ringColor.r * 255),
|
||||
(unsigned char)(ringColor.g * 255),
|
||||
(unsigned char)(ringColor.b * 255), 40));
|
||||
nvgStrokeWidth(args.vg, 1.4f);
|
||||
nvgStroke(args.vg);
|
||||
|
||||
// Proportional value arc.
|
||||
if (v > 0.001f) {
|
||||
nvgBeginPath(args.vg);
|
||||
nvgArc(args.vg, c.x, c.y, radius, start, start + v * 2.f * M_PI, NVG_CW);
|
||||
nvgStrokeColor(args.vg, ringColor);
|
||||
nvgStrokeWidth(args.vg, 1.9f);
|
||||
nvgLineCap(args.vg, NVG_ROUND);
|
||||
nvgStroke(args.vg);
|
||||
|
||||
// Soft glow halo — the frontend "glow not shadow" signature.
|
||||
nvgGlobalCompositeOperation(args.vg, NVG_LIGHTER);
|
||||
nvgBeginPath(args.vg);
|
||||
nvgArc(args.vg, c.x, c.y, radius, start, start + v * 2.f * M_PI, NVG_CW);
|
||||
nvgStrokeColor(args.vg, nvgRGBAf(ringColor.r, ringColor.g, ringColor.b, 0.25f * v));
|
||||
nvgStrokeWidth(args.vg, 4.0f);
|
||||
nvgStroke(args.vg);
|
||||
nvgGlobalCompositeOperation(args.vg, NVG_SOURCE_OVER);
|
||||
}
|
||||
|
||||
Widget::drawLayer(args, layer);
|
||||
}
|
||||
};
|
||||
File diff suppressed because it is too large
Load diff
409
vcv/src/iml.hpp
Normal file
409
vcv/src/iml.hpp
Normal file
|
|
@ -0,0 +1,409 @@
|
|||
// iml.hpp — Self-contained runtime IML/MLP for the MEMLNaut VCV module.
|
||||
//
|
||||
// This is a VENDORED, runtime-shaped re-implementation of the nisps core
|
||||
// `nisps::IML<float>` / `nisps::MLP<float>` surface the VCV module relies on.
|
||||
// The retired `nisps-core` header tree (`-I../nisps-core/include`) is gone, and
|
||||
// the templated firmware/WASM `nisps/ml` core is fixed-size — neither is a clean
|
||||
// fit for a runtime 8→16 module. So we ship a small native MLP here that matches
|
||||
// the firmware/browser TRAINING SEMANTICS as closely as a runtime form allows:
|
||||
//
|
||||
// • ReLU hidden layers, sigmoid output (sigmoid maps to [0,1]).
|
||||
// • A trailing bias node (1.0) appended to the input vector.
|
||||
// • spread-aware weight init: uniform [-1,1] (spread=0) → Xavier 1/√fan_in
|
||||
// (spread=1), interpolated per layer.
|
||||
// • spread-aware perturbation (RL "move weights"): flat noise (spread=0) →
|
||||
// per-layer Xavier-scaled noise + 10%·spread weight decay (spread=1).
|
||||
// • Plain SGD / MSE training over the example dataset.
|
||||
// • DETERMINISTIC per-instance RNG (seeded), so behaviour is reproducible and
|
||||
// the threading double-buffer stays race-free (each MLP owns its own RNG).
|
||||
//
|
||||
// It is NOT bit-identical to the firmware core (different optimiser internals),
|
||||
// and that divergence is an accepted follow-up (see vcv/SPEC.md delta #5). The
|
||||
// public method names mirror the core so `MEMLNaut.cpp` is unchanged in spirit.
|
||||
//
|
||||
// MPL-2.0 in spirit with the rest of nisps; wrapper code under the VCV module's
|
||||
// licence. British spelling in comments where it reads naturally.
|
||||
#pragma once
|
||||
|
||||
#include <vector>
|
||||
#include <cstddef>
|
||||
#include <cmath>
|
||||
#include <cstdint>
|
||||
#include <limits>
|
||||
#include <algorithm>
|
||||
|
||||
namespace nisps {
|
||||
|
||||
// ── Tiny deterministic RNG (xorshift128) ──────────────────────────────
|
||||
// Per-instance state; seeded in the constructor. No std::random_device, no
|
||||
// shared global generator — this is what keeps the audio/worker MLP pair free
|
||||
// of data races and makes parity reproducible.
|
||||
class DetRng {
|
||||
public:
|
||||
explicit DetRng(uint32_t seed = 0x1234567u) { reseed(seed); }
|
||||
void reseed(uint32_t seed) {
|
||||
s_[0] = seed ? seed : 0xA5A5A5A5u;
|
||||
s_[1] = s_[0] ^ 0x9E3779B9u;
|
||||
s_[2] = s_[0] * 0x85EBCA6Bu + 1u;
|
||||
s_[3] = s_[0] * 0xC2B2AE35u + 0x27D4EB2Fu;
|
||||
}
|
||||
uint32_t next_u32() {
|
||||
uint32_t t = s_[3];
|
||||
uint32_t const u = s_[0];
|
||||
s_[3] = s_[2]; s_[2] = s_[1]; s_[1] = u;
|
||||
t ^= t << 11;
|
||||
t ^= t >> 8;
|
||||
s_[0] = t ^ u ^ (u >> 19);
|
||||
return s_[0];
|
||||
}
|
||||
// Uniform in [0,1)
|
||||
float uniform01() { return (next_u32() >> 8) * (1.0f / 16777216.0f); }
|
||||
// Uniform in [-1,1)
|
||||
float uniform_pm1() { return uniform01() * 2.0f - 1.0f; }
|
||||
// Approx standard-normal: sum of 3 uniforms (matches the core's gen_randn shape)
|
||||
float gaussian() {
|
||||
return (uniform_pm1() + uniform_pm1() + uniform_pm1()) * 0.5773502692f; // /√3 → unit-ish variance
|
||||
}
|
||||
private:
|
||||
uint32_t s_[4];
|
||||
};
|
||||
|
||||
// ── MLP ───────────────────────────────────────────────────────────────
|
||||
template<typename T = float>
|
||||
class MLP {
|
||||
public:
|
||||
// 3D weight store: [layer][node][weight] where the final weight per node is
|
||||
// the bias (the previous layer's activations get a trailing 1.0).
|
||||
using mlp_weights = std::vector<std::vector<std::vector<T>>>;
|
||||
|
||||
// layers_nodes: full sizes including input (with bias) and output, e.g.
|
||||
// {n_in + 1, h0, h1, h2, n_out}
|
||||
MLP(const std::vector<size_t>& layers_nodes, uint32_t seed)
|
||||
: layers_nodes_(layers_nodes), rng_(seed) {
|
||||
build_();
|
||||
draw_weights_spread_(static_cast<T>(0));
|
||||
}
|
||||
|
||||
size_t num_layers() const { return weights_.size(); }
|
||||
|
||||
// Forward pass. `input_with_bias` has the trailing 1.0 already appended.
|
||||
void forward(const std::vector<T>& input_with_bias, std::vector<T>& out) const {
|
||||
std::vector<T> act = input_with_bias;
|
||||
for (size_t l = 0; l < weights_.size(); ++l) {
|
||||
const auto& layer = weights_[l];
|
||||
const bool is_output = (l + 1 == weights_.size());
|
||||
std::vector<T> next(layer.size());
|
||||
for (size_t n = 0; n < layer.size(); ++n) {
|
||||
const auto& w = layer[n];
|
||||
T sum = 0;
|
||||
// w has act.size()+1 entries? No: w spans the *current* input
|
||||
// size (which already includes the bias slot of `act`).
|
||||
const size_t lim = std::min(w.size(), act.size());
|
||||
for (size_t k = 0; k < lim; ++k) sum += w[k] * act[k];
|
||||
next[n] = is_output ? sigmoid_(sum) : relu_(sum);
|
||||
}
|
||||
// For hidden layers, append a bias term for the next layer's input.
|
||||
if (!is_output) next.push_back(static_cast<T>(1));
|
||||
act = std::move(next);
|
||||
}
|
||||
out = std::move(act);
|
||||
}
|
||||
|
||||
// ── spread-aware weight init ──────────────────────────────────────
|
||||
void draw_weights_spread(T spread) { draw_weights_spread_(spread); }
|
||||
|
||||
// ── spread-aware perturbation (RL move_weights) ───────────────────
|
||||
void move_weights_spread(T speed, T spread) {
|
||||
const T decay = static_cast<T>(1) - static_cast<T>(0.1) * spread;
|
||||
for (size_t l = 0; l < weights_.size(); ++l) {
|
||||
const T fan_in = static_cast<T>(input_size_of_layer_(l));
|
||||
const T xavier = (fan_in > 0) ? static_cast<T>(1) / std::sqrt(fan_in) : static_cast<T>(1);
|
||||
// spread=0 → flat noise (scale 1); spread=1 → per-layer Xavier scale
|
||||
const T noiseScale = (static_cast<T>(1) - spread) + spread * xavier;
|
||||
for (auto& node : weights_[l]) {
|
||||
for (auto& w : node) {
|
||||
if (spread > 0) w *= decay; // weight decay only when spread>0
|
||||
w += rng_.gaussian() * speed * noiseScale;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ── plain SGD / MSE training ──────────────────────────────────────
|
||||
// features: each row is input WITHOUT bias; labels: target outputs in [0,1].
|
||||
void train(const std::vector<std::vector<T>>& features,
|
||||
const std::vector<std::vector<T>>& labels,
|
||||
int max_iterations, T learning_rate, T convergence) {
|
||||
const size_t n = std::min(features.size(), labels.size());
|
||||
if (n == 0) return;
|
||||
for (int iter = 0; iter < max_iterations; ++iter) {
|
||||
T epoch_loss = 0;
|
||||
for (size_t s = 0; s < n; ++s) {
|
||||
std::vector<T> in = features[s];
|
||||
in.push_back(static_cast<T>(1)); // bias
|
||||
epoch_loss += backprop_(in, labels[s], learning_rate);
|
||||
}
|
||||
epoch_loss /= static_cast<T>(n);
|
||||
if (epoch_loss < convergence) break;
|
||||
}
|
||||
}
|
||||
|
||||
mlp_weights get_weights() const { return weights_; }
|
||||
void set_weights(const mlp_weights& w) {
|
||||
// Only adopt if the topology matches; otherwise ignore (keeps the audio
|
||||
// path safe against malformed snapshots from the bridge / patch files).
|
||||
if (w.size() != weights_.size()) return;
|
||||
for (size_t l = 0; l < w.size(); ++l) {
|
||||
if (w[l].size() != weights_[l].size()) return;
|
||||
}
|
||||
weights_ = w;
|
||||
}
|
||||
|
||||
private:
|
||||
static T relu_(T x) { return x > 0 ? x : 0; }
|
||||
static T sigmoid_(T x) { return static_cast<T>(1) / (static_cast<T>(1) + std::exp(-x)); }
|
||||
static T dsigmoid_from_out_(T y) { return y * (static_cast<T>(1) - y); }
|
||||
|
||||
size_t input_size_of_layer_(size_t l) const {
|
||||
// The number of weights per node in layer l (incl. bias slot).
|
||||
return weights_[l].empty() ? 0 : weights_[l][0].size();
|
||||
}
|
||||
|
||||
void build_() {
|
||||
weights_.clear();
|
||||
// layers_nodes_[0] is the input layer WITH bias already counted.
|
||||
for (size_t l = 1; l < layers_nodes_.size(); ++l) {
|
||||
const size_t in_sz = layers_nodes_[l - 1]; // includes bias slot
|
||||
const size_t out_sz = layers_nodes_[l];
|
||||
std::vector<std::vector<T>> layer(out_sz, std::vector<T>(in_sz, 0));
|
||||
weights_.push_back(std::move(layer));
|
||||
}
|
||||
}
|
||||
|
||||
void draw_weights_spread_(T spread) {
|
||||
for (size_t l = 0; l < weights_.size(); ++l) {
|
||||
const T fan_in = static_cast<T>(layers_nodes_[l]); // incl. bias
|
||||
const T xavier = (fan_in > 0) ? static_cast<T>(1) / std::sqrt(fan_in) : static_cast<T>(1);
|
||||
const T scale = (static_cast<T>(1) - spread) + spread * xavier;
|
||||
for (auto& node : weights_[l]) {
|
||||
for (size_t k = 0; k < node.size(); ++k) {
|
||||
// bias (last weight) initialised to 0, like the core
|
||||
const bool is_bias = (k + 1 == node.size());
|
||||
node[k] = is_bias ? static_cast<T>(0) : rng_.uniform_pm1() * scale;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// One SGD step on a single example; returns the MSE for this example.
|
||||
T backprop_(const std::vector<T>& in_with_bias, const std::vector<T>& target,
|
||||
T lr) {
|
||||
// Forward, caching activations per layer.
|
||||
std::vector<std::vector<T>> acts;
|
||||
acts.reserve(weights_.size() + 1);
|
||||
acts.push_back(in_with_bias);
|
||||
std::vector<T> act = in_with_bias;
|
||||
for (size_t l = 0; l < weights_.size(); ++l) {
|
||||
const auto& layer = weights_[l];
|
||||
const bool is_output = (l + 1 == weights_.size());
|
||||
std::vector<T> next(layer.size());
|
||||
for (size_t nidx = 0; nidx < layer.size(); ++nidx) {
|
||||
const auto& w = layer[nidx];
|
||||
T sum = 0;
|
||||
const size_t lim = std::min(w.size(), act.size());
|
||||
for (size_t k = 0; k < lim; ++k) sum += w[k] * act[k];
|
||||
next[nidx] = is_output ? sigmoid_(sum) : relu_(sum);
|
||||
}
|
||||
if (!is_output) next.push_back(static_cast<T>(1));
|
||||
acts.push_back(next);
|
||||
act = next;
|
||||
}
|
||||
|
||||
// Output error.
|
||||
const size_t L = weights_.size();
|
||||
std::vector<T>& out = acts[L];
|
||||
T loss = 0;
|
||||
std::vector<T> delta(out.size());
|
||||
for (size_t o = 0; o < out.size(); ++o) {
|
||||
const T t = (o < target.size()) ? target[o] : static_cast<T>(0);
|
||||
const T e = out[o] - t;
|
||||
loss += e * e;
|
||||
delta[o] = e * dsigmoid_from_out_(out[o]); // MSE × sigmoid'
|
||||
}
|
||||
loss /= static_cast<T>(out.size() ? out.size() : 1);
|
||||
|
||||
// Backprop through layers L-1 .. 0.
|
||||
std::vector<T> nextDelta;
|
||||
for (size_t li = L; li-- > 0;) {
|
||||
const auto& prevAct = acts[li]; // input activations to layer li
|
||||
auto& layer = weights_[li];
|
||||
const bool is_output = (li + 1 == L);
|
||||
// Compute delta to propagate to the previous layer (excludes bias node).
|
||||
const size_t prevSize = prevAct.size(); // includes bias slot
|
||||
std::vector<T> propagate(prevSize, 0);
|
||||
for (size_t nidx = 0; nidx < layer.size(); ++nidx) {
|
||||
const T d = delta[nidx];
|
||||
auto& w = layer[nidx];
|
||||
const size_t lim = std::min(w.size(), prevSize);
|
||||
for (size_t k = 0; k < lim; ++k) {
|
||||
propagate[k] += d * w[k];
|
||||
w[k] -= lr * d * prevAct[k]; // gradient step
|
||||
}
|
||||
}
|
||||
// Turn `propagate` into next-layer delta via ReLU' (skip for input).
|
||||
if (li > 0) {
|
||||
const auto& actPrev = acts[li]; // activations of layer li-1's output
|
||||
nextDelta.assign(actPrev.size(), 0);
|
||||
for (size_t k = 0; k < actPrev.size(); ++k) {
|
||||
const T relud = actPrev[k] > 0 ? static_cast<T>(1) : static_cast<T>(0);
|
||||
nextDelta[k] = propagate[k] * relud;
|
||||
}
|
||||
// Drop the trailing bias slot's delta (it has no upstream weights).
|
||||
if (!nextDelta.empty()) nextDelta.pop_back();
|
||||
delta = nextDelta;
|
||||
}
|
||||
(void)is_output;
|
||||
}
|
||||
return loss;
|
||||
}
|
||||
|
||||
std::vector<size_t> layers_nodes_;
|
||||
mlp_weights weights_;
|
||||
mutable DetRng rng_;
|
||||
};
|
||||
|
||||
// ── Dataset (FIFO ring, max 100 examples) ─────────────────────────────
|
||||
template<typename T = float>
|
||||
class Dataset {
|
||||
public:
|
||||
static constexpr size_t kMax_examples = 100;
|
||||
void add(const std::vector<T>& feat, const std::vector<T>& label) {
|
||||
if (features_.size() >= kMax_examples) {
|
||||
features_.erase(features_.begin());
|
||||
labels_.erase(labels_.begin());
|
||||
}
|
||||
features_.push_back(feat);
|
||||
labels_.push_back(label);
|
||||
}
|
||||
void clear() { features_.clear(); labels_.clear(); }
|
||||
size_t count() const { return features_.size(); }
|
||||
const std::vector<std::vector<T>>& features() const { return features_; }
|
||||
const std::vector<std::vector<T>>& labels() const { return labels_; }
|
||||
void load(const std::vector<std::vector<T>>& f, const std::vector<std::vector<T>>& l) {
|
||||
clear();
|
||||
const size_t n = std::min(f.size(), l.size());
|
||||
for (size_t i = 0; i < n; ++i) add(f[i], l[i]);
|
||||
}
|
||||
private:
|
||||
std::vector<std::vector<T>> features_;
|
||||
std::vector<std::vector<T>> labels_;
|
||||
};
|
||||
|
||||
// ── IML ───────────────────────────────────────────────────────────────
|
||||
template<typename Float = float>
|
||||
class IML {
|
||||
public:
|
||||
enum class Mode { Inference, Training };
|
||||
|
||||
IML(size_t n_inputs, size_t n_outputs,
|
||||
std::vector<size_t> hidden_layers = {16, 24, 16},
|
||||
size_t max_iterations = 200,
|
||||
Float learning_rate = static_cast<Float>(0.1),
|
||||
Float convergence_threshold = static_cast<Float>(0.00001),
|
||||
uint32_t seed = 0xC0FFEEu)
|
||||
: n_inputs_(n_inputs), n_outputs_(n_outputs),
|
||||
max_iterations_(max_iterations), learning_rate_(learning_rate),
|
||||
convergence_threshold_(convergence_threshold) {
|
||||
std::vector<size_t> sizes;
|
||||
sizes.push_back(n_inputs_ + 1); // + bias
|
||||
for (size_t h : hidden_layers) sizes.push_back(h);
|
||||
sizes.push_back(n_outputs_);
|
||||
mlp_ = std::make_unique<MLP<Float>>(sizes, seed);
|
||||
input_state_.assign(n_inputs_, static_cast<Float>(0.5));
|
||||
output_state_.assign(n_outputs_, static_cast<Float>(0));
|
||||
}
|
||||
|
||||
size_t num_inputs() const { return n_inputs_; }
|
||||
size_t num_outputs() const { return n_outputs_; }
|
||||
|
||||
void set_input(size_t i, Float v) {
|
||||
if (i >= n_inputs_) return;
|
||||
input_state_[i] = std::clamp(v, static_cast<Float>(0), static_cast<Float>(1));
|
||||
input_updated_ = true;
|
||||
}
|
||||
|
||||
const Float* get_outputs() const { return output_state_.data(); }
|
||||
|
||||
void process() {
|
||||
if (!input_updated_) return;
|
||||
std::vector<Float> in = input_state_;
|
||||
in.push_back(static_cast<Float>(1));
|
||||
mlp_->forward(in, output_state_);
|
||||
if (output_state_.size() < n_outputs_) output_state_.resize(n_outputs_, 0);
|
||||
input_updated_ = false;
|
||||
}
|
||||
|
||||
void set_mode(Mode m) {
|
||||
if (m == Mode::Inference && mode_ == Mode::Training) train_();
|
||||
mode_ = m;
|
||||
}
|
||||
Mode get_mode() const { return mode_; }
|
||||
|
||||
void add_example(const Float* inputs, size_t n_in, const Float* outputs, size_t n_out) {
|
||||
std::vector<Float> in(inputs, inputs + std::min(n_in, n_inputs_));
|
||||
in.resize(n_inputs_, static_cast<Float>(0));
|
||||
std::vector<Float> out(outputs, outputs + std::min(n_out, n_outputs_));
|
||||
out.resize(n_outputs_, static_cast<Float>(0));
|
||||
dataset_.add(in, out);
|
||||
}
|
||||
void clear_dataset() { dataset_.clear(); }
|
||||
|
||||
void randomise_weights(Float spread) { mlp_->draw_weights_spread(spread); refresh_(); }
|
||||
void move_weights(Float speed, Float spread) { mlp_->move_weights_spread(speed, spread); refresh_(); }
|
||||
|
||||
typename MLP<Float>::mlp_weights get_weights() const { return mlp_->get_weights(); }
|
||||
void set_weights(typename MLP<Float>::mlp_weights& w) { mlp_->set_weights(w); }
|
||||
|
||||
size_t get_example_count() const { return dataset_.count(); }
|
||||
size_t get_max_examples() const { return Dataset<Float>::kMax_examples; }
|
||||
std::vector<std::vector<Float>> get_example_features() const { return dataset_.features(); }
|
||||
std::vector<std::vector<Float>> get_example_labels() const { return dataset_.labels(); }
|
||||
void load_examples(const std::vector<std::vector<Float>>& f,
|
||||
const std::vector<std::vector<Float>>& l) { dataset_.load(f, l); }
|
||||
|
||||
Float nearest_example_distance(const Float* input, size_t n_in) const {
|
||||
const auto& feats = dataset_.features();
|
||||
if (feats.empty()) return static_cast<Float>(-1);
|
||||
Float best = std::numeric_limits<Float>::max();
|
||||
const size_t dims = std::min(n_in, n_inputs_);
|
||||
for (const auto& f : feats) {
|
||||
Float d = 0;
|
||||
for (size_t k = 0; k < dims && k < f.size(); ++k) {
|
||||
const Float diff = f[k] - input[k];
|
||||
d += diff * diff;
|
||||
}
|
||||
best = std::min(best, std::sqrt(d));
|
||||
}
|
||||
return best;
|
||||
}
|
||||
|
||||
private:
|
||||
void refresh_() { input_updated_ = true; process(); }
|
||||
void train_() {
|
||||
if (dataset_.count() == 0) return;
|
||||
mlp_->train(dataset_.features(), dataset_.labels(),
|
||||
static_cast<int>(max_iterations_), learning_rate_,
|
||||
convergence_threshold_);
|
||||
refresh_();
|
||||
}
|
||||
|
||||
size_t n_inputs_, n_outputs_, max_iterations_;
|
||||
Float learning_rate_, convergence_threshold_;
|
||||
Mode mode_ = Mode::Inference;
|
||||
bool input_updated_ = false;
|
||||
std::vector<Float> input_state_, output_state_;
|
||||
Dataset<Float> dataset_;
|
||||
std::unique_ptr<MLP<Float>> mlp_;
|
||||
};
|
||||
|
||||
} // namespace nisps
|
||||
|
|
@ -136,6 +136,7 @@ inline std::vector<uint8_t> messageString(const std::string& address,
|
|||
class OscServer {
|
||||
public:
|
||||
using StringCallback = std::function<void(const std::string&)>;
|
||||
using FloatVecCallback = std::function<void(const std::vector<float>&)>;
|
||||
|
||||
OscServer() = default;
|
||||
~OscServer() { stop(); }
|
||||
|
|
@ -144,9 +145,15 @@ public:
|
|||
OscServer(const OscServer&) = delete;
|
||||
OscServer& operator=(const OscServer&) = delete;
|
||||
|
||||
// Register handlers before starting
|
||||
// Register handlers before starting.
|
||||
// onState — full JSON state snapshot (/nisps/state <s>)
|
||||
// onWeights — weights-only JSON (/nisps/weights <s>)
|
||||
// onInput — live input vector (browser drives the model) (/nisps/input <f…f>)
|
||||
// onFeedback— verdict op JSON (thumbs/place/rand/clear) (/nisps/feedback <s>)
|
||||
void onState(StringCallback cb) { stateCallback_ = std::move(cb); }
|
||||
void onWeights(StringCallback cb) { weightsCallback_ = std::move(cb); }
|
||||
void onInput(FloatVecCallback cb) { inputCallback_ = std::move(cb); }
|
||||
void onFeedback(StringCallback cb) { feedbackCallback_ = std::move(cb); }
|
||||
|
||||
// Set the target address for sending (where the webapp bridge listens).
|
||||
// Default: 127.0.0.1:9001
|
||||
|
|
@ -240,12 +247,24 @@ public:
|
|||
sendPacket(msg);
|
||||
}
|
||||
|
||||
// Send current input values (2 floats)
|
||||
// Send current input values (N floats)
|
||||
void sendInputs(const float* values, size_t count) {
|
||||
auto msg = osc::messageFloats("/nisps/input", values, count);
|
||||
sendPacket(msg);
|
||||
}
|
||||
|
||||
// Send a full JSON state snapshot (module → browser).
|
||||
void sendState(const std::string& json) {
|
||||
auto msg = osc::messageString("/nisps/state", json);
|
||||
sendPacket(msg);
|
||||
}
|
||||
|
||||
// Send weights-only JSON (module → browser).
|
||||
void sendWeights(const std::string& json) {
|
||||
auto msg = osc::messageString("/nisps/weights", json);
|
||||
sendPacket(msg);
|
||||
}
|
||||
|
||||
private:
|
||||
void recvLoop() {
|
||||
uint8_t buf[65536];
|
||||
|
|
@ -291,6 +310,21 @@ private:
|
|||
std::string payload = osc::readString(buf, len, offset);
|
||||
if (weightsCallback_) weightsCallback_(payload);
|
||||
}
|
||||
} else if (address == "/nisps/feedback") {
|
||||
// Verdict op as a JSON string:
|
||||
// {"op":"up|down|rand|clear","spread":f,"input":[…],"output":[…]}
|
||||
if (tags.size() >= 2 && tags[1] == 's') {
|
||||
std::string payload = osc::readString(buf, len, offset);
|
||||
if (feedbackCallback_) feedbackCallback_(payload);
|
||||
}
|
||||
} else if (address == "/nisps/input") {
|
||||
// Live input vector from the browser → drive the model inputs.
|
||||
std::vector<float> values;
|
||||
for (size_t i = 1; i < tags.size(); ++i) {
|
||||
if (tags[i] == 'f') values.push_back(osc::readFloat(buf, len, offset));
|
||||
else break;
|
||||
}
|
||||
if (!values.empty() && inputCallback_) inputCallback_(values);
|
||||
}
|
||||
// Unknown addresses are silently ignored
|
||||
}
|
||||
|
|
@ -337,6 +371,8 @@ private:
|
|||
// Callbacks
|
||||
StringCallback stateCallback_;
|
||||
StringCallback weightsCallback_;
|
||||
StringCallback feedbackCallback_;
|
||||
FloatVecCallback inputCallback_;
|
||||
|
||||
// Send target
|
||||
std::mutex sendMutex_;
|
||||
|
|
|
|||
52
vcv/src/palette.hpp
Normal file
52
vcv/src/palette.hpp
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
// palette.hpp — MEMLNaut ring colours, hand-synced from the frontend tokens.
|
||||
//
|
||||
// Source of truth: docs/redesign/manifold-export/tokens/colors.css. These are
|
||||
// the exact hex values from that file, so the VCV module's LED rings read as the
|
||||
// same instrument as the Manifold browser front-end. If a token changes there,
|
||||
// update the matching constant here (small hand-sync; see SPEC delta #3).
|
||||
//
|
||||
// The 16 output rings are assigned across the design-token accents + group/pin
|
||||
// colours as a clean orange→cyan-anchored ramp, so outputs in the same mode
|
||||
// group glow the same colour as the browser heatmap/Console grouping.
|
||||
#pragma once
|
||||
|
||||
#include <rack.hpp>
|
||||
|
||||
namespace memlnaut {
|
||||
namespace palette {
|
||||
|
||||
// ── Design tokens (colors.css) ────────────────────────────────────────
|
||||
inline NVGcolor accent() { return nvgRGB(0xff, 0x6a, 0x00); } // --accent warm primary
|
||||
inline NVGcolor accent2() { return nvgRGB(0x00, 0xcc, 0xff); } // --accent-2 cool secondary
|
||||
inline NVGcolor accent3() { return nvgRGB(0xff, 0xa8, 0x60); } // --accent-3 warm hover/tint
|
||||
inline NVGcolor good() { return nvgRGB(0x6b, 0xc2, 0x6b); } // --good
|
||||
inline NVGcolor warn() { return nvgRGB(0xf5, 0xc4, 0x5e); } // --warn
|
||||
inline NVGcolor info() { return nvgRGB(0x5b, 0x9e, 0xef); } // --info
|
||||
inline NVGcolor pin3() { return nvgRGB(0xb4, 0x64, 0xff); } // --pin-3 base (violet)
|
||||
inline NVGcolor danger() { return nvgRGB(0xff, 0x44, 0x66); } // --danger (bipolar / perturbed)
|
||||
inline NVGcolor bgTrack() { return nvgRGB(0x24, 0x24, 0x24); } // --bg-3 (ring track)
|
||||
|
||||
// ── 16-ring palette ───────────────────────────────────────────────────
|
||||
// Groups cycle through the token accents/group colours. Outputs 0..15 read as a
|
||||
// coherent orange→cyan family with the semantic accents woven in.
|
||||
inline NVGcolor ring(int outIdx) {
|
||||
static const NVGcolor kRing[16] = {
|
||||
// group 0 — formant/primary (orange family)
|
||||
nvgRGB(0xff, 0x6a, 0x00), nvgRGB(0xff, 0x82, 0x2a), nvgRGB(0xff, 0xa8, 0x60), nvgRGB(0xff, 0xc4, 0x90),
|
||||
// group 1 — amp (green)
|
||||
nvgRGB(0x6b, 0xc2, 0x6b), nvgRGB(0x86, 0xcf, 0x86),
|
||||
// group 2 — filter (amber/warn)
|
||||
nvgRGB(0xf5, 0xc4, 0x5e), nvgRGB(0xf8, 0xd4, 0x84),
|
||||
// group 3 — mod (violet/pin-3)
|
||||
nvgRGB(0xb4, 0x64, 0xff), nvgRGB(0xc6, 0x86, 0xff),
|
||||
// group 4 — fx (blue/info)
|
||||
nvgRGB(0x5b, 0x9e, 0xef), nvgRGB(0x82, 0xb6, 0xf3),
|
||||
// group 5 — pitch/data (cyan family → accent-2)
|
||||
nvgRGB(0x3a, 0xd0, 0xf0), nvgRGB(0x1d, 0xce, 0xf7), nvgRGB(0x00, 0xcc, 0xff), nvgRGB(0x55, 0xdd, 0xff),
|
||||
};
|
||||
if (outIdx < 0) outIdx = 0;
|
||||
return kRing[outIdx % 16];
|
||||
}
|
||||
|
||||
} // namespace palette
|
||||
} // namespace memlnaut
|
||||
Loading…
Reference in a new issue