memlnaut-nisps/vcv/README.md

MEMLNaut for VCV Rack

MEMLNaut is a CV-to-CV mapper powered by a neural network that you train in real time using reinforcement learning. Patch any CV sources into its 8 inputs, connect its 16 outputs to your synth parameters, and shape the mapping by giving thumbs-up/thumbs-down feedback. The module learns your preferences, producing complex, nonlinear modulation that evolves with your taste. It does not generate sound itself -- it generates control voltages.

Installation

MEMLNaut is built from source against the VCV Rack 2 SDK. Pre-built .vcvplugin bundles are published at https://meml.lnfinitemonkeys.org/next/vcv/ (locally-built bundles land in the git-ignored dist/; see DISTRIBUTION.md).

Requirements

• VCV Rack 2 (Community Edition or Pro)
• VCV Rack SDK (v2.x)
• C++20 compiler (GCC 10+, Clang 12+, or MSVC 2019+)
• GNU Make

Build

git clone --recursive https://github.com/monkey-w1n5t0n/MEMLNaut-NISPS.git
cd MEMLNaut-NISPS/vcv

# Point to your Rack SDK (or set in environment)
export RACK_DIR=/path/to/Rack-SDK

make
make install   # copies plugin to your VCV Rack plugins directory

The default RACK_DIR is ~/.local/share/Rack2/Rack-SDK. If your SDK lives there, you can skip the export.

The ML engine is the repository's shared nisps/ C++20 core, reached via relative includes from src/iml.hpp -- the plugin must be built from a full repo checkout. See BUILDING.md.

Quick Start

1. Add MEMLNaut from the module browser.
2. Patch LFOs (or any CV source) into one or more of the 8 IN jacks.
3. Connect several of the 16 outputs to parameters on your synth voice -- filter cutoff, oscillator pitch, waveshape, VCA level, etc.
4. Press RAND to randomize the network. You should hear your synth respond as the LFOs sweep.
5. Flip the LEARN switch on.
6. When you hear something you like, press + (thumbs up). When you hear something you dislike, press - (thumbs down).
7. Keep exploring. The mapping will converge toward sounds you prefer.

Panel Controls

Knobs

Control	Description
SPREAD	Controls weight initialization scale, RL noise amplitude, and weight decay. Low values produce extreme, polarized mappings. High values produce balanced, subtle mappings. Default: 60%.
RATE	Inference rate. Full CCW = ~170 Hz (block rate, cheapest). Noon = ~2.8 kHz (good for CV). Full CW = 44.1 kHz (audio rate, most expensive).

Buttons

Control	Description
+	Thumbs up. Captures the current input/output pair as a training example and trains the network. Requires LEARN to be enabled.
-	Thumbs down. Increases exploration noise and perturbs the network weights to try something different. Requires LEARN to be enabled.
LEARN	Toggle switch. Enables/disables RL feedback. When off, the module still runs inference -- it just ignores +/- presses.
RAND	Randomize all network weights (using current SPREAD setting).
CLEAR	Long-press (~1 second) to clear all training examples and reset the network.

Inputs

Port	Description
IN 1-8	CV inputs feeding the 8-input network. Default range: 0-10V (unipolar); per-input bipolar toggle in the context menu.
SPREAD CV	CV modulation of the SPREAD knob (added to knob value, 0-10V).
LEARN	Gate input. High = enable learning. Works alongside the LEARN toggle (either enables it).
+ TRIG	Trigger input for thumbs-up. Alternative to pressing the + button.
- TRIG	Trigger input for thumbs-down. Alternative to pressing the - button.

Outputs

Port	Description
OUT 1-16	Raw MLP outputs. Default: 0-10V unipolar (per-output bipolar toggle in the context menu).

Each output has a trimpot attenuverter (-100% to +100%) for scaling and inverting individual outputs without external modules, and is surrounded by an LED ring whose arc fill tracks the output's current level (color-coded per output, orange-to-cyan ramp).

LEDs & Display

Indicator	Meaning
LEARN (green)	Lit when learning is enabled.
TRAIN (yellow)	Flashes during background training.
Output LED rings	Arc fill tracks each output's current level.

The built-in display shows all 16 output levels as color-coded bars, the current noise level (N:), the example count (e.g. 42/100), and "TRAIN" during active training.

RL Workflow

The reinforcement learning loop works like this:

1. Start exploring. Patch LFOs or sequencers into the IN jacks. Connect outputs to interesting synth parameters. Press RAND a few times to hear different random mappings.

2. Enable learning. Flip the LEARN switch on (or send a gate to the LEARN input).

3. Thumbs up (+) when you like what you hear. This:

   • Saves the current input position and output values as a training example
   • Trains the network to reproduce this mapping
   • Slightly reduces exploration noise (the network becomes more "settled")

4. Thumbs down (-) when you dislike what you hear. This:

   • Increases exploration noise
   • Perturbs the network weights to try a different mapping
   • Does NOT save any training example

5. Repeat. Over time, the network learns to produce outputs you tend to like across the input space. Regions near your thumbs-up examples will be stable; distant regions remain exploratory.

6. Disable learning when you are happy with the mapping. The module continues running inference with the trained network. You now have a complex, personalized CV source.

Tips

• Give thumbs-up at several different input positions to teach the network about different regions of the input space.
• The network holds up to 100 examples. Oldest examples are dropped when full (FIFO).
• Use the SPREAD knob to control how wild the exploration is. Low spread = dramatic changes. High spread = subtle refinements.

Context Menu

Right-click the module to access these settings:

• Output ranges -- toggle each of the 16 outputs between unipolar (0-10V, default) and bipolar (+/-5V).
• Input ranges -- toggle each of the 8 inputs between unipolar (0-10V, default) and bipolar (+/-5V). Set to bipolar if your source produces +/-5V signals (e.g., standard LFOs).
• Compute derived stats (Mean/Std/Delta) -- internal statistics toggle (no dedicated output jacks in the current module).
• Output slew -- smoothing time applied when network weights change (after training or perturbation). Prevents clicks from sudden output jumps. Options: 0, 5, 10, 20, 50, 100 ms. Default: 10 ms.
• Presets (.nisps) -- save/load the full module state (weights, training examples, knob positions, ranges) to/from a .nisps JSON file.
• Browser bridge (WS<->OSC) -- enable the OSC server for the companion browser app; choose the listen port (7001, 7002, 7003, 9000, 9001).

Presets (.nisps)

MEMLNaut uses .nisps files for saving and sharing trained networks. The format (version 3, flat core-exact weight vector) is specified in docs/specs/vcv-module.md -- the single source of truth. Files saved by pre-2026-07 builds (version 1, nested weight arrays) no longer load.

Full module state is also saved automatically with your VCV Rack patch file; you do not need .nisps exports to preserve work between sessions.

Browser Integration (Manifold)

MEMLNaut can be driven live from the Manifold browser app (manifold/, VCV backend) over a WebSocket-to-OSC bridge (manifold/osc-bridge/bridge.ts, Deno).

1. Right-click the module and enable OSC server (default port 7001; each module instance offsets its default port automatically).
2. Run the bridge with --osc-port matching the module's port.
3. In Manifold, select the VCV backend.

What flows over the wire (the complete protocol -- see docs/specs/vcv-module.md):

OSC Address	Direction	Content
/nisps/input	Web -> VCV	Input vector (browser drives the model: "bridged mode")
/nisps/output	VCV -> Web	Current output values (~10 times/sec)
/nisps/input	VCV -> Web	Current input values (~10 times/sec)
/nisps/feedback	Web -> VCV	Verdict ops (up / down / rand / clear)

Training is bidirectional: both the browser verdict loop and the module's panel buttons drive the same network. There is no weight/state sync channel -- persistence belongs to the Rack patch.

Technical Details

Network architecture

Inputs:  8
Hidden:  16 -> 24 -> 16  (3 hidden layers, ReLU activation)
Output:  16 (sigmoid activation, producing values in [0, 1])

The ML engine is the repository's shared nisps/ core (nisps::ml::MLPCore<DynamicStorage> behind the thin adapter src/iml.hpp) -- bit-identical training/inference semantics to the MEMLNaut firmware and the browser WASM engine.

Inference

The MLP runs in the VCV process() callback. The RATE knob controls how often inference runs, from once per 256 samples (~170 Hz) to every sample (44.1 kHz). Between inference steps, outputs are linearly interpolated to avoid staircase artifacts.

Threading

• Audio thread: Reads CV inputs, runs MLP inference, writes CV outputs. Never blocks.
• Worker thread: Handles training (thumbs-up) and weight perturbation (thumbs-down). When complete, signals the audio thread to crossfade to the new outputs.
• Each module instance has its own independent worker thread and ML engine.

Building

See BUILDING.md (build steps, troubleshooting) and DISTRIBUTION.md (packaging .vcvplugin bundles, cross-platform builds).

Project structure

vcv/
  Makefile          # Build configuration (C++20; core via relative includes)
  plugin.json       # Plugin manifest (name, version, tags)
  src/
    plugin.cpp      # Plugin initialization
    plugin.hpp      # Plugin globals
    MEMLNaut.cpp    # Module logic, widget/UI, serialization, OSC wiring
    iml.hpp         # Thin adapter over the shared nisps/ ML core
    osc_server.hpp  # OSC bridge server (transport only)
    LedRing.hpp     # Per-output LED ring widget
    palette.hpp     # Color palette from the frontend design tokens
  res/
    MEMLNaut-wide.svg      # 44HP panel (the one the module loads)
    MEMLNaut.svg           # Unused 30HP variant
    MEMLNaut-expander.svg  # Unused 8HP expander panel
  dist/             # Built .vcvplugin bundles
  dep/              # Build dependencies