memlnaut-nisps/docs/specs/backends-spec.md

kind	stability	layer	counterpart
spec	evolving	binding	aimmersive-clone-spec.md

Output Backends — Specification

Workstream E. Design-only, read-only audit 2026-06-27. The new app is Vite + React + TS in manifold/, wired to the parity-tested TS engine lifted from playground/src. British spelling in product copy. The built-in synth is the "Powerful Synth Engine" — the string "C15" must never reach the user.

Naming guard (non-negotiable). The codename C15 survives only in internal module/file names that the user never sees (c15-adapter.js, c15-bridge.js). Every label, tooltip, dock entry, menu item, status string, and aria-label says "Powerful Synth Engine" (or just "Synth"). A lint allowlist + a Playwright assertion (expect(page).not.toContainText('C15')) enforce this across manifold/ and the VCV panel SVG/strings.

---

0. The one idea: backends are adapters behind one interface

Today the deployed app (js/a-app.js) fans output out to four ad-hoc sinks inline in routeOutputs() (a-app.js:2425): synth (activeEngine.setParam), MIDI CC (midiOutput.sendBatch), audio-canvas, and visual (visualizer.setParams). Each has its own throttle, dead-zone, and override handling copy-pasted. That fan-out is the debt this workstream removes.

Replace it with one OutputBackend interface and a registry. Exactly one backend is "active" at a time, chosen in the Console dock. The reactive spine (per engine-architecture.md §2 and findings-design-and-manifold.md §4) ends in a single side-effect that calls activeBackend.send(routedOutput). Swapping backends swaps nothing else — the input pipeline, ML, output pipeline, training loop, and verdict loop are all backend-agnostic.

 gesture → input pipeline → ML (WasmIML) → output pipeline → [ activeBackend.send(routed) ]
                                                                        │
        ┌───────────────┬───────────────┬───────────────┬─────────────┼──────────────┐
   WebAudioBackend  ParticleBackend  WebMidiBackend  OscBridgeBackend  CvGateBackend  VcvBridgeBackend
   (Powerful Synth) (flow field)     (advanced CC)    (paths+ranges)    (1V/oct+gate)  (8→model→16, LED rings)

The active backend is a property of the output dock (the Console's right rail / a-immersive's Mode drawer). Backends self-describe (id, label, capability) so the dock renders a picker without hard-coding the list.

---

1. The OutputBackend adapter interface (TS)

Lives at manifold/src/engine/backends/backend.ts. The engine never imports a concrete backend; it imports the interface + the registry. Concrete backends may import the engine's pure helpers (curves, param-map data) but never React.

// manifold/src/engine/backends/backend.ts

/** What a backend needs to know about the active mode to map outputs. */
export interface BackendContext {
  modeId: string;
  outputCount: number;             // model output dims actually in use (≤ 126)
  paramMeta: ReadonlyArray<ParamMeta>;  // name/label/min/max/curve/group per output
  sampleRate?: number;             // for audio backends
  audioContext?: AudioContext;     // lazily provided; only audio backends use it
}

export interface ParamMeta {
  id: string;        // stable machine id, e.g. 'Env_A_Att'
  label: string;     // user-facing
  min: number;       // baseline range floor (normalised 0..1 maps here)
  max: number;       // baseline range ceil
  curve: number;     // 0..1, 0.5 = linear (see §3 universal mapping)
  group: string;     // for colour grouping (LED rings, heatmap)
}

export type BackendId =
  | 'synth' | 'particles' | 'midi' | 'osc' | 'cvgate' | 'vcv';

export interface BackendDescriptor {
  id: BackendId;
  label: string;             // dock label — NEVER "C15"
  description: string;
  /** crossOriginIsolated / WebMIDI / WebSocket etc. availability probe. */
  isAvailable(): boolean;
  /** true when this backend can ALSO feed inputs back (VCV bridge, OSC return). */
  bidirectional?: boolean;
}

export interface OutputBackend {
  readonly descriptor: BackendDescriptor;

  /** Called once when this backend becomes active. May be async (audio start,
   *  WS connect, MIDI access). Resolve only when ready to receive send(). */
  start(ctx: BackendContext): Promise<void>;

  /** Hot per-frame path. `routed` is the post-pipeline Float32Array (0..1),
   *  length = ctx.outputCount. MUST NOT allocate; MUST NOT mutate `routed`.
   *  Throttling/dead-zone live INSIDE each backend (rates differ per sink). */
  send(routed: Float32Array): void;

  /** Called when switching away or unmounting. Release WS/MIDI/audio/threads. */
  teardown(): Promise<void>;

  /** Optional input return path for bidirectional backends. The engine
   *  subscribes; values drive model inputs (e.g. VCV CV-in, OSC /nisps/input). */
  onInputs?(cb: (values: Float32Array) => void): () => void;

  /** Optional: backends that own training transport (VCV/OSC bridge) expose
   *  the remote verdict/example surface here. See §7. */
  remote?: RemoteTrainingBridge;
}

Registry (manifold/src/engine/backends/registry.ts): a Map<BackendId, () => OutputBackend> of lazy factories. The dock reads descriptors (filtered by isAvailable()); selecting one calls engine.setBackend(id), which teardown()s the old and start()s the new with the current BackendContext.

Why send(Float32Array) and not per-param events: matches the spine's single transferable-buffer effect (engine-architecture.md §2.1), keeps the hot path allocation-free, and lets each backend decide its own decimation. The legacy code already proves the pattern — every sink takes the full output vector and self-throttles (synth 50 ms/0.002 dead-zone a-app.js:2425; MIDI 50 ms/Δ1 midi-output.js:114; OSC 50 ms/0.002 osc-output.js:97).

---

2. The backends

2.1 Built-in Synth — the "Powerful Synth Engine" (web-audio.ts + synth.ts)

Two cooperating backends, both labelled as the synth in the UI, but architecturally distinct:

• WebAudioBackend wraps the parity-tested repo engine: EngineHost (playground/src/audio/engine-host.ts) + the worklet nisps-processor.ts running _nisps_engine_process_block (nisps/wasm/bindings.cpp). This is the firmware-parity audio path — the engine is the sound. send() → EngineHost.setParams(routed) → worklet. This is the default and the one that satisfies browser-parity chokepoint C.
• PowerfulSynthBackend (the C15 path) wraps deployments/meml-aimmersive/js/synth/c15-adapter.js → c15-bridge.js (SharedArrayBuffer ring + its own c15_engine.wasm worklet). Param mapping comes from param-map.js (SYNTH_PARAM_MAP, 126 entries) and presets.js (tiered presets). setParam(index, normalised) maps index→hardware id (c15-adapter.js:121). This is browser-only (firmware has no C15), and its 126-param surface + group/section overrides power the synth visualiser and the group-override drawer.

Reuse, verbatim: c15-adapter.js, c15-bridge.js, param-map.js, presets.js move under manifold/src/engine/backends/synth/ unchanged (internal names keep "c15"; UI strings do not). The engine-interface.js SynthEngine base maps cleanly onto OutputBackend: init(ctx)→start, setParam loop driven by send, stop→teardown. Curve/override math is applyCurve/applyGroupOverride (param-map.js:287) — fold into the universal mapping (§3).

Throttle (keep — load-bearing): ≥50 ms send interval + 0.002 dead-zone per param prevents flooding the C15 ring buffer at 126×30 fps (a-immersive.html clone-spec §10 flags this).

isAvailable(): WebAudio + (for the C15 path) crossOriginIsolated === true (SAB needs COOP/COEP; already server-scoped per findings-engine-surface.md).

2.2 Particle System — faithful port of visualizer.js (particles.ts)

A ParticleBackend whose send(routed) calls a ported FlowFieldVisualizer.setParams(routed). The visual canvas runs in its own requestAnimationFrame loop (per findings-design-and-manifold.md §4.3 — rAF touches drawing only, never inference); send() only updates the param struct. The port MUST look and behave exactly as the deployed version. The full algorithm is documented in §4 so the React port is byte-faithful.

isAvailable(): always (Canvas2D). This backend produces no audio — it is the "visual" output mode.

2.3 MIDI out — advanced CC config (web-midi.ts)

A WebMidiBackend wrapping the salvaged midi-output.js (Web MIDI API, sendBatch, per-CC dead-zone + 50 ms throttle, device hot-plug handling — midi-output.js). The advanced CC config comes from workstream D's config model, persisted as a CC map: per output dim → { name, cc (0–127), channel (1–16), min, max, curve, muted, fixedValue }. The map shape and storage are already defined in midi-cc-map.js (createCCParam, loadCCMap/saveCCMap, well-known CC_NAMES, default 8-CC starter set). Lift that file verbatim into manifold/src/engine/backends/midi/cc-map.ts.

send(routed): for each non-muted CC param, value = round(applyGroupOverride(routed[i], curve, min, max) * 127); batch the changed ones; midiOutput.sendBatch(...). Storage key stays engine-scoped (nisps-midi-cc-map:<modeId>) for migration continuity.

isAvailable(): !!navigator.requestMIDIAccess.

2.4 OSC out — paths + ranges (osc-bridge.ts)

An OscBridgeBackend that salvages the existing OSC bridge rather than reinventing it. Two pieces already exist and are good:

• Browser client: deployments/meml-aimmersive/js/synth/osc-output.js (param-named WS messages, 50 ms/0.002 dead-zone) and the richer js/nisps/osc-client.js (NispsOscClient — EventTarget, sendState/sendWeights/sendParams, onOutputsReceived/onInputsReceived, auto-reconnect with backoff). Lift osc-client.js as the transport (it already speaks the bridge protocol and is bidirectional).
• Bridge server: deployments/meml-aimmersive/osc-bridge/bridge.ts — a Deno WebSocket↔UDP-OSC bridge, zero-dependency OSC encode/decode, bidirectional. Keep it as-is; it is the canonical transport between browser and any OSC target (VCV, SuperCollider).

OSC path + range contract (salvaged from bridge.ts):

Direction	Address	Args	Meaning
browser→target	/nisps/<param_name>	f	one param, post-baseline-mapping value (see §3)
browser→target	/nisps/state	s	full JSON state (weights + examples + config)
browser→target	/nisps/weights	s	weights-only JSON
target→browser	/nisps/output	f…f	output vector (visualisation / monitoring)
target→browser	/nisps/input	f…f	input vector → drives model inputs

Ranges: OSC floats are sent in the param's mapped range by default (applyGroupOverride applied before send, matching osc-output.js), with a per-backend toggle to send raw normalised 0..1 instead (some OSC targets want 0..1 and do their own scaling). Address prefix (/nisps), target host/port (default 127.0.0.1:9000), and listen port (default 9001) are configurable — bridge.ts already exposes --osc-host/--osc-port/--osc-prefix/--ws-port/--listen-port.

isAvailable(): always (attempts WS to ws://localhost:8765; surfaces a "bridge not running" status if the connect fails — osc-client.js already reconnects with backoff). Bidirectional (onInputs wired to /nisps/input).

2.5 CV / gate backend (cvgate.ts)

For browser-side CV/gate there is no native hardware path, so this backend has two transports selectable in config:

1. DC-coupled WebAudio CV (browser-native): each output dim drives a ConstantSourceNode (or a sample-accurate AudioWorklet channel) whose offset = mapped voltage, summed/routed to the audio interface's output channels. Gate outputs are derived from a configurable threshold on a chosen dim (value > τ → high). Pitch (1V/oct) uses a per-output "voltage role" config: { role: 'cv' | 'gate' | 'voct', vmin, vmax, gateThreshold }. This is the only way to emit real CV from a browser (DC-coupled interface required; surfaced as a caveat in the UI).
2. Bridged CV via VCV / OSC (recommended default): reuse the OSC/VCV transport — the VCV module's 16 CV outputs (§5) are the real CV/gate jacks. In this mode CvGateBackend is a thin alias that delegates to VcvBridgeBackend with a "treat outputs as CV/gate" preset (per-output unipolar 0–10 V / bipolar ±5 V / 1V-oct, matching the VCV per-output range menu in MEMLNaut.cpp:818).

isAvailable(): WebAudio path always; native-CV quality flagged as "requires DC-coupled interface". Open choice: whether browser-native DC CV is worth shipping vs. making CV strictly a VCV-bridge concern (recommendation: ship the VCV-bridge alias first, defer DC-coupled WebAudio CV).

2.6 VCV Rack module — first-class (vcv-bridge.ts browser side + vcv/ C++ side)

The headline new backend. A first-class VCV Rack 2 module (MEMLNaut) with 8 CV inputs → model → 16 CV outputs, an LED ring around each of the 16 outputs, and a browser↔VCV bridge so the tool is controllable AND trainable from both inside Rack and entirely from the browser. Full design in §5–§7.

The browser-side adapter VcvBridgeBackend reuses NispsOscClient (§2.4) as transport. When active in bridged mode, the browser supplies inputs in real time (/nisps/input) and the verdict/example loop is mirrored over the bridge (/nisps/state, /nisps/weights, new /nisps/feedback).

---

3. Universal per-output baseline mapping

Every backend shares ONE baseline mapping from a normalised model output v ∈ [0,1] to a sink value, so behaviour is identical across sinks and the override UI (heatmap popup, group drawer) is backend-agnostic. This is the existing curve math (param-map.js:287), promoted to manifold/src/engine/backends/mapping.ts:

// 0.5 = linear; <0.5 ease-in, >0.5 ease-out. Bit-identical to legacy + nisps/core/math.hpp.
export function applyCurve(v: number, c: number): number {
  return c === 0.5 ? v : Math.pow(v, Math.pow(2, 4 * (c - 0.5)));
}

/** Per-output baseline: curve, then scale into [min,max]; honour freeze/mute. */
export function mapOutput(v: number, p: OutputMapping): number {
  if (p.frozen) return p.fixedValue;        // pinned, ignores model
  return p.min + applyCurve(v, p.curve) * (p.max - p.min);
}

export interface OutputMapping {
  min: number; max: number; curve: number;   // baseline range + curve
  frozen: boolean; fixedValue: number;        // pin
  muted?: boolean;                            // excluded from this sink
}

Backends then apply only their sink-specific final transform on top of the baseline:

Backend	Baseline → sink transform
Powerful Synth	mapOutput → setParam(i, value) (value already in param range)
Particles	raw 0..1 (the visualiser owns its own param ranges — see §4); curve/freeze still applied, min/max default to 0..1
MIDI CC	mapOutput → round(value * 127) clamped 0–127
OSC	mapOutput (or raw 0..1 if "send raw" toggled) → /nisps/<name> <f>
CV/gate	mapOutput → voltage by role (cv: value*10 or (value-0.5)*10; voct: 1V/oct; gate: value>τ ? high : 0)
VCV	model output 0..1 sent raw over bridge; the module applies its own per-output range + attenuverter (MEMLNaut.cpp:342 outputToVoltage)

The override store (one per mode, OutputMapping[]) is owned by the engine and shared by all backends; freeze/mute/curve/range edits in the UI apply uniformly. Note the legacy split where the heatmap calls it frozen and the group drawer calls it muted but both map to the same field (aimmersive-clone-spec.md §10) — unify to the single OutputMapping above.

---

4. Particle system — faithful-port plan with the documented algorithm

Source of truth: deployments/meml-aimmersive/js/ui/visualizer.js (289 lines, read in full). The React port (manifold/src/engine/backends/particles/flow-field.ts) must reproduce this exactly. Below is the complete algorithm with line citations so the port is verifiable.

4.1 Constants & noise source

• Particle count: numParticles = 400 (visualizer.js:53).
• Noise: a self-contained 2-D value/Perlin-style noise, not a library. A Uint8Array(512) permutation table PERM is built once at module load by Fisher–Yates shuffling [0..255] then duplicating (:5–14). fade(t)=t³(t(6t−15)+10) (:16), lerp (:17), grad(hash,x,y) using hash & 3 (:19–24), noise2D(x,y) doing the standard 4-corner bilinear-with-fade interpolation (:26–44). The shuffle uses Math.random() at module load, so the field is non-deterministic per page load — the port must keep this (or seed it; flagged as an open choice if determinism is wanted for tests).
• TWO_PI = Math.PI*2 (:46); this.time advances += 0.003 per draw() (:185).

4.2 Output→param mapping (20 dims, setParams, :159–181) — reproduce verbatim

Guard: returns early if outputs.length < 20 (:160). Then:

angleOffset       = out0 * TWO_PI
scale             = 0.001 + out1 * 0.009
speed             = 0.5   + out2 * 4.5
hueBase           = out3 * 360
hueSpread         = out4 * 120
particleSize      = 1     + out5 * 5
fadeRate          = 0.01  + out6 * 0.14
turbulence        = out7 * 2
attractStrength   = 0.1   + out8 * 2.9
attractRadius     = 40    + out9 * 420
dispersionRate    = 0.2   + out10 * 8
dispersionAmount  = out11 * 3
particleLifetime  = 30    + out12 * 470
respawnStyle      = out13                       // 0=random,1=edge,2=center-burst
advectionMode     = out14                       // flow→orbit→radial blend
inertia           = out15 * 0.98
drag              = out16 * 0.35
repulsorStrength  = out17 * 4.5
repulsorCount     = floor(out18 * 4.999)        // 0..4
repulsorOrbitRate = 0.1   + out19 * 2.9

The 20 output names (for labels/the heatmap colouring) are VISUAL_PARAM_NAMES in a-app.js:46: Flow, Scale, Speed, Hue, Spread, Size, Trail, Turb, Attract, Radius, DispRate, DispAmt, Lifetime, Respawn, Advection, Inertia, Drag, Repulse, RepCnt, RepRate.

4.3 Particle lifecycle (:104–156)

• makeParticle(id): random x,y in canvas; age = floor(rand*particleLifetime); life = computeLifetime(); vx=vy=0 (:104–114).
• computeLifetime(): max(10, floor(particleLifetime * (0.65 + rand*0.7))) (:116–119).
• respawnParticle(p): mode = min(2, floor(respawnStyle*2.999)) (:122):
  • edge (1): spawn on a random one of 4 sides; give inward impulse of magnitude 2.0 toward centre (:125–137).
  • center-burst (2): spawn within radius min(w,h)*0.08 of centre at random angle; velocity 2.5 outward along that angle (:138–145).
  • random (0): random position; velocity (rand*2−1)*0.5 each axis (:146–152).
  • reset age=0, life=computeLifetime().

4.4 Per-frame draw() (:183–287) — the exact integration

1. time += 0.003. Trail fade: fill whole canvas with rgba(13,13,13, fadeRate) (:188) — this is the trail length control, not a clear.
2. For each particle, with centre (cx,cy)=(w/2,h/2):
   • Flow field: nx=p.x*scale, ny=p.y*scale; angle = noise2D(nx+time, ny)*TWO_PI + angleOffset; curl = noise2D(nx+100, ny+100+time*0.5)*turbulence (:196–199).
   • Three advection fields (:202–214): flow (cos(angle+curl), sin(angle+curl))*speed; orbit = perpendicular to the radial-from-centre unit vector, *speed; radial = radial unit vector *speed.
   • Blend by advectionMode (:216–225): modeBlend = advectionMode*2; if <1 lerp(flow→orbit, modeBlend) else lerp(orbit→radial, modeBlend−1).
   • Inertia + drag (:226–229): vx = vx*inertia + targetVx*(1−inertia) (same for vy); then vx *= (1−drag).
   • nextX = x+vx, nextY = y+vy.
   • Central attractor (:233–242): pull toward centre with falloff = 1/(1+normDist²) where normDist = min(dist/attractRadius, 2); add nCenter * attractStrength * falloff.
   • Dispersion pulse (:244–248): pulse = 0.5+0.5*sin(time*dispersionRate + id*0.07); subtract nCenter * dispersionAmount*pulse*falloff (pushes outward near centre).
   • Orbiting repulsors (:250–264): repulsorRadius = min(w,h)*0.28; for each of repulsorCount: phase from time*repulsorOrbitRate + (r/4)*TWO_PI, wobble = 0.6+0.15*r, position uses cos(phase*(1+wobble))/sin(phase*(1.3+wobble)); inverse-square push force = repulsorStrength*(650/distSq) with distSq = d²+160.
   • Wrap edges (:270–273); age += 1; respawn when age >= life (:275–276).
   • Colour (:279–282): hue = (hueBase + (id/numParticles)*hueSpread) % 360; lightness = 50 + sin(id*0.1 + time)*15; fill hsl(hue, 75%, lightness%); arc(x,y,particleSize).

4.5 Port plan

• Class, not component. FlowFieldVisualizer is a plain TS class taking a <canvas> ref — identical to today. The React <ParticleCanvas> mounts it in onMount, drives draw() from one requestAnimationFrame loop, and calls resize() on the window resize handler (DPR scaling at :84–92). React renders the canvas element; the class owns all pixels.
• ParticleBackend.send(routed) → viz.setParams(routed) (no alloc; just field writes). Because the visualiser reads outputs[0..19], the backend asserts ctx.outputCount >= 20 and slices/pads to 20.
• Faithfulness gate (Playwright): pixel-diff a fixed seed (seed the PERM shuffle behind a ?seed= for tests) at fixed param vectors against a golden capture from the deployed app; assert SSIM ≥ threshold. Also unit-test setParams mapping numerically (the §4.2 table).
• Verbatim copy is allowed: the noise + integration math is pure and has no DOM coupling beyond ctx/canvas; lift :1–288 essentially unchanged into TS, add types, keep numeric constants exact.

---

5. VCV Rack module design (MEMLNaut, 8 in / 16 out)

There is already a working VCV module at vcv/ (MEMLNaut.cpp, 959 lines; plugin.json; SPEC.md; osc_server.hpp; Makefile; res/*.svg) — but it is 2-in / 12-out + 5 derived. The new requirement is 8-in / 16-out with LED rings. This is an evolution of the existing module, reusing its threading model, OSC server, and serialization wholesale.

5.1 What changes vs. the existing module

• NUM_ML_INPUTS: 2 → 8 (the existing MAX_ML_INPUTS = 8 already anticipated this; MEMLNaut.cpp:14). All 8 are first-class jacks (not "reserved").
• NUM_ML_OUTPUTS: 12 → 16. The 5 derived outputs (MEAN/STD/DELTA/NOVELTY/CONFIDENCE) remain but become optional context-menu extras or move to the expander — the 16 raw outputs are the headline.
• MLP shape: stays within "the modular N×M envelope" — nisps::IML<float> iml{8, 16, {16, 24, 16}} (the existing default hidden stack is fine; sized for real-time inference per SPEC.md). Inputs feed model input dims; the 16 outputs are the inference outputs.
• LED ring per output replaces the single SmallLight<WhiteLight> next to each jack (MEMLNaut.cpp:804).

5.2 Panel layout (Wide, ~32–44 HP)

┌────────────────────────────────────────────┐
│              MEMLNaut                       │   ← brand; "Powerful Synth" wording N/A (this is the CV mapper)
│  ┌──────────────────────────────────────┐  │
│  │  DISPLAY: 16 bars + XY dot + metrics │  │   ← NanoVG LedDisplay/drawLayer (existing :699)
│  └──────────────────────────────────────┘  │
│   SPREAD   RATE          LEARN● RAND CLEAR  │   ← knobs + buttons (existing controls)
│   [+] [−]   (verdict buttons)               │
│                                             │
│  INPUTS  (8 jacks, 2 rows × 4)              │
│   IN1 IN2 IN3 IN4                           │
│   IN5 IN6 IN7 IN8   + LEARN_GATE +TRIG −TRIG│
│                                             │
│  OUTPUTS (16, 4 rows × 4), each:            │
│   ◉jack  with an LED RING around the jack   │
│   [(◯1)(◯2)(◯3)(◯4)]                         │
│   [(◯5)(◯6)(◯7)(◯8)]                         │
│   [(◯9)(◯10)(◯11)(◯12)]                      │
│   [(◯13)(◯14)(◯15)(◯16)]                     │
│  (optional) MEAN STD DELTA NOVELTY CONF     │
└────────────────────────────────────────────┘

Each output is a PJ301MPort jack with a LedRingWidget drawn concentric around it (no separate attenuverter trimpot in the default skin — attenuverter moves to right-click/expander to make room for the ring; keep PARAM_ATTEN_* in the model for range scaling).

5.3 The LED-ring widget + palette mapping

A custom widget that draws a ring whose arc fill is proportional to the output value and whose colour comes from the frontend design tokens. Per the Rack manual, self-illuminating custom widgets override drawLayer(args, 1) and draw on layer 1 (so they stay bright when room brightness is lowered) (VCV custom lights, Migrate2).

// vcv/src/LedRing.hpp  (new)
struct LedRingWidget : Widget {
    MEMLNaut* module = nullptr;
    int outIdx = 0;
    NVGcolor ringColor = nvgRGB(0xff, 0x6a, 0x00);  // default --accent
    float radius = 7.f;   // mm-ish, around a PJ301M jack
    void drawLayer(const DrawArgs& args, int layer) override {
        if (layer != 1 || !module) return;
        float v = clamp(module->slewOutputs[outIdx], 0.f, 1.f);   // 0..1
        Vec c = box.size.div(2);
        // track (dim full ring)
        nvgBeginPath(args.vg);
        nvgArc(args.vg, c.x, c.y, radius, -M_PI/2, -M_PI/2 + 2*M_PI, NVG_CW);
        nvgStrokeColor(args.vg, nvgRGBA(ringColor.r*255, ringColor.g*255, ringColor.b*255, 40));
        nvgStrokeWidth(args.vg, 1.4f); nvgStroke(args.vg);
        // value arc (proportional)
        nvgBeginPath(args.vg);
        nvgArc(args.vg, c.x, c.y, radius, -M_PI/2, -M_PI/2 + v*2*M_PI, NVG_CW);
        nvgStrokeColor(args.vg, ringColor);
        nvgStrokeWidth(args.vg, 1.8f); nvgStroke(args.vg);
        // glow halo (matches the frontend "glow not shadow" signature)
        nvgGlobalCompositeOperation(args.vg, NVG_LIGHTER); /* … bloom pass … */
    }
};

Palette mapping — derived from docs/redesign/manifold-export/tokens/colors.css (read). Ring colours come from the design tokens so VCV matches the frontend. A kRingPalette[16] table assigns each output a colour by its parameter group, cycling through the token accents and group/pin colours:

Source token (colors.css)	Hex	Used for
--accent	#ff6a00	primary outputs / group 0 (orange — the live colour)
--accent-2	#00ccff	data outputs / group 1 (cyan)
--accent-3	#ffa860	group 2 (warm tint)
--pin-3 base #b464ff	#b464ff	group 3 (violet)
--good	#6bc26b	group 4 (green)
--warn	#f5c45e	group 5 (amber)
--info	#5b9eef	group 6 (blue)
--danger	#ff4466	bipolar / perturbed outputs (red)

Mapping rule: ringColor = kRingPalette[paramMeta[i].group % paletteLen], so outputs in the same mode-group glow the same colour, identical to the heatmap/Console grouping. Bipolar outputs (per-output range menu) tint toward --danger. The palette is a single header (vcv/src/palette.hpp) generated from colors.css so a token change propagates to both frontend and module (a small codegen step; open choice whether to automate or hand-sync).

5.4 Reused, unchanged from the existing module

• Threading model (MEMLNaut.cpp:67–312): audio-thread iml + worker-thread imlShadow, job queue (Train/Perturb/Randomize/Clear), atomic weight hand-off, single-writer invariant. Already correct; just resize the I/O.
• Inference-rate decimation + slew + post-swap crossfade (:447–499).
• Verdict loop (:412–445): +/− buttons and +TRIG/−TRIG gated by LEARN. + → add example (current inputs→current outputs) + enqueue Train + decay noise ×0.97; − → bump noise (cap 0.3(1−s)+0.05s) + enqueue Perturb. Identical semantics to the browser verdict loop.
• Serialization (dataToJson/dataFromJson, :548–696) + .nisps preset save/load (:862–924) + OSC server (osc_server.hpp). The .nisps format is the shared interchange with the browser (vcv/NISPS-FORMAT.md).
• Per-output / per-input range menu (uni/bipolar, attenuverter) (:818–843).

5.5 Plugin scaffold files

vcv/
├── plugin.json          # slug MEMLNaut (exists) — bump version; tags Controller/Utility/Random
├── Makefile             # VCV SDK Makefile (exists); RACK_DIR-driven
├── Makefile.dist        # dist packaging (exists)
├── src/
│   ├── plugin.{hpp,cpp} # plugin init / model registration (exist)
│   ├── MEMLNaut.cpp     # module — resize 8 in / 16 out (edit)
│   ├── LedRing.hpp      # NEW — LED-ring widget (§5.3)
│   ├── palette.hpp      # NEW — ring colour table from colors.css (§5.3)
│   └── osc_server.hpp   # OSC server (exists; extend with /nisps/feedback §7)
├── res/
│   ├── MEMLNaut.svg / -wide.svg / -expander.svg   # panels (exist; redraw for 8/16 layout)
├── test/smoke_test.cpp  # exists
├── plugin.json, SPEC.md, BUILDING.md, README.md, NISPS-FORMAT.md  (exist)

---

6. Browser↔VCV bridge design

The module must be controllable AND trainable from BOTH inside Rack AND entirely from the browser. Both ends operate on the same model; the bridge keeps them coherent.

6.1 Transport — propose options (operator open choice)

The existing salvage path is WebSocket↔UDP-OSC (osc-bridge/bridge.ts + osc-client.js + osc_server.hpp). This is the recommended default and already works. Three options to flag:

Option	How	Pros	Cons
A. WS↔OSC bridge server (recommended)	Browser ⇄ bridge.ts (Deno WS server, localhost:8765) ⇄ UDP OSC ⇄ module's osc_server.hpp (port 9000/9001)	Already built + bidirectional; standard OSC; works with SuperCollider/Max too; no browser perms	Needs a helper process running locally (Deno or the compiled bridge.mjs)
B. Direct WebMIDI	Browser ⇄ Web MIDI ⇄ a virtual MIDI port ⇄ a tiny MIDI-in path in the module	No helper process if a virtual MIDI port exists; browser-native	7-bit/14-bit only — too coarse for weights/state; really only for live CC; module would need MIDI parsing
C. Native (module hosts a WS server)	The VCV module itself opens a WebSocket/HTTP server; browser connects directly	No external bridge process	Adds a WS/TLS stack inside the plugin; COOP/COEP + mixed-content (https:// page → ws://localhost) friction; more attack surface in the audio plugin

Recommendation: ship A (it exists, it's bidirectional, it already targets this very module — see osc-client.js:1 "Connects the webapp to VCV Rack MEMLNaut module"). Keep B as a live-performance CC convenience only. Treat C as a future "no-helper" nicety. This transport choice is an explicit operator open choice.

6.2 Two modes

• Standalone: module runs entirely inside Rack (CV in → model → CV out; verdict via panel buttons/triggers). No browser. Works today.
• Bridged: browser connects via the bridge. In bridged mode the browser supplies inputs in real time — the Manifold pointer / joystick / pads stream /nisps/input <f…f> to the module, which uses them instead of (or blended with, configurable) the physical CV-in jacks. The module streams /nisps/output and /nisps/input back at ~100 ms (MEMLNaut.cpp:536) for browser visualisation. State/weights sync both ways via /nisps/state + /nisps/weights (already staged atomically into the audio thread, :360–384).

6.3 Coherence model

One model, one owner of weights at a time. The bridge sends whole-model snapshots (/nisps/state, /nisps/weights) on any structural change (train completes, randomize, clear, load preset), and continuous I/O vectors (/nisps/input, /nisps/output) for live feel. Last-writer-wins on weights with a short "training in progress" lock (the module already coalesces jobs at queue depth 1). The browser's WasmIML and the module's nisps::IML use the same .nisps weight layout (vcv/NISPS-FORMAT.md), so a snapshot from either side loads losslessly — provided the architectures match. Caveat / open choice: the browser engine is fixed MLP<2,…,126> while the module is 8→{16,24,16}→16. For true weight transfer the bridged session must run a matched architecture (e.g. a browser mode configured to 8-in/16-out, within the modular envelope) — otherwise the bridge degrades to I/O + example transfer only (no raw-weight sync). Flag this explicitly.

---

7. Training over the bridge — both directions

The verdict loop (place examples / thumbs-up / thumbs-down / undo) must work from either end and stay coherent. Add one OSC address and a small RemoteTrainingBridge:

// manifold/src/engine/backends/backend.ts
export interface RemoteTrainingBridge {
  thumbsUp(): void;                 // add current (input,output) example + train
  thumbsDown(): void;               // perturb weights (explore)
  addExample(input: Float32Array, output: Float32Array): void;
  randomise(spread: number): void;
  clear(): void;
  undo(): void;
  onState(cb: (s: NispsState) => void): () => void;   // remote → local sync
}

New OSC verb: /nisps/feedback <s> carrying {"op":"up|down|rand|clear|undo","spread":f,"input":[…],"output":[…]}. The module's osc_server.hpp gets an onFeedback callback (mirroring the existing onState/onWeights at MEMLNaut.cpp:120–132) that stages the op atomically for the audio thread, which routes it through the same enqueueJob/add_example path the panel buttons use (:412–445). So:

• Browser → VCV training: user clicks thumbs-up in the Manifold → VcvBridgeBackend.remote.thumbsUp() → /nisps/feedback {op:up,input,output} → module stages → worker trains imlShadow → atomic weight swap → module streams /nisps/state back → browser WasmIML.setWeights updates so the UI/heatmap reflect the new mapping. The browser need not run its own training in bridged mode (or runs it and pushes weights; configurable — see §6.3 caveat).
• VCV → browser training: user presses +/− on the panel (or sends +TRIG) → module trains/perturbs → streams /nisps/state → browser applies, so the verdict placed in Rack appears in the browser's example list and weight-health view.
• Example placing over the bridge: either side can addExample; examples ride in /nisps/state (the module already serializes examples.features/examples.labels, :587–604) so the dataset stays in sync. Undo is local-history on each side, but a remote undo can be sent as /nisps/feedback {op:undo} to roll the module's last job (module keeps a one-deep snapshot, matching the browser undo stack semantics).

Result: the tool is fully controllable and trainable from inside VCV Rack and entirely from the browser, with the verdict loop and example-placing working either way over the same bridge.

---

8. Build / install notes

Browser side

Backends live under manifold/src/engine/backends/; lifted JS (c15-*, param-map, presets, midi-output, midi-cc-map, osc-client, visualizer) ported to TS, parity-checked. No new build step — they ride the existing Vite manifold build. COOP/COEP stays server-scoped (needed by the Powerful Synth's SAB path).

OSC bridge server

cd osc-bridge
deno run --allow-net bridge.ts           # or the compiled bridge.mjs
#   --osc-host 127.0.0.1 --osc-port 9000 --ws-port 8765 --listen-port 9001

Ship bridge.mjs (already compiled) + compile.sh so users without Deno can run it via Node. Surface "bridge not running" in the OSC/VCV backend status (the client auto-reconnects).

VCV module

cd vcv
export RACK_DIR=/path/to/Rack-SDK            # VCV Rack 2 SDK
make                                          # builds plugin.so/.dylib/.dll
make install                                  # copies into the VCV user plugins dir
# distribution: make dist   (per Makefile.dist; produces the .vcvplugin)

Per the search, the standard flow is export RACK_DIR=…; make clean; make dist (Plugin Development Tutorial). Requires the VCV Rack 2 SDK; nisps-core is header-only C++20 (symlinked under vcv/dep/ per SPEC.md). Ship v2-only (rationale in SPEC.md's v1-compat section). License caveat: VCV SDK is GPLv3, nisps-core is MPL-2.0 — combined binary is effectively GPL; not submitting to the VCV Library initially (SPEC.md §License).

---

9. Open choices for the operator

1. Bridge transport (§6.1): confirm A — WS↔OSC bridge server as default (recommended; already built and bidirectional), with WebMIDI as a live-CC-only convenience and a native in-module WS server deferred. This is the biggest call.
2. Bridged weight-sync vs I/O-only (§6.3): the browser engine is fixed MLP<2,…,126>; the module is 8→16. Either (a) run a matched 8-in/16-out browser mode for true raw-weight transfer, or (b) accept that bridged sessions sync I/O + examples only and each side trains its own weights. Recommendation: (b) for v1, (a) when the modular N×M browser MLP lands (workstream F).
3. CV/gate native path (§2.5): ship only the VCV-bridge CV alias for v1, or also build browser-native DC-coupled WebAudio CV? Recommendation: VCV-bridge first; defer DC CV.
4. Particle noise determinism (§4.1): keep the Math.random()-seeded permutation (non-deterministic per load, faithful to today) or add a ?seed= for reproducible visuals/tests? Recommendation: keep default behaviour, add an opt-in test seed.
5. Ring palette sync (§5.3): auto-generate vcv/src/palette.hpp from colors.css via a codegen step, or hand-sync? Recommendation: small codegen so a token change updates both surfaces.
6. Derived outputs on the 16-out module (§5.1): keep MEAN/STD/DELTA/NOVELTY/CONFIDENCE as menu-toggled extras / move to the expander, or drop them? Recommendation: move to the expander; keep the 16 raw outputs + LED rings as the headline panel.
7. Default active backend: confirm WebAudioBackend (firmware-parity engine) is the default, with the Powerful Synth (C15 path), particles, MIDI, OSC, CV, VCV selectable in the dock.

---

10. Cited source files (absolute paths)

• Particle algorithm (faithful port): /home/w1n5t0n/deployments/meml-aimmersive/js/ui/visualizer.js (:5–288)
• Built-in synth: /home/w1n5t0n/deployments/meml-aimmersive/js/synth/c15-adapter.js, …/c15-bridge.js, …/param-map.js (:287 curve math), …/presets.js
• MIDI: /home/w1n5t0n/deployments/meml-aimmersive/js/midi/midi-output.js (:114 batch throttle), …/midi/midi-cc-map.js
• OSC client: /home/w1n5t0n/deployments/meml-aimmersive/js/nisps/osc-client.js; param-named client …/js/synth/osc-output.js (:97)
• OSC bridge server: /home/w1n5t0n/deployments/meml-aimmersive/osc-bridge/bridge.ts (addresses :298–305, encode/decode :70–182), bridge.mjs, compile.sh
• VCV module: /home/w1n5t0n/src/MEMLNaut-NISPS/vcv/src/MEMLNaut.cpp (threading :67–312, verdict :412–445, ranges :818–843, display :699, LED :804), …/vcv/src/osc_server.hpp, …/vcv/plugin.json, …/docs/specs/vcv-module.md, …/vcv/Makefile, …/vcv/NISPS-FORMAT.md, …/vcv/res/*.svg
• Design tokens (ring palette): /home/w1n5t0n/src/MEMLNaut-NISPS/docs/redesign/manifold-export/tokens/colors.css
• Spine/engine context: /home/w1n5t0n/src/MEMLNaut-NISPS/docs/specs/engine-architecture.md (§2), …/recon/findings-design-and-manifold.md (§4), …/recon/findings-engine-surface.md, …/aimmersive-clone-spec.md (routeOutputs §6, §7 visual table, §10)

Sources (VCV SDK / widgets): VCV custom lights, Migrate2 (drawLayer/layer 1), Plugin Development Tutorial (RACK_DIR/make dist), Plugin API Guide.

Verification corrections (adversarial pass, 2026-06-27) — verdict: minor-issues

Strongly grounded: every cited file exists, the particle algorithm (§4) is reproduced faithfully, all 8 token hex values verify against colors.css, the VCV cites + OSC bridge protocol + MIDI/OSC salvage files + C15 adapter chain are accurate. Apply these fixes:

• applyCurve must keep the input clamp. Real code (param-map.js:287-291) clamps to [0,1] BEFORE the pow: Math.pow(clamp01(value), exponent). Drop the "bit-identical" wording or add the clamp to the TS port.
• ParamMeta.id vs name conflation — the example id: 'Env_A_Att' is actually the name field; keep id (slug) and name (param key) distinct when porting param-map.js.
• VCV module already at vcv/ (2-in/12-out) — evolve to 8→16, do not rebuild; reuse its threading/OSC/serialisation.

---

Implementation status (2026-06-28)

MIDI + OSC backends are BUILT in manifold/src/backends/ (OutputBackend interface + BackendManager consuming the spine; WebMIDI out with per-output CC#/ch/name/range; OSC-over-WS to the Deno bridge in manifold/osc-bridge/). The Outputs dock panel specialises per backend (manifold/src/dock/OutputsBackendConfig.tsx) with named presets (manifold/src/backends/presets.ts). Audio gated via engine.audio.setMuted on non-synth modes.

VCV module — see docs/specs/vcv-module.md "⚠️ BUILD DELTAS (2026-06-28)" for the authoritative build target: 8 inputs × 16 outputs, an LED ring per output (drawLayer + nvgArc), palette from the frontend tokens, WS↔OSC bridge (browser OSC backend → manifold/osc-bridge/ Deno relay → the module's OSC server), bidirectional training, and the nisps-core→nisps/ core-path repoint. The existing vcv/ module (2→12) is evolved, not rebuilt.

Still TODO (clear in-code): the VCV↔browser bridge browser-side wiring to the new /nisps/feedback verb; the particle backend is a no-op passthrough pending the visualizer.js faithful port; CV/gate transport.