1.1 SLP-Workshop is a distinct nisps mode (`nisps::modes::SLPWorkshopMode`, mode_id `slp_workshop`) that **MUST** reuse the MEMLCelium engine and MLP shape verbatim: engine_id stays `memlcelium`, and the MLP is `MLP<4, 10, 14, 18, 56>`.
**Why:** the workshop wanted its own identity (preset directory, display name) and a UI that foregrounds the two adaptive-learning gestures, but no new synthesis behaviour — so the synthesis mapping is byte-for-byte MEMLCelium's, and only the mode wrapper, identity, and surfaced controls differ.
1.2 The synthesis mapping in SLP-Workshop **MUST** stay identical to `MEMLCeliumMode` for the same seed and inputs.
**Why:** the two adaptive-learning gestures live in the shared base (§5), not in this mode; SLP-Workshop adds only identity and control surfacing. A test asserts SLP-Workshop ≡ MEMLCelium (same seed, identical audio) with the new features off.
1.3 Firmware compiles exactly one mode at a time (`MEMLNAUT_MODE_TYPE`), so the per-mode code cost of an extra mode is negligible.
**Why:** modes are selected at compile time, not linked together; an unused mode contributes nothing to the firmware image.
---
## 2. Why the gestures exist (the investigation)
2.1 Upstream `MusicallyEmbodiedML/MEMLNaut-NISPS` advanced its `memllib` submodule past our fork point (`188496d` → `e291192`). Both upstream MEMLCelium and DJFX share the **same** learning code, `InterfaceRL` (reinforcement: thumbs + joystick); there is no per-mode learning algorithm.
**Why:** establishes that the only thing to port was learning-algorithm changes in one shared file, not mode-specific behaviour.
2.2 The only post-fork **learning-algorithm** gains our fork lacked were **Jolt** (upstream commit `9fcd459`) and **Ornstein-Uhlenbeck exploration noise** (`d0d8a72`). These two **were** ported.
**Why:** scopes the port to exactly these two algorithms.
2.3 The upstream "dislike / geometric-push" cluster **MUST NOT** be ported.
**Why:** it depends on a firmware-only `ReplayMemory` our fork intentionally does not keep (recorded in `ALIGNMENT.md`); porting it would reintroduce that buffer.
2.4 The upstream `staticmlp` refactor (`770a990`) **MUST NOT** be ported.
**Why:** our `MLP` already exceeds it — it is already a fixed-size, no-heap template.
2.5 DJFX's FX DSP chain **MUST NOT** be ported.
**Why:** the operator explicitly did not want DJFX's FX; only the learning changes were in scope.
---
## 3. Jolt (`ml/jolt.hpp`)
3.1 Jolt is a **HELD** gesture operating on the MLP's flat weight buffer via `get_weights()` / `set_weights()` / `weight_count()`, so it **MUST** be architecture-agnostic (independent of layer count or width).
**Why:** the same gesture must work unchanged across every mode's MLP shape.
3.2 On press, Jolt **MUST** pick `num_weights = 40` random global indices into the flat weight buffer, each with a random target in `[-1.2, 0.9]`.
**Why:** these are the upstream `kJolt*` constants reproduced verbatim; the target range matches the weight-init range so jolted weights stay in a plausible regime.
3.3 Each control tick while held, for each selected weight `w`: `w += 0.017 * (target - w)`; when `|target - w| < 0.05`, the target for that weight **MUST** be re-rolled.
**Why:** an EMA glide toward a moving target means the morph never settles — re-rolling on arrival keeps the sound continuously evolving while the button is held.
3.4 On release, Jolt **MUST** freeze the weights where they landed (the change is permanent) and re-arm a learning-rate ramp that climbs `0 → 1` at `0.001`/tick (~5 s @ 200 Hz). `jolt_lr_scale()` returns `0` while held, then the ramping value.
**Why:** the morph is meant to be a committed edit, and easing training back in afterward prevents a resumed trainer from immediately yanking the net off the just-jolted sound. Callers that train multiply their LR by `jolt_lr_scale()`; pure-example modes that train only on explicit gestures may ignore it.
3.5 Jolt **MUST** own a per-instance deterministic `Rng` (seeded by the caller).
**Why:** per-instance deterministic RNG is the project-wide rule that cross-platform parity tests rely on.
3.6 A freshly constructed Jolt **MUST** be inert: inactive, `step()` a no-op that does not advance the RNG, and `lr_scale()` == 1.
**Why:** inertness is what makes adding Jolt to the shared base parity-safe — a mode that never presses the button behaves bit-identically to one without Jolt (§5).
---
## 4. OU explore (`ml/ou_noise.hpp`)
4.1 `OUNoise<N>` adds an Ornstein-Uhlenbeck random walk to the N-channel output vector **before** it reaches the engine.
**Why:** perturbing the post-inference parameter vector (not the weights) lets the mapping stay the anchor and reverts the walk toward it.
4.2 Per channel the update **MUST** be `state += theta * (-state) * dt + noise`, with `theta = 0.02`, `dt = 0.001`, and per-step noise std `= stationary_std * sqrt(2 * theta * dt)` times a unit gaussian; then `out = clamp(out + state, 0, 1)`.
**Why:** these are the upstream OU constants; the `sqrt(2·theta·dt)` factor makes the discrete walk's stationary standard deviation equal the requested `stationary_std`. Mean reversion is to zero (mu = 0), so the network's own output remains the anchor.
**Why:** `0.65` is the upstream full-scale amplitude in parameter space.
4.4 The walk **MUST** be temporally correlated (smooth multi-second sweeps), not per-frame jitter, and learning **MUST** stay live while it roams.
**Why:** correlated drift produces explorable sound trajectories rather than noise, and keeping training live means "likes" registered during the wander steer the network toward what the player wants.
4.5 `OUNoise`**MUST** own a per-instance deterministic `Rng`, and **MUST** default inert (intensity 0, `apply()` a no-op that neither advances the RNG nor touches the output).
**Why:** same parity-safety rationale as §3.5–3.6 — intensity 0 leaves the output pass-through bit-identical.
---
## 5. Integration into the shared base
5.1 Both gestures **MUST** be wired into the shared CRTP base `nisps::ModeBase` (`nisps/modes/base.hpp`), so every mode gains `jolt_press()` / `jolt_release()` / `jolt_active()` / `jolt_lr_scale()` and `set_explore_intensity()` / `explore_intensity()`.
**Why:** the gestures are mode-agnostic; putting them in the base avoids per-mode duplication. SLP-Workshop is simply the mode that surfaces them.
5.2 Gesture selection **MUST** be runtime, not compile-time: there is no `#ifdef`/template flag choosing examples-vs-RL or enabling Jolt/OU. Both gestures are plain methods on the same MLP/mode, selected by which control the user touches.
**Why:** runtime selection was a user requirement — the player switches gesture by reaching for a different control, with no rebuild.
5.3 Because §3.6 and §4.5 make both gestures inert by default, existing modes **MUST** stay bit-identical. In `ModeBase::tick_control` the jolt block is guarded by `if (jolt_.active())` and the OU block by `if (ou_.enabled())`; otherwise the original direct path runs.
**Why:** parity-safety: untouched modes hit the original code path unchanged. Proven by the SLP ≡ MEMLCelium test (§1.2) plus unchanged golden and native↔WASM parity.
5.4 The jolt weight-copy **MUST** be an out-of-line `apply_jolt_()` marked `NISPS_NOINLINE` (a macro added to `nisps/core/perf.hpp`), and the members `jolt_`, `ou_`, `out_buf_`, `jolt_buf_`**MUST** be declared **after**`events_` in `ModeBase`.
**Why (load-bearing):** inlining the large get/set-weights copy into `tick_control`, or placing the large `jolt_buf_` before the lock-free `events_` ring buffer, each provoked a spurious GCC `-Wstringop-overflow` (an error under `-Werror`) on the ring's atomic index at `-O3`. Both arrangements are behaviourally identical; this layout is purely to dodge the false-positive diagnostic.
---
## 6. Control mappings
6.1 Firmware (`firmware/MEMLNaut-NISPS/glue/peripherals.hpp`, generic to all modes): **TogB1** toggle **MUST** drive Jolt (up = morph/hold, down = freeze) and **RVX1** pot **MUST** drive exploration amount via `set_explore_intensity`.
**Why:** Jolt is a held gesture, but the MEMLNaut momentary buttons fire only on press — a toggle gives a stable held state. These ride on previously-unused inputs so existing mappings are undisturbed.
6.2 Browser playground (`SLPWorkshopMode.tsx`, via `mode-runtime.ts``jolt`/`explore`) **MUST** expose a press-and-hold "⚡ Jolt" button and an "Explore" 0..1 slider; the mode runs the MEMLCelium engine via WASM.
**Why:** mirrors the firmware control surface in the browser playground.
---
## 7. Schema & codegen
7.1 `schemas/modes/slp_workshop.json` (mode_id `slp_workshop`, engine_id `memlcelium`) **MUST** declare 56 params identical to MEMLCelium, and codegen **MUST** emit `nisps/modes/generated/slp_workshop_schema.hpp` and `playground/src/modes/generated/slp_workshop_schema.ts`.
**Why:** the mode shares MEMLCelium's parameter contract; codegen auto-discovers `schemas/modes/*.json`, and the golden test enforces byte-identical regeneration.
---
## 8. Firmware wiring & build
8.1 The firmware variant alias `MEMLNautModeSLPWorkshop`**MUST** exist in `firmware/MEMLNaut-NISPS/glue/mode_select.hpp`, with a selectable `MEMLNAUT_MODE_TYPE` line in `MEMLNaut-NISPS.ino`; `scripts/build-firmware.sh` auto-discovers it as variant `SLPWorkshop`.
**Why:** this is the standard firmware-mode registration path; auto-discovery means `scripts/build-firmware.sh SLPWorkshop` works without extra wiring.
9.1 `memlcelium` is **one** MLP (`MLP<4,[10,14,18],56>`), not two. Its 56 outputs split: `[0..13]`**sequencer** (2 sequences × 7 ratio-seq params), `[14..55]`**synthesis** (42 continuous params, Voice 0 + Voice 1). The single net produces both the continuous values and the RatioSeq params; an internal RatioSeq tick turns the seq params into note triggers, exposed via `pop_events()`.
**Why:** the planned output modes are *slices/reshapes of this one net*, and the gate stream already exists inside the engine — so gate outputs are a routing/consumption feature, not new DSP. (A true two-MLP-head "split" topology is genuinely different and therefore browser-only; §11.2.)
9.2 The 7 ratio-seq params per track are: `ratios[0..2]` (3), `phasor_mul` (1), `phase_off` (1), `amp_ratios[0..1]` (2).
**Why:** firmware parity. (The April browser uSEQ-Celium used 8, adding a pulse-width param; pulse width is an optional 8th later — §13.)
---
## 10. The unifying model: two output streams
10.1 Everything collapses to two output **streams**, each driven by an MLP head:
| **Continuous** | 1 value per channel | smooth 0..1 values | MIDI CC / CV jack |
| **Rhythm** | 7 params per gate-sequence track | clock-driven gates | MIDI note / gate jack |
A **mode** is just *which streams are active* and *whether they share a network*.
10.2 The Rhythm stream **SHOULD** generate gates via RatioSeq, per track per control tick: (1) a shared internal-BPM clock advances a bar phasor; (2) `seq_phasor = (bar_phasor × phasor_mul + phase_off) mod 1`; (3) `ratio_seq_3(seq_phasor, ratios, pw=0.5)` → boolean gate; (4) `ratio_seq_2(amp_ratios)` → 2-level velocity (127/64); (5) rising edge → note-on / gate-high, falling edge → note-off / gate-low.
**Why:** a gate is **clock + learned pattern**, not a threshold on a continuous value — this is what makes Rhythm a distinct stream and why it needs its own 7 params per track rather than one output per gate. Defaults (pulse-width 0.5, 2-level velocity) match firmware.
10.3 Where RatioSeq runs (browser): when the active engine *is* the memlcelium/SLP WASM engine, consume `pop_events()`; for CV/MIDI modes that don't run that audio engine, a small **TS RatioSeq** fed by the Rhythm MLP's 7-params/track drives the gates.
**Why:** the gate generator must work whether or not the WASM audio engine is the active backend.
---
## 11. Modes
11.1 **Firmware (compile-time, one chosen at build)****MUST** always use exactly 2 ratio sequences (memlcelium verbatim). Three slices of the `slp_workshop` mode:
**Why:** the firmware does not vary sequence count (locked decision); exact output_size counts for the sliced modes are the firmware agent's call — this spec fixes the shapes.
11.2 **Browser (Manifold, dynamic, switchable live)****MAY** offer the same three plus a **split-nets** variant (a separate Continuous MLP and Rhythm MLP), with **as many ratio sequences as the user wants** (each gate sequence = its own 7-param track; the net reshapes to suit). The browser mode is **implied by the Outputs config** (§12), not a separate picker: continuous>0 & gates=0 ⇒ Continuous-only; gates>0 & Shared net ⇒ hybrid-shared; gates>0 & Separate net ⇒ hybrid-split; continuous=0 ⇒ Rhythm-only.
**Why:** the browser has the compute for two MLP heads and dynamic reshape; making the mode a consequence of the config avoids a redundant mode dropdown. Shared-net `output_size = continuous + 7 × n_gates` must respect the 126-output WASM cap (~16 gates max shared); a separate Rhythm net is sized independently and scales further.
12.1 Two output kinds: **Continuous** → CC (MIDI) / CV jack (CV); **Gate sequence** → note (MIDI) / gate jack (CV), each = one RatioSeq track.
12.2 **MIDI mode****SHOULD** expose two independent counts — Continuous (CC) and Gate-sequences — capped by the model output budget.
12.3 **CV mode** hardware is fixed (11 PWM/CV-capable jacks + 3 digital gate-only jacks), so the two counts are **linked** and gates are optional. The rule **MUST** be `gates ∈ [0, 14]`; `CV = gates ≤ 3 ? 11 : 14 − gates`.
**Why:** the first 3 gate sequences land on the dedicated gate-only jacks and cost no CV (0→11 CV, 3→11 CV, 6→8 CV, 14→0 CV); converting CV jacks only begins past 3. Pure CV (0 gates) is valid.
12.4 Wire-protocol impact **MUST** be none: a CV jack acting as a gate carries 0/full (or the 2-level velocity) in its `u16` slot; the 3 dedicated gate bits stay digital pins (`docs/specs/useq-cv-protocol.md`). `CvSpec` extends so PWM jacks can also be gate targets.
**Why:** reusing the existing protocol slots avoids a protocol revision for a routing feature.
12.5 The **Rhythm network** toggle (shown only when gate sequences > 0) selects **Separate** (default) or **Shared**. Separate gives the Rhythm stream its own MLP and routes each input channel to one engine, automatically by source kind: XY pad → a second on-screen pad (pad 1 → Continuous, pad 2 → Rhythm); gamepad → double-stick (left → Continuous, right → Rhythm); MIDI controller → per-CC `Continuous | Rhythm` toggle. Shared uses one MLP for both streams (all inputs feed it; the hardware-parity hybrid).
**Why:** the decision surface stays **two numbers + one toggle** — the second pad / double-stick / per-CC tag is a *consequence* of Separate, surfaced inline, reusing the existing input layer (`InputSource.axisCount()` / `axisLabels()`).
12.6 `BackendAdvanced` (full-depth modal) **MAY** override per-output defaults: which model output drives each CC/CV; which rhythm track drives each note/gate; gate threshold; MIDI note#/channel; CV polarity.
**Why:** identity defaults cover the common case, so advanced overrides stay optional.
---
## 13. Proposed UI (the "simple" target)
13.1 The Outputs panel **SHOULD** read, top to bottom: (1) **Output kinds** — `Continuous [n]` + `Gate sequences [n]` (MIDI) or the linked `Gate sequences 0–14` slider with a live `CV n · Gate n` readout (CV); (2) the **Rhythm network** toggle `( Shared | Separate )` plus its inline input surface (2nd pad / double-stick / per-CC tags), shown only when gate sequences > 0; (3) the existing per-output rows (off/fixed/live, mute, arm, min/max/curve) and per-backend specifics (CC#, CV jack, note#).
**Why:** the counts + toggle *are* the mode, so no mode dropdown is needed for the rhythm/continuous split.
- **Browser↔firmware noise parity is intentionally NOT guaranteed.** `playground/src/ml/jolt.ts` and `playground/src/output/ou-explore.ts` reimplement the gesture math in TypeScript (driving weights via the existing `nisps_ml_get/set_weights` bindings) rather than calling C++ `ml::Jolt` / `ml::OUNoise` through WASM, and they use `Math.random()` not the deterministic `Rng`. Acceptable for stochastic exploration aids, but the noise itself will not be bit-identical across firmware and browser.
- **The post-release LR ramp (§3.4) is not ported to the browser.** The browser SLP mode trains only on explicit gestures, so there is no resumed continuous trainer to ease back in.
- **Verified in the dev environment:** C++ host tests (`tests/cpp/test_mlp_jolt.cpp`, `test_mlp_ou_noise.cpp`, `test_mode_learning.cpp`, `test_mode_concepts.cpp`), lint, WASM build, native↔WASM parity, codegen golden, playground typecheck.
- **NOT verifiable in the dev environment (hardware/toolchain-bound; verification chokepoints A/B):** firmware compile (no arduino-cli / submodules), Playwright e2e, on-hardware audio. To close: run `scripts/build-firmware.sh SLPWorkshop`, then flash on the RP2350.
Part II (planned) build deltas, when we proceed:
- **Manifold engine:** today one MLP head on the spine. "Separate" needs a second Rhythm MLP head + per-input-axis engine routing (engine + input-layer scope).
- **TS RatioSeq:** a browser-side RatioSeq (or a `pop_events()` bridge) to turn Rhythm-MLP params into gate/note events for the CV & MIDI backends (2 sequences on firmware; arbitrary count in the browser).
- **MIDI backend:** gate sequences → note on/off (it currently sends CC only).
- **CvSpec:** allow PWM jacks (`cv1..cv11`) to be gate targets (incl. velocity-CV), with the 3 digital pins as the first-used gate jacks.
- **Firmware output_size** for the "Continuous only" / "Rhythm only" slices of `slp_workshop` — the firmware agent's call; this spec fixes the shapes, not exact counts.
- **Deferred niceties (explicitly not blocking):** external-MIDI-clock sync; per-track pulse-width (gate length, the optional 8th ratio-seq param); continuous velocity/accent beyond the 127/64 two-level.
