Add a "grab → move → drop" gesture that moves an existing positive
example's output to a new input position, preserving the output. This is
the new core's home for upstream InterfaceRL's drag-store/reposition-commit,
distinct from Explore→Place (which places newly-auditioned scratchpad sounds).
- nisps/ml/feedback.hpp: begin_reposition()/commit_reposition()/repositioning().
Reuses the Placing state (static_output holds the carried vector) but a
reposition_ flag makes commit AND the mode-switch teardown SKIP the weight
restore — the real net is never set aside here, so restoring snapshot_ would
clobber the live trained net. Guards cancel_place + abort_explore_place.
- firmware glue: state-gate Toggle B. Exploring → reroll/nudge (unchanged);
Idle → MomB1 grab, MomB2 drop (commit + add_example + train). The 4D variant
has no joystick button, so the gesture lives on the momentary toggle. Also
fix a stale top-of-file control-map comment that contradicted the bindings.
- tests: 4 reposition cases (hold without snapshot; commit stores carried
output with no restore; mode-switch aborts without clobber; begin-only-Idle).
- MAP.md: document the full ExploreAndPlace lifecycle + reposition wiring.
Audio-hold (carrying the sound audibly during the move) remains the existing
unwired static_output() TODO and affects Explore→Place identically.
Firmware compile unverified (no arduino-cli); host tests + lint pass.
New SLP-Workshop firmware variant (Synth Library Portland), built on the
MEMLCelium engine + MLP shape. Ports the two post-fork learning-algorithm
changes from upstream memllib InterfaceRL into the shared nisps/ml core,
runtime-configurable (no compile-time switch), inert by default:
- nisps/ml/jolt.hpp: Jolt — held continuous weight morph over the flat
weight buffer + post-release LR ramp (kJolt* constants verbatim).
- nisps/ml/ou_noise.hpp: OUNoise<N> — Ornstein-Uhlenbeck exploration walk
on the output vector (theta=0.02, dt=0.001, kMaxAmplitude=0.65).
Both wired into ModeBase so every mode gains jolt_press/jolt_release/
jolt_lr_scale + set_explore_intensity; gated so existing modes stay
bit-identical (parity + golden tests green). Firmware surfaces them on
TogB1 (Jolt) and RVX1 (explore). New SLPWorkshopMode mode + schema +
codegen; firmware alias + .ino variant; playground mode registration.
Tests: jolt + OU unit tests, ModeBase learning integration incl. an
inert-parity test proving SLP-Workshop == MEMLCelium with features off.
Verified: cpp tests, wasm build, native↔wasm parity, lint, codegen
golden, playground typecheck. Firmware compile/e2e/hardware are
environment-bound (no arduino-cli/submodules/browser here).
Refs ergo 019f0fca.
Platform-agnostic neutral-pin mechanism (set_input_pinned/pin_value) on
ModeBase, lifted from ebb953a. Consumed by the firmware Joystick Dual/Single
settings menu; the browser fixed-MLP<4> half of ebb953a is superseded by main's
MLP<32> N-D input work and intentionally dropped.
Wire the modular input layer into the Console and reshape the browser
engine so input axes are genuine independent dimensions.
Inputs (manifold/src/inputs/):
- gamepad-source: emit press+release edges with standard-mapping labels
(enables hold-and-move); single/double-stick already present.
- midi-input-source: single-device selection + batch "MIDI Learn"
(every CC swept while armed becomes an axis); notes stay discrete.
- input-layer: compose() forwards each axis 1:1 (no mean-blend);
add onReducedInput so the manifold tracks gamepad/MIDI position.
- types: InputAction.phase, InputMode.
Console (manifold/src/console/):
- ConsoleApp: bind gamepad buttons to verdicts (RB up / LB down /
X randomise / Y nudge / B undo / A-hold reposition); mirror composed
position onto the manifold.
- Drawers: rebuilt Inputs drawer (source picker, gamepad legend, MIDI
device picker + batch-learn flow, learned-control meters).
Engine (nisps/wasm, manifold/src/engine):
- DefaultMLP widened MLP<2,..> -> MLP<32,..> (32 = MAX_AXES); each
active axis gets a dedicated slot, unused slots held at 0 (inert).
Rebuilt nisps.wasm (playground + manifold).
- spine/engine-api: setInputs writes the full N-D vector (was dropping
arr[2+]); primary pair keeps the 2-D pipeline; process() re-ticks the
whole vector via spine.reprocess().
Tests:
- parity_check/parity_wasm: ParityMLP -> 32 inputs, widen example bufs.
- CMakeLists: build parity binary with -ffp-contract=off so native
matches FMA-free WASM (training amplified the gap past 1e-5).
Inputs dock is still an exclusive picker; mixing toggles, reshape modal,
and the >2-D slider view (inputs-spec.md) are groundwork-laid but not
yet wired. See docs/redesign/midi-gamepad-inputs-worklog.md.
CMakeLists.txt:
- Native host build by default; Emscripten-target detection plumbed but
WASM emit deferred to stream 7 (playground build script).
- Header-only INTERFACE library `nisps_core`.
- Host test executable `nisps_core_tests` compiled with -Wall -Wextra
-Werror -Wpedantic (Chris's rules: clean build is non-negotiable).
tests/cpp/test_helpers.hpp:
- Minimal NISPS_TEST / NISPS_EXPECT / NISPS_EXPECT_NEAR macros, no external
deps. Rationale documented in-file: Catch2/doctest would add ~10MB and 30s
for what is currently <100 LOC of test runtime.
22 unit tests covering FixedBuffer (5), RingBuffer (5), Rng (7), math (5).
All green; verified via `cmake --build nisps/build && ./nisps/build/nisps_core_tests`.
b290144 made matrix cells opt-in to prevent joystick-silences-voice,
but that broke modular-ui.updateLive(): the matrix DOM stopped
reflecting live MLP outputs because matrixIndexCache was empty when
buildMatrixIndex() walked paramMeta. This was the same regression
6072fe8 had previously fixed.
Fix it structurally at the DSP layer instead: amp_val now computes
as `clamp(base_amp + max(0, mod_amp)) * level * vel_gain`, so matrix
d08_amp cells can only boost the amp floor — never cut it. base_amp
defaults to 1.0 (always audible), and presets that want classic
envelope-gated voices (slow pad, plucky bass, crystal, morphing
drone) drop base_amp to 0 and layer a positive ADSR→amp route on top.
With the DSP guard in place, all 480 matrix cells can safely live
in paramMeta again, and updateLive() gets its live visual feedback
back. Revert the opt-in gate in _rebuildParamMeta and the 32-param
test counts, and add a regression test asserting that every matrix
destination has 48 cells in paramMeta — that's what updateLive needs.
Modular sub-engines computed amp_val as a pure function of mod_amp (the
matrix d08_amp destination sum), so once the MLP drove the matrix cells
every joystick movement had a chance to silence the voice: matrix cells
have signed range [-1, 1], sigmoid outputs near 0.5 denormalise to 0,
and the amp gate collapsed. Additive survived in scattered regions
because it only has one kill-switch (d08_amp); subtractive and fm were
almost always dead because they also have d05_cutoff and d01_op1_level.
Two changes:
1. DSP: each sub-engine gets a base_amp hslider (default 1.0) so
amp_val = clamp(base_amp + mod_amp) * level * vel_gain. At the
default the voice is always fully open and d08_amp modulation is
purely additive decoration; drop base_amp to 0 for classic
ADSR-gated VCA behaviour.
2. ModularEngine._rebuildParamMeta: restore the _exposedMatrixCells
gate (default empty). paramCount drops from 512 to 32 (4 ADSR * 4
+ 8 LFO * 2); matrix cells are opt-in via setExposeMatrixCell.
_applyDefaultPatch no longer writes s00_d08_amp since base_amp
keeps the voice audible without routing.
Tests updated for the new 32-param baseline; matrix-cell persistence
test now calls setExposeMatrixCell(1, 5, true) before asserting the
cell lands in paramMeta. Drive-by: engine-switching test bumped from
3 to 4 engine cards (stale since the modular engine was added).
New "Modular" engine in a-immersive with three hot-swappable Faust
sub-engines (subtractive/additive/fm) sharing a common modulation pool:
16 ADSR slots + 32 LFO slots (single-knob sine->tri->square->saw
wavemorph) routed through a 48-source x 10-destination matrix per
engine. Per-connection scalar amounts in [-1, 1], summed at each
destination. Default MLP output count is 512 (32 mod-source params +
480 matrix cells); model reinits on sub-engine swap, count change, or
engine-param exposure toggle.
Faust layer:
- mod-pool.lib: shared ADSR/LFO/source-bus library
- gen-modular-dsp.py: byte-reproducible generator (source of truth)
- modular-subtractive: faithful Minimoog (3 osc, ladder filter, no envs)
- modular-additive: 64-partial, spectral shape + formants, no envs/LFOs
- modular-fm: 4-op matrix + self-feedback, no envs
- All three share d08=amp, d09=pan conventions
- MODULAR_DESTINATIONS.md: authoritative destination table
JS layer:
- ModularEngine: self-contained SynthEngine with getState/setState,
setSubEngine, setModSourceCount, setExposeEngineParam
- modular-ui: drawer with sub-engine toggle, ADSR/LFO count steppers,
per-slot enable switches, matrix grid editor (tap-cycle, long-press
precise, right-click menu, negative amounts), preset overlay
- modular-presets: 6 named presets (Slow pad, Plucky bass, Crystal,
DX bell, Morphing drone + default)
- a-app.js: Modular mode registered, paramMeta:change -> resizeMLP,
modular DSP state persisted under modularDspState, window.__nisps
debug hooks for programmatic control
Tests: tests/e2e/modular-mode.spec.js (11 Playwright tests, all passing
including DSP state survives reload, sub-engine swap keeps paramCount,
preset apply verification).
Also fixes a pre-existing build.sh bug where the -e flag caused faust
to overwrite .wasm outputs with expanded DSP source text, leaving
additive/fm-matrix/eoc-* committed as invalid WebAssembly. Rebuilt all
affected engines with the corrected script. Added an early-message
buffer to faust-worklet-processor.js so setParam calls arriving before
wasm instantiation are queued rather than dropped (needed when the user
configures modular state before clicking Start Audio).
Tests cover:
- Default state (C15, 126 params, heatmap cells)
- Engine switcher UI (3 cards, active state)
- Switch to Additive (48 params, outputs bounded, training works)
- Switch to FM Matrix (55 params, outputs bounded, training works)
- Round-trip switching (C15→Add→C15, C15→FM→Add→C15)
- Warm-start weight preservation across resize
- EOC chain accessibility across engines
- SynthVisualizer visibility across engines
- No console errors during any switch
Also fixes: faustJsonToParamMeta now filters [hidden:1] params,
giving correct counts (48 additive, 55 FM) instead of including
freq/gate/_vel control inputs.
- Add window.__nisps debug probe (gated on ?debug=1) exposing iml state,
getOutputs/getLoss/getWeights/getExampleCount, and action triggers
(thumbsUp/thumbsDown/train/randomise/clearExamples/saveState)
- Fix WasmIML bug: this.dataset was a plain object; import Dataset and
use new Dataset(100) so computeWeights() is available for training
- Fix WasmIML.addExample/clearDataset to use Dataset API methods
- 44 Playwright e2e tests across 4 spec files:
- ml-engine.spec.js: WASM inference bounds, training loss, thumbs
up/down behavior, async training, example capture semantics
- ui-interactions.spec.js: drawer open/close, mode switching,
heatmap bar counts, preset chips, keyboard shortcuts (1/2/Z)
- input-pipeline.spec.js: input→output variation, clamping, joystick
drag, post-training output bounds across the full input space
- persistence.spec.js: URL params (?preset, ?spread), localStorage
round-trip, saveState probe
