Move the lazy AudioCanvas initialization above the outputCountForMode()
call in setOutputMode() so that the real output count is available
instead of the 12-output fallback.
- Per-file loop count (0-4) with stepper UI on each waveform cell;
dynamic MLP output sizing (8 range + N_loops × 4 position outputs)
- Beat-synced loop length: global bars/beats selector (2/4/8/16/32/64)
with per-loop overrides via long-press dropdown, persistent badges
- Global param range groups: dual-thumb range sliders (volume, pitch,
position, filter) with ML-driven min/max and per-loop position mapping
- Pitch lock toggle (1×) to freeze playback rate across all submodes
- Fix: _resizeOutputs now syncs _loopOutputs to prevent stale reads
- Fix: granular visuals consolidated per-clip to prevent multi-loop flicker
When switching from C15 to Additive or FM engines, the MLP was resized
to 0 outputs because paramMeta wasn't loaded until init() (which needs
AudioContext). The SynthVisualizer also disappeared because its section
map was hardcoded to C15's 18 param groups.
- Add FaustEngineBase.loadParamMeta() for eager JSON-only loading
- Call loadParamMeta() before setActiveEngine() in engine switch handler
- Add SynthVisualizer.rebuild(paramMeta) to derive sections from groups
- Add rebuildParamToSection() / restoreC15ParamToSection() for engine-agnostic
param-to-section mapping
- Fix tooltip to use engineParamOverrides for non-C15 engines
- Engine switcher now lives in the Synth drawer (was incorrectly in NISPS params drawer)
- Params drawer renamed to "NISPS" (it's for ML engine tuning, not synth engine selection)
- Output mode button says "Synth" (generic) not "C15 Shaper-Feedback" (engine-specific)
- Dock icon for params drawer now labelled "NISPS" with correct tooltip
Additive engine: 5 presets across 3 tiers (Spectral Basics, Formant Play,
Harmonic Sculptor, Phase Explorer, Full Spectrum). FM engine: 5 presets
across 3 tiers (Simple FM, Bell Tones, Matrix Explorer, Feedback Machine,
Full Matrix).
Engine-aware preset system: applyPreset() dispatches to C15 groupOverrides
or new flat engineParamOverrides depending on active engine. Preset selector
dropdown rebuilds on engine switch. URL ?preset param searches all engines.
- Add totalOutputCount() helper: engine params + EOC params when nispsMode === 'shared'
- setActiveEngine() calls resizeMLP(totalOutputCount()) and rebuilds heatmap with combined paramMeta
- eoc:change listener handles nispsMode-changed and module-* events: resizes MLP, rebuilds heatmap, toggles .shared-mode CSS class
- routeOutputs() slices outputs: indices < engineParamCount go to engine, remainder to eocChain.setParam() in Shared mode
- CSS: .heatmap-strip.shared-mode adds green bottom border and '+ FX' label to signal combined output space
- Add AdditiveEngine (48 params) and FMEngine (55 params) wrapping FaustEngineBase
- Remove comingSoon flags from ENGINES descriptors
- Replace stub onSwitch handler with real engine construction + init + setActiveEngine
- Fix setActiveEngine EOC rewire to work with both C15 (_bridge.audioContext) and Faust (_audioCtx)
- Fix _startEocChain audioCtx resolution to support Faust engines
- Fix synth param send loop: use N_OUTPUTS instead of hardcoded SYNTH_PARAM_MAP.length
- Add rebuildHeatmap(paramMeta): resets rawParamValues/_lastSentParams and rebuilds
#heatmap-cells from engine paramMeta; colors from SYNTH_PARAM_COLORS for C15,
_colorFromGroup() hash for other engines
- Replace static SYNTH_PARAM_NAMES/COLORS lookups in buildHeatmap, setHeatmapValue,
and showParamPopup with activeEngine.paramMeta[index] lookups
- Scope MIDI CC localStorage key per engine via midiCCStorageKey()
- Add reloadMidiCCMap() helper; called in setActiveEngine() after engine switch
- loadCCMap/saveCCMap in midi-cc-map.js accept optional key parameter
Adds EngineSwitcher module with clickable engine cards (active highlight,
coming-soon greyed state, loading bar animation, confirm-on-switch when
training data exists). Wired into the Engine drawer; C15 active,
Additive/FM marked coming-soon with toast. Dock title reflects active
engine; engineId persisted to/restored from localStorage.
- input-pipeline.js: guard circular clamp against div-by-zero when
input is exactly at center (0.5, 0.5) — dist=0 produced NaN
- snapshot-stack.js: jumpTo() used slice(0, index) which excluded the
target snapshot; fixed to slice(0, index + 1)
- a-app.js: _lastSentParams not resized when MLP output count changes
during mode switch, causing stale throttle state and potential param
flood on first frame after switch
Add WasmIML.extractWeights() and WasmIML.createWithWarmStart() to preserve
learned joystick mappings across output-count changes; resizeMLP() now
transfers hidden-layer weights and shared output nodes instead of cold-starting.
Add evalLoss, inferBatch, and getLayerStats to the window.__nisps
debug probe so Playwright tests and dev console can access the new
WasmIML capabilities. InputHeatmap (Phase 3) is not yet wired into
a-app.js, so batch inference heatmap integration is deferred.
- Add window.__nisps debug probe (gated on ?debug=1) exposing iml state,
getOutputs/getLoss/getWeights/getExampleCount, and action triggers
(thumbsUp/thumbsDown/train/randomise/clearExamples/saveState)
- Fix WasmIML bug: this.dataset was a plain object; import Dataset and
use new Dataset(100) so computeWeights() is available for training
- Fix WasmIML.addExample/clearDataset to use Dataset API methods
- 44 Playwright e2e tests across 4 spec files:
- ml-engine.spec.js: WASM inference bounds, training loss, thumbs
up/down behavior, async training, example capture semantics
- ui-interactions.spec.js: drawer open/close, mode switching,
heatmap bar counts, preset chips, keyboard shortcuts (1/2/Z)
- input-pipeline.spec.js: input→output variation, clamping, joystick
drag, post-training output bounds across the full input space
- persistence.spec.js: URL params (?preset, ?spread), localStorage
round-trip, saveState probe
Add MIDI CC preset system for bundled device configurations:
- midi-cc-presets.js: listPresets/loadPreset/loadPresetFromFile API
- presets/polybrute.json: Arturia PolyBrute CC map
- Preset selector in quick controls bar and MIDI CC drawer
- File import button for loading JSON presets from disk
Bug fixes in a-app.js:
- Destroy IML instances before resizeMLP() to free WASM memory
- Fix randomiseWeights() call (was drawWeights())
- Fix thumbs-up to store rawParamValues instead of post-pipeline outputs
Browser-side OSCOutput module sends parameter values over WebSocket to
a companion Deno bridge script that converts them to OSC/UDP messages.
Enables controlling SuperCollider, Max/MSP, Pure Data, TouchDesigner,
or any OSC-capable software from the NISPS playground.
- Browser module (js/synth/osc-output.js): WebSocket client with
auto-reconnect, throttle (~20fps), and dead-zone filtering
- Deno bridge (osc-bridge/bridge.ts): zero-dependency, compiles to
standalone binaries via deno compile for Linux/macOS/Windows
- Test receiver (osc-bridge/test-receive.ts): terminal dashboard
showing live OSC parameter values with bar charts
- OSC pill button in floating bar for quick connect/disconnect
- Help modal section with platform-aware download, setup guide,
and examples for SuperCollider/PD/Max
- GitHub Actions workflow for cross-platform binary builds
- OSC sends in both visual and synth modes
Restore the original arpeggiator as the default sequencer. ShapeSeq
code is preserved but gated behind a URL feature flag (?shapeseq=1):
- Arpeggiator import, state, and all wiring fully restored
- ShapeSeq modules loaded via dynamic import() only when flag is on
- ensureShapeSeqInit() creates engine/viz/UI lazily on first audio start
- animate loop routes inputs to ShapeSeq only when flag + playing
- setOutputMode shows/hides ShapeSeq container only when flag is on
- HTML: ShapeSeq container added but hidden by default
- CSS: ShapeSeq styles added (inert when container is hidden)
All 13 ShapeSeq modules in playground/js/shapeseq/ are untouched.
To test ShapeSeq: add ?shapeseq=1 to the URL.
- Remove Arpeggiator import and all arp references from a-app.js
- Add ShapeSeq imports (ShapeSeqEngine, StepVisualizer, ChainBuilderUI,
getDefaultBus) and lazy initialization on first audio start
- In synth mode: show ShapeSeq container (circular viz + chain builder),
hide flow-field particles. Visual mode unchanged.
- Route joystick or hand tracking inputs to shapeSeq.setSequenceInputs()
each frame; render step viz at 60fps
- Add HTML: #shapeseq-container with #shapeseq-viz canvas and
#shapeseq-chain wrapper, sequencer transport controls (play/stop,
tempo, step count)
- Add CSS: full-screen flex layout, viz takes upper 45%, chain builder
scrollable below, hidden by default
- Quick play button wires to ShapeSeq start/stop instead of arp
Closes meml-6ry. This completes ShapeSeq Phase 1.
Hand tracking at 30fps was writing 126 params per frame to the C15 ring
buffer (capacity 512), flooding it and starving arpeggiator noteOn/noteOff
messages — causing stuck/dropped notes.
Three-part fix:
- Arpeggiator now runs in a dedicated Web Worker with direct
SharedArrayBuffer access, bypassing the main thread entirely for
note timing. Worker setInterval isn't subject to main thread jank.
- RingBufferWriter uses CAS (Atomics.compareExchange) for multi-producer
safety — both main thread (params) and worker (notes) write safely.
- routeOutputs throttles synth parameter sends: dead-zone filter
(0.2% change threshold) + rate cap (~20fps), reducing ring buffer
pressure from ~3800 msg/s to ~50-100 msg/s of actual changes.
Hand tracking via webcam as alternative to joystick input:
- Right hand tracks 14 derived features (palm XY, finger curls, spread,
roll, pitch, pinch) through a separate 14-input MLP
- Left hand gesture recognition (1 finger = thumbs up, 2 = thumbs down)
with 400ms debounce hold
- Split-zone PIP display (no camera feed, skeleton only) with dashed
divider and cross-zone dimming
- Dual IML architecture: independent imlJoy (2 inputs) and imlHand
(14 inputs) with pointer swap, preserving training data per mode
- Dev panel (?devmode=true): draggable/collapsible floating panel with
sliders for MediaPipe confidence thresholds, smoothing, gesture hold
time, world landmarks toggle, and live feature bar monitor
- Visual presets always route to joystick IML (prevents dimension mismatch)
- Race condition guard on input mode switching
Compile nisps-core C++ MLP to WASM (36KB) and use it as the ML engine
in the playground, replacing the JavaScript port for inference, training,
and weight manipulation.
- Add extern "C" WASM bindings with spread-aware drawWeights/moveWeights
- WasmIML class is a drop-in replacement for the JS IML
- Inference runs on main thread via WASM (fast, synchronous)
- Training runs in a Web Worker with its own WASM instance (non-blocking)
- Interactive training (thumbs-up, train button) no longer freezes UI/audio
- Preset loading and state restore still use sync training
4 tiers of progressive complexity (Beginner 15 params → Expert 126),
13 presets total with per-param min/max/curve overrides that bias
distributions without clamping extremes. Preset dropdown in UI,
persisted to localStorage, supports ?preset= URL param.
SVF count was 7 (actual 9), Gap Filter section was missing (6 params),
FB Mix count was 10 (actual 9) — causing cascading mislabeling of all
synth sections from SVF onwards. Also sync volume/tempo changes from
bottom sheet back to quick-play controls.
- Glass-style modal explains NISPS, how learning works, and what to expect
- Documents all controls: touch, keyboard, and gamepad bindings
- Shows automatically on first visit (localStorage flag)
- Dismissible via close button, "Got it", overlay click, or Escape
- Reopenable via ? button at bottom right
- Map A=Train, B=Clear Examples, X=Randomize, LB=Thumbs Down, RB=Thumbs Up
- Add periodic gamepad polling fallback for environments where
gamepadconnected event doesn't fire reliably (Steam Deck, some Linux browsers)
- Show "Press any gamepad button to connect" hint on page load
When synth controls (volume, tempo, etc.) have focus after dragging,
keyboard shortcuts 1/2 for thumbs down/up were intercepted by the
range input. Now all four UIs skip shortcut handling when an input,
select, or textarea element has focus.
Relocated the output mode toggle from a floating top-right position into
the bottom floating bar where it's more discoverable. Removed the
duplicate toggle from the expanded sheet. Compact pill-toggle-sm style
fits the toolbar layout.
Merge Examples/RL tabs into single unified toolbar across all four UIs.
Users can now freely mix supervised learning (Add Example + Train) with
RL feedback (thumbs up/down) without switching modes. Param bar dragging
is always enabled.
Add Web MIDI input module (js/synth/midi-input.js) for external MIDI
controllers — routes note on/off to C15 synth, CC 1/2 to joystick.
Add gamepad module (js/ui/gamepad.js) with auto-detection, deadzone,
and axis normalization across all UIs.
Fix synth parameter sync: routeOutputs() now called after trainModel()
in onThumbsUp and loadState to prevent stale params on first joystick
move. Add separate Clear Examples button (dataset only, keeps weights).
Add ?spread=0-1 URL param that controls weight initialization scaling,
RL noise scaling per layer, noise cap, and weight decay to prevent
sigmoid output saturation. At spread=0 (original behavior) weights are
uniform [-1,1] and outputs polarise near 0/1. At spread=1 weights use
Xavier scaling (1/sqrt(fan_in)), noise is proportionally reduced, and
10% weight decay per thumbs-down prevents unbounded magnitude drift.
Also fix randomise to re-inject current joystick position and re-run
inference before routing outputs, eliminating the jump on first
joystick move after randomise.
Defaults: tame=1, spread=0.6 across all app variants.
Three redesigns of the playground UI, each exploring a different
interaction philosophy for the NISPS interactive ML engine:
A) Immersive — fullscreen canvas with floating glassmorphism overlays
B) Workbench — dashboard with 2D mapping heatmap as centerpiece
C) Journey — phased UI that dissolves (Explore → Teach → Perform)
All share the same 126-output MLP ([32,48,64] hidden layers), C15
synth bridge, and flow field visualizer. Each adds:
- SynthVisualizer (126 grouped vertical bars) replacing particles in synth mode
- RL as default learning mode with keyboard shortcuts (1/2)
- Follow mode for trackpad exploration
- ?tame= URL param for safe-range constraining
- localStorage persistence (random on first boot)
- Live iframe previews on designs.html homepage
See devlog/2026-03-21-ui-redesign-explorations.md and meml-2c3 for
remaining refinements identified during browser testing.
