- Wire EOCChain into audio graph after c15.start() (both start-btn and
quick-play paths); guarded by _eocInited flag so init happens once per
AudioContext lifetime
- getOutputNode() on C15Adapter now returns limiter (last node before
destination) so EOC inserts correctly between limiter and destination
- Expose limiterNode getter on C15Bridge
- Add getCurrentParamValue(i) to EOCModule base class, backed by _paramValues
array that setParam() writes to
- Add _buildParamSliders() to EOCChainUI: collapsible ▶ params toggle per
module row, one range slider per paramMeta entry, calls module.setParam()
on input; labelled "manual" when nispsMode=bypass
- eoc:change listener in a-app.js logs paramCount (hook point for future modes)
- saveState()/loadState() persist eocModules (id, enabled, params) and
eocNispsMode; restore re-adds modules via moduleFactory and re-applies values
- CSS: eoc-params-section, eoc-params-toggle, eoc-param-row/label/slider;
eoc-module-row gains flex-wrap to accommodate params section below controls
Adds EngineSwitcher module with clickable engine cards (active highlight,
coming-soon greyed state, loading bar animation, confirm-on-switch when
training data exists). Wired into the Engine drawer; C15 active,
Additive/FM marked coming-soon with toast. Dock title reflects active
engine; engineId persisted to/restored from localStorage.
- input-pipeline.js: guard circular clamp against div-by-zero when
input is exactly at center (0.5, 0.5) — dist=0 produced NaN
- snapshot-stack.js: jumpTo() used slice(0, index) which excluded the
target snapshot; fixed to slice(0, index + 1)
- a-app.js: _lastSentParams not resized when MLP output count changes
during mode switch, causing stale throttle state and potential param
flood on first frame after switch
Adds the Faust DSP toolchain infrastructure: placeholder additive and FM DSP
files, build.sh (faust -lang wasm per .dsp), faustJsonToParamMeta() to convert
Faust JSON UI trees into the standard paramMeta format, FaustEngineBase
(SynthEngine subclass wiring init/setParam/noteOn/noteOff through AudioWorklet
messages), and FaustWorkletProcessor base class for concrete engine processors.
Add WasmIML.extractWeights() and WasmIML.createWithWarmStart() to preserve
learned joystick mappings across output-count changes; resizeMLP() now
transfers hidden-layer weights and shared output nodes instead of cold-starting.
InputHeatmap.update() now accepts options.inferBatchFn to evaluate
all grid points (plus the divergence center point) in a single WASM
call instead of 256 separate round-trips. The per-point inferFn path
is preserved as a fallback when inferBatchFn is not provided.
Add evalLoss, inferBatch, and getLayerStats to the window.__nisps
debug probe so Playwright tests and dev console can access the new
WasmIML capabilities. InputHeatmap (Phase 3) is not yet wired into
a-app.js, so batch inference heatmap integration is deferred.
Expose inferBatch, trainEx, moveWeightsEx, evalLoss, and getLayerStats
from the WASM binary into the JavaScript layer:
- inferBatch: batch inference for heatmap sampling
- trainEx: replaces train() with per-iteration loss history capture
- moveWeightsEx: native output pin mask support (removes save/restore hack)
- evalLoss: compute loss without updating weights
- getLayerStats: per-layer weight statistics (meanAbs, maxAbs, dead/sat fracs)
Worker also upgraded to trainEx, returning full lossHistory in payload.
Five new C functions for the WASM module:
- nisps_mlp_infer_batch: N-point batch inference in a single call
- nisps_mlp_train_ex: training with per-iteration loss history output
- nisps_mlp_move_weights_ex: moveWeights with output pin mask to skip pinned nodes
- nisps_mlp_eval_loss: compute MSE loss without updating weights
- nisps_mlp_get_layer_stats: per-layer weight magnitude, dead, and saturation stats
- Add window.__nisps debug probe (gated on ?debug=1) exposing iml state,
getOutputs/getLoss/getWeights/getExampleCount, and action triggers
(thumbsUp/thumbsDown/train/randomise/clearExamples/saveState)
- Fix WasmIML bug: this.dataset was a plain object; import Dataset and
use new Dataset(100) so computeWeights() is available for training
- Fix WasmIML.addExample/clearDataset to use Dataset API methods
- 44 Playwright e2e tests across 4 spec files:
- ml-engine.spec.js: WASM inference bounds, training loss, thumbs
up/down behavior, async training, example capture semantics
- ui-interactions.spec.js: drawer open/close, mode switching,
heatmap bar counts, preset chips, keyboard shortcuts (1/2/Z)
- input-pipeline.spec.js: input→output variation, clamping, joystick
drag, post-training output bounds across the full input space
- persistence.spec.js: URL params (?preset, ?spread), localStorage
round-trip, saveState probe
Add MIDI CC preset system for bundled device configurations:
- midi-cc-presets.js: listPresets/loadPreset/loadPresetFromFile API
- presets/polybrute.json: Arturia PolyBrute CC map
- Preset selector in quick controls bar and MIDI CC drawer
- File import button for loading JSON presets from disk
Bug fixes in a-app.js:
- Destroy IML instances before resizeMLP() to free WASM memory
- Fix randomiseWeights() call (was drawWeights())
- Fix thumbs-up to store rawParamValues instead of post-pipeline outputs
Add Dataset.computeWeights() with three modes:
- global: exponential recency decay (newest examples weighted higher)
- local: spatial suppression of older examples near the current input
- combined: both applied together
IML and WasmIML now compute weights on every train() call using the
active mode. Exposes recencyBias, weightingMode, localRadius properties.
WASM worker path passes sampleWeights through to C++ via the new binding.
Add optional sample_weights parameter to MLP::Train() and the WASM
nisps_mlp_train binding. When provided, weights replace the uniform
1/N scaling per sample — enabling recency, spatial, or any custom
importance weighting without changing the training interface.
- OSC pill: add status dot indicator (gray=off, green=connected,
pulsing yellow=connecting), use flexbox for reliable alignment
- OSC pill: add hover state for better affordance
- Synth panel: add separator line above OSC row so it doesn't
blend into the arpeggiator controls
- Bottom sheet: increase expanded height from 55vh to 60vh so
OSC row isn't cut off on mobile
- Help modal: update step 3 to mention the OSC pill in the
bottom bar as the primary connection method
Browser-side OSCOutput module sends parameter values over WebSocket to
a companion Deno bridge script that converts them to OSC/UDP messages.
Enables controlling SuperCollider, Max/MSP, Pure Data, TouchDesigner,
or any OSC-capable software from the NISPS playground.
- Browser module (js/synth/osc-output.js): WebSocket client with
auto-reconnect, throttle (~20fps), and dead-zone filtering
- Deno bridge (osc-bridge/bridge.ts): zero-dependency, compiles to
standalone binaries via deno compile for Linux/macOS/Windows
- Test receiver (osc-bridge/test-receive.ts): terminal dashboard
showing live OSC parameter values with bar charts
- OSC pill button in floating bar for quick connect/disconnect
- Help modal section with platform-aware download, setup guide,
and examples for SuperCollider/PD/Max
- GitHub Actions workflow for cross-platform binary builds
- OSC sends in both visual and synth modes
Design spec for input zoom, compound control axes, pinning system,
exploration noise controls, and visualization enhancements. Covers
the full parameter inventory with implementation phases and open
design questions for experimentation.
Re-apply fixes from ddb6b77 that were lost in the arpeggiator restore:
- play-btn: align-self: flex-start (pin to top)
- Play drawer: trigger on .play-btn:hover instead of parent :hover
(prevents preset select from opening the drawer)
- preset-select: align-self: flex-start (pin to top)
Restore the original arpeggiator as the default sequencer. ShapeSeq
code is preserved but gated behind a URL feature flag (?shapeseq=1):
- Arpeggiator import, state, and all wiring fully restored
- ShapeSeq modules loaded via dynamic import() only when flag is on
- ensureShapeSeqInit() creates engine/viz/UI lazily on first audio start
- animate loop routes inputs to ShapeSeq only when flag + playing
- setOutputMode shows/hides ShapeSeq container only when flag is on
- HTML: ShapeSeq container added but hidden by default
- CSS: ShapeSeq styles added (inert when container is hidden)
All 13 ShapeSeq modules in playground/js/shapeseq/ are untouched.
To test ShapeSeq: add ?shapeseq=1 to the URL.
- Remove Arpeggiator import and all arp references from a-app.js
- Add ShapeSeq imports (ShapeSeqEngine, StepVisualizer, ChainBuilderUI,
getDefaultBus) and lazy initialization on first audio start
- In synth mode: show ShapeSeq container (circular viz + chain builder),
hide flow-field particles. Visual mode unchanged.
- Route joystick or hand tracking inputs to shapeSeq.setSequenceInputs()
each frame; render step viz at 60fps
- Add HTML: #shapeseq-container with #shapeseq-viz canvas and
#shapeseq-chain wrapper, sequencer transport controls (play/stop,
tempo, step count)
- Add CSS: full-screen flex layout, viz takes upper 45%, chain builder
scrollable below, hidden by default
- Quick play button wires to ShapeSeq start/stop instead of arp
Closes meml-6ry. This completes ShapeSeq Phase 1.
Four modules that build on the Layer 0 foundation:
- primitive.js: base class with category system (generator/processor/
timing/converter), param schema validation, boundary enforcement
(clamp/wrap/scaled), symbolic process() interface, state management
for freeze, and applyBoundary() utility for delta control
- clock.js: AudioContext lookahead scheduling (25ms interval, 100ms
window) replacing setTimeout arpeggiator. Reads pattern descriptions
for per-step swing (timeOffset) and ratchet (subdivisions). Emits
seq.noteOn/noteOff/step/loopStart via event bus.
- projection.js: composable transform chain with 5 transforms
(VelocityCurve, GateThreshold, RangeMap, OctaveFolder, StutterMap)
and 3 presets (expressive, percussive, fullRange). Pitch quantization
deliberately excluded (handled by Interval Lock primitive).
- step-viz.js: Canvas2D circular step visualizer with even angular
spacing for any step count. Pitch→radius, velocity→node size,
accent→color. 60fps-friendly with pre-allocated coordinate buffers.
Tap interaction for step toggling.
Five independent modules with no interdependencies:
- event-bus.js: namespaced pub/sub (seq.*/ml.*/ui.*) with wildcard
subscriptions and automatic AudioContext/performance.now timestamps
- prng.js: seedable mulberry32 PRNG with pure-functional API, fork()
for independent per-primitive streams, serializable state for freeze
- pattern.js: symbolic pattern description data structure with
create/clone/merge/validate, additive and multiplicative merge modes
- param-map.js: maps fixed 16-output MLP to variable-count primitive
params via linear interpolation, with optional per-param min/max scaling
- seq-iml.js: factory for second WasmIML instance (2 inputs, 16 outputs,
[16,16,16] hidden layers) following existing imlJoy/imlHand pattern
All modules are ES modules with no build step. PRNG, pattern, and
param-map are port-ready (explicit state, typed arrays, no closures).
- Change drawer hover trigger from .synth-quick-controls:hover to
.play-btn:hover ~ so hovering preset selector no longer opens it
- Add .play-drawer:hover to keep drawer open when mouse moves to it
- Set align-self: flex-start on play-btn and preset-select to prevent
vertical layout shift when drawer appears
Hand tracking at 30fps was writing 126 params per frame to the C15 ring
buffer (capacity 512), flooding it and starving arpeggiator noteOn/noteOff
messages — causing stuck/dropped notes.
Three-part fix:
- Arpeggiator now runs in a dedicated Web Worker with direct
SharedArrayBuffer access, bypassing the main thread entirely for
note timing. Worker setInterval isn't subject to main thread jank.
- RingBufferWriter uses CAS (Atomics.compareExchange) for multi-producer
safety — both main thread (params) and worker (notes) write safely.
- routeOutputs throttles synth parameter sends: dead-zone filter
(0.2% change threshold) + rate cap (~20fps), reducing ring buffer
pressure from ~3800 msg/s to ~50-100 msg/s of actual changes.
Hand tracking via webcam as alternative to joystick input:
- Right hand tracks 14 derived features (palm XY, finger curls, spread,
roll, pitch, pinch) through a separate 14-input MLP
- Left hand gesture recognition (1 finger = thumbs up, 2 = thumbs down)
with 400ms debounce hold
- Split-zone PIP display (no camera feed, skeleton only) with dashed
divider and cross-zone dimming
- Dual IML architecture: independent imlJoy (2 inputs) and imlHand
(14 inputs) with pointer swap, preserving training data per mode
- Dev panel (?devmode=true): draggable/collapsible floating panel with
sliders for MediaPipe confidence thresholds, smoothing, gesture hold
time, world landmarks toggle, and live feature bar monitor
- Visual presets always route to joystick IML (prevents dimension mismatch)
- Race condition guard on input mode switching
Compile nisps-core C++ MLP to WASM (36KB) and use it as the ML engine
in the playground, replacing the JavaScript port for inference, training,
and weight manipulation.
- Add extern "C" WASM bindings with spread-aware drawWeights/moveWeights
- WasmIML class is a drop-in replacement for the JS IML
- Inference runs on main thread via WASM (fast, synchronous)
- Training runs in a Web Worker with its own WASM instance (non-blocking)
- Interactive training (thumbs-up, train button) no longer freezes UI/audio
- Preset loading and state restore still use sync training
4 tiers of progressive complexity (Beginner 15 params → Expert 126),
13 presets total with per-param min/max/curve overrides that bias
distributions without clamping extremes. Preset dropdown in UI,
persisted to localStorage, supports ?preset= URL param.
SVF count was 7 (actual 9), Gap Filter section was missing (6 params),
FB Mix count was 10 (actual 9) — causing cascading mislabeling of all
synth sections from SVF onwards. Also sync volume/tempo changes from
bottom sheet back to quick-play controls.
- Glass-style modal explains NISPS, how learning works, and what to expect
- Documents all controls: touch, keyboard, and gamepad bindings
- Shows automatically on first visit (localStorage flag)
- Dismissible via close button, "Got it", overlay click, or Escape
- Reopenable via ? button at bottom right
- Map A=Train, B=Clear Examples, X=Randomize, LB=Thumbs Down, RB=Thumbs Up
- Add periodic gamepad polling fallback for environments where
gamepadconnected event doesn't fire reliably (Steam Deck, some Linux browsers)
- Show "Press any gamepad button to connect" hint on page load
When synth controls (volume, tempo, etc.) have focus after dragging,
keyboard shortcuts 1/2 for thumbs down/up were intercepted by the
range input. Now all four UIs skip shortcut handling when an input,
select, or textarea element has focus.
Relocated the output mode toggle from a floating top-right position into
the bottom floating bar where it's more discoverable. Removed the
duplicate toggle from the expanded sheet. Compact pill-toggle-sm style
fits the toolbar layout.
