- Remove Arpeggiator import and all arp references from a-app.js
- Add ShapeSeq imports (ShapeSeqEngine, StepVisualizer, ChainBuilderUI,
getDefaultBus) and lazy initialization on first audio start
- In synth mode: show ShapeSeq container (circular viz + chain builder),
hide flow-field particles. Visual mode unchanged.
- Route joystick or hand tracking inputs to shapeSeq.setSequenceInputs()
each frame; render step viz at 60fps
- Add HTML: #shapeseq-container with #shapeseq-viz canvas and
#shapeseq-chain wrapper, sequencer transport controls (play/stop,
tempo, step count)
- Add CSS: full-screen flex layout, viz takes upper 45%, chain builder
scrollable below, hidden by default
- Quick play button wires to ShapeSeq start/stop instead of arp
Closes meml-6ry. This completes ShapeSeq Phase 1.
- Change drawer hover trigger from .synth-quick-controls:hover to
.play-btn:hover ~ so hovering preset selector no longer opens it
- Add .play-drawer:hover to keep drawer open when mouse moves to it
- Set align-self: flex-start on play-btn and preset-select to prevent
vertical layout shift when drawer appears
Hand tracking via webcam as alternative to joystick input:
- Right hand tracks 14 derived features (palm XY, finger curls, spread,
roll, pitch, pinch) through a separate 14-input MLP
- Left hand gesture recognition (1 finger = thumbs up, 2 = thumbs down)
with 400ms debounce hold
- Split-zone PIP display (no camera feed, skeleton only) with dashed
divider and cross-zone dimming
- Dual IML architecture: independent imlJoy (2 inputs) and imlHand
(14 inputs) with pointer swap, preserving training data per mode
- Dev panel (?devmode=true): draggable/collapsible floating panel with
sliders for MediaPipe confidence thresholds, smoothing, gesture hold
time, world landmarks toggle, and live feature bar monitor
- Visual presets always route to joystick IML (prevents dimension mismatch)
- Race condition guard on input mode switching
4 tiers of progressive complexity (Beginner 15 params → Expert 126),
13 presets total with per-param min/max/curve overrides that bias
distributions without clamping extremes. Preset dropdown in UI,
persisted to localStorage, supports ?preset= URL param.
- Glass-style modal explains NISPS, how learning works, and what to expect
- Documents all controls: touch, keyboard, and gamepad bindings
- Shows automatically on first visit (localStorage flag)
- Dismissible via close button, "Got it", overlay click, or Escape
- Reopenable via ? button at bottom right
Relocated the output mode toggle from a floating top-right position into
the bottom floating bar where it's more discoverable. Removed the
duplicate toggle from the expanded sheet. Compact pill-toggle-sm style
fits the toolbar layout.
Merge Examples/RL tabs into single unified toolbar across all four UIs.
Users can now freely mix supervised learning (Add Example + Train) with
RL feedback (thumbs up/down) without switching modes. Param bar dragging
is always enabled.
Add Web MIDI input module (js/synth/midi-input.js) for external MIDI
controllers — routes note on/off to C15 synth, CC 1/2 to joystick.
Add gamepad module (js/ui/gamepad.js) with auto-detection, deadzone,
and axis normalization across all UIs.
Fix synth parameter sync: routeOutputs() now called after trainModel()
in onThumbsUp and loadState to prevent stale params on first joystick
move. Add separate Clear Examples button (dataset only, keeps weights).
- Fix RL buttons stuck high on desktop: media query was overriding
bottom to 196px, now matches mobile 92px
- Add floating Visual/Synth toggle at top-right of immersive view,
synced with the existing sheet toggle
Three redesigns of the playground UI, each exploring a different
interaction philosophy for the NISPS interactive ML engine:
A) Immersive — fullscreen canvas with floating glassmorphism overlays
B) Workbench — dashboard with 2D mapping heatmap as centerpiece
C) Journey — phased UI that dissolves (Explore → Teach → Perform)
All share the same 126-output MLP ([32,48,64] hidden layers), C15
synth bridge, and flow field visualizer. Each adds:
- SynthVisualizer (126 grouped vertical bars) replacing particles in synth mode
- RL as default learning mode with keyboard shortcuts (1/2)
- Follow mode for trackpad exploration
- ?tame= URL param for safe-range constraining
- localStorage persistence (random on first boot)
- Live iframe previews on designs.html homepage
See devlog/2026-03-21-ui-redesign-explorations.md and meml-2c3 for
remaining refinements identified during browser testing.
Control all sonically meaningful C15 parameters through NISPS ML outputs.
Excludes hardware routing, macros, scale/tuning, key tracking, velocity,
envelope mod depths, discrete switches, and dangerous volume/pitch params.
- Expand param-map.js with 126 curated params grouped by synthesis section
- Widen MLP architecture from [3,10,10,14,20] to [3,32,48,64,126]
- Add ParamDisplay.rebuild() for mode-dependent bar count (20 visual, 126 synth)
- Add scrollable compact layout for synth mode param bars
- Pad visual presets and old saves to 126 outputs for backward compatibility
Adds a secondary Synth mode (toggled via collapsible left side panel)
where the NISPS ML engine's 20 outputs control curated C15 synthesizer
parameters (oscillator PM, shapers, filters, reverb, echo, flanger,
output mixer) through a WASM AudioWorklet bridge.
Includes a chord progression arpeggiator with controls for tempo (BPM),
octave range, octave offset, and 4 selectable progressions. The C15
engine runs in an AudioWorklet with lock-free SharedArrayBuffer ring
buffer communication.
New files: c15/ (WASM assets), js/synth/ (bridge, arpeggiator, param map),
serve-coop.py (COOP/COEP headers for SharedArrayBuffer support).
Faithful JS port of nisps-core MLP + IML engine with a touch-friendly
UI for exploring neural parameter mapping visually. Two learning modes:
example-based (set slider targets) and RL feedback (thumbs up/down with
exploration noise). Flow field particle system controlled by 8 MLP outputs.
