w1n5t0n/memlnaut-nisps
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
memlnaut-nisps/playground
History
monkey-w1n5t0n 01c1346dfd fix(modular): restore matrix in paramMeta; amp floor via positive-only mod_amp 
b290144 made matrix cells opt-in to prevent joystick-silences-voice,
but that broke modular-ui.updateLive(): the matrix DOM stopped
reflecting live MLP outputs because matrixIndexCache was empty when
buildMatrixIndex() walked paramMeta. This was the same regression
6072fe8 had previously fixed.

Fix it structurally at the DSP layer instead: amp_val now computes
as `clamp(base_amp + max(0, mod_amp)) * level * vel_gain`, so matrix
d08_amp cells can only boost the amp floor — never cut it. base_amp
defaults to 1.0 (always audible), and presets that want classic
envelope-gated voices (slow pad, plucky bass, crystal, morphing
drone) drop base_amp to 0 and layer a positive ADSR→amp route on top.

With the DSP guard in place, all 480 matrix cells can safely live
in paramMeta again, and updateLive() gets its live visual feedback
back. Revert the opt-in gate in _rebuildParamMeta and the 32-param
test counts, and add a regression test asserting that every matrix
destination has 48 cells in paramMeta — that's what updateLive needs.
2026-04-11 09:39:58 +02:00
..
c15 feat(playground): add C15 synth mode with NISPS-controlled parameters and arpeggiator 2026-03-21 22:26:37 +02:00
css Merge branch 'worktree-agent-a6e16a29' 2026-04-03 18:22:42 +01:00
devlog feat(playground): add three alternative UI design explorations 2026-03-22 00:57:58 +02:00
faust fix(modular): restore matrix in paramMeta; amp floor via positive-only mod_amp 2026-04-11 09:39:58 +02:00
js fix(modular): restore matrix in paramMeta; amp floor via positive-only mod_amp 2026-04-11 09:39:58 +02:00
osc-bridge feat(vcv,playground): complete Phases 8, 9, 10 — all phases done 2026-03-28 01:48:44 +02:00
wasm feat(wasm): add batch inference, extended training, pin mask, eval loss, and layer stats bindings 2026-04-03 17:11:49 +01:00
a-immersive.html feat(playground): add ShapeSeq container to immersive layout 2026-04-07 00:01:57 +01:00
ARCHITECTURE.md docs(playground): add ARCHITECTURE.md 2026-04-06 23:17:42 +01:00
b-workbench.html feat(playground): unify learning modes, add MIDI input and gamepad support 2026-03-22 11:01:50 +02:00
c-journey.html feat(playground): unify learning modes, add MIDI input and gamepad support 2026-03-22 11:01:50 +02:00
designs.html style(playground): unify dark theme with orange accents and improve RL button layout 2026-03-22 01:36:00 +02:00
index.html tweak(playground): simplify descriptions in card body text 2026-03-22 02:22:47 +02:00
PLAN-solidjs-migration.md Improve solidjs migration plan 2026-04-07 03:44:36 +01:00
README.md feat(playground): implement Phase 1 control surface — compound axes, input pipeline, enhanced joy-map 2026-03-26 10:24:24 +02:00
serve-coop.py feat(playground): add C15 synth mode with NISPS-controlled parameters and arpeggiator 2026-03-21 22:26:37 +02:00
serve.sh feat(playground): add C15 synth mode with NISPS-controlled parameters and arpeggiator 2026-03-21 22:26:37 +02:00
SPEC-controls.md feat(playground): implement Phases 2-4 of control surface spec 2026-03-26 10:48:12 +02:00
SPEC-shapeseq.md docs(playground): revise ShapeSeq spec after fresh-eyes review 2026-03-24 00:30:43 +02:00
test-midi-receiver.py feat(playground): MIDI CC device presets + bug fixes 2026-04-02 20:35:38 +01:00
TODOS.md hello come back to me 2026-03-30 17:05:45 +03:00

README.md

NISPS Playground

Browser-based interactive demo of the NISPS ML engine.

Run locally

python3 -m http.server
# Open http://localhost:8000

Run the command from the playground/ directory.

What it does

  • Maps 2D joystick input (X/Y) through an MLP ([3, 32, 48, 64, 126]) to 126 outputs.
  • Visual mode: first 20 outputs drive a flow-field particle visualization on Canvas2D.
  • Synth mode (C15): all 126 outputs control the C15 WASM synthesizer — envelopes, oscillators, shapers, filters, feedback/output mixers, cabinet, and effects.
  • Supports two learning modes:
    • Examples: add explicit input/output pairs and train.
    • RL Feedback: give thumbs up/down while exploring outputs.

C15 synth parameters

The 126 synth parameters (js/synth/param-map.js) cover all sonically meaningful continuous parameters of the C15 engine. Excluded from the C15's 287 total params:

  • Hardware routing (56) — no physical MIDI in browser
  • Macro controls (12) — conflicts with direct ML control
  • Scale/tuning (13) — would break pitch
  • Key tracking / velocity (22) — depend on note context ML can't observe
  • Envelope mod depths (19) — too many multiplicative interactions for 2-input ML
  • Discrete/structural/dangerous (19) — pitch sweep, volume, switches, resets
  • Secondary config (15) — curves, chirp, shaper blend, source selects

UI controls

  • Output mode tabs (floating bar): switch between Visual and Synth modes.
  • Expand / Collapse chevron: expands bottom sheet with training controls, synth settings, advanced param sliders.
  • Synth presets: tiered presets (Beginner → Expert) control which of the 126 params the ML engine can modify.
  • Help (?) overlay: in-app usage guide.
  • Follow mode: double-click joystick in RL mode to toggle no-hold interaction.
  • Keyboard in Follow mode (RL): 2 = thumbs up, 1 = thumbs down.
  • Gamepad in RL: RB = thumbs up, LB = thumbs down, A = train, X = randomize, B = clear.

Control Surface

The immersive app has a control surface system (Phase 1 of SPEC-controls.md) for tuning how exploration and learning feel.

Compound axes on the floating bar (3 sliders: Bold, Mem, Prec):

Axis Controls Low end High end
Boldness Input zoom, noise cap, noise growth, LR, weight decay Cautious: small changes, heavy regularisation Bold: full range, explosive exploration, fast learning
Memory Max examples, example decay, noise decay, convergence Amnesia: only last few interactions matter Elephant: every example sacred, stable mapping
Precision Input curve, deadzone, smoothing, slew rate, momentum Raw: 1:1 with physical movement, twitchy Precise: heavy shaping, deadzones, smooth

Settings drawer (gear icon, bottom-right): individual param overrides for Input, Training, Exploration, and Output sections. Manual overrides persist as offsets when compound axes move (trim-pot model). Double-tap an axis to re-link all params.

Control presets: Default, First Touch, Jazz Hands, Sculptor, Improviser, Microscope.

Input pipeline: joystick input is processed through deadzone → zoom → curve → smoothing → momentum-as-zoom before reaching the MLP. Zoom narrows the effective input window around an anchor. Zoom-at-zero freezes input.

Enhanced joy-map: zoom minimap with adaptive grid (4×4 → 32×32), vanishing trail with tap-to-return, dual noise rings (zoom + noise level).

Files

Original app (index.html)

  • index.html - page structure.
  • css/style.css - layout and visual styling.
  • js/app.js - app wiring and interaction logic.

Immersive app (a-immersive.html)

  • a-immersive.html - fullscreen immersive UI.
  • css/a-immersive.css - immersive layout and styling.
  • js/a-app.js - immersive app wiring, state management, persistence.

Shared modules

  • js/nisps/ - JavaScript MLP + IML core (also WASM variant).
  • js/ui/visualizer.js - flow-field particle system (Canvas2D).
  • js/ui/joystick.js - virtual joystick component.
  • js/ui/gamepad.js - gamepad input handling.
  • js/ui/hand-tracker.js - MediaPipe hand tracking (14 features).
  • js/ui/input-pipeline.js - input processing pipeline (zoom, deadzone, curve, smoothing, momentum).
  • js/ui/control-surface.js - compound axes, override resolution, control presets.
  • js/ui/control-surface-ui.js - settings drawer and floating bar axis sliders.
  • js/ui/joy-map-enhanced.js - zoom minimap, vanishing trail, dual noise rings.
  • js/synth/ - C15 WASM bridge, parameter map, presets, arpeggiator.
  • c15/ - C15 engine WASM binary and parameter definitions.
  • SPEC-controls.md - comprehensive control surface spec (4 phases).