No description
5a4924728f
Hand tracking via webcam as alternative to joystick input: - Right hand tracks 14 derived features (palm XY, finger curls, spread, roll, pitch, pinch) through a separate 14-input MLP - Left hand gesture recognition (1 finger = thumbs up, 2 = thumbs down) with 400ms debounce hold - Split-zone PIP display (no camera feed, skeleton only) with dashed divider and cross-zone dimming - Dual IML architecture: independent imlJoy (2 inputs) and imlHand (14 inputs) with pointer swap, preserving training data per mode - Dev panel (?devmode=true): draggable/collapsible floating panel with sliders for MediaPipe confidence thresholds, smoothing, gesture hold time, world landmarks toggle, and live feature bar monitor - Visual presets always route to joystick IML (prevents dimension mismatch) - Race condition guard on input mode switching |
||
|---|---|---|
| .beads | ||
| .vscode | ||
| data | ||
| modes | ||
| nisps-core | ||
| playground | ||
| src | ||
| voicespaces | ||
| .gitignore | ||
| .gitmodules | ||
| AGENTS.md | ||
| ChannelStripAudioApp.hpp | ||
| CLAUDE.md | ||
| IMLInterface.hpp | ||
| LICENSE | ||
| MEMLNaut-NISPS.ino | ||
| NISPS_CORE_EXTRACTION_PLAN.md | ||
| NISPS_CORE_TASKS.md | ||
| PAFSynthAudioApp.hpp | ||
| README.md | ||
| ThruAudioApp.hpp | ||
| XiasriAnalysis.cpp | ||
| XiasriAnalysis.hpp | ||
| XIASRIAudioApp.hpp | ||
Neural Interactive Shaping of Parameter Spaces
https://musicallyembodiedml.github.io/memlnaut/approaches/nisps
Web Playground
Try NISPS in your browser — no hardware required:
cd playground
python3 -m http.server
# Open http://localhost:8000
Train a neural network to map joystick positions to generative visuals through interactive machine learning. Two learning modes: direct example mapping and reinforcement learning with thumbs up/down feedback.
The playground UI includes an Expand toggle on the visual surface so you can make the canvas nearly full-screen while compressing parameter/control panels into a minimal strip beneath it.