No description
9e59eb04ce
Wire the modular input layer into the Console and reshape the browser engine so input axes are genuine independent dimensions. Inputs (manifold/src/inputs/): - gamepad-source: emit press+release edges with standard-mapping labels (enables hold-and-move); single/double-stick already present. - midi-input-source: single-device selection + batch "MIDI Learn" (every CC swept while armed becomes an axis); notes stay discrete. - input-layer: compose() forwards each axis 1:1 (no mean-blend); add onReducedInput so the manifold tracks gamepad/MIDI position. - types: InputAction.phase, InputMode. Console (manifold/src/console/): - ConsoleApp: bind gamepad buttons to verdicts (RB up / LB down / X randomise / Y nudge / B undo / A-hold reposition); mirror composed position onto the manifold. - Drawers: rebuilt Inputs drawer (source picker, gamepad legend, MIDI device picker + batch-learn flow, learned-control meters). Engine (nisps/wasm, manifold/src/engine): - DefaultMLP widened MLP<2,..> -> MLP<32,..> (32 = MAX_AXES); each active axis gets a dedicated slot, unused slots held at 0 (inert). Rebuilt nisps.wasm (playground + manifold). - spine/engine-api: setInputs writes the full N-D vector (was dropping arr[2+]); primary pair keeps the 2-D pipeline; process() re-ticks the whole vector via spine.reprocess(). Tests: - parity_check/parity_wasm: ParityMLP -> 32 inputs, widen example bufs. - CMakeLists: build parity binary with -ffp-contract=off so native matches FMA-free WASM (training amplified the gap past 1e-5). Inputs dock is still an exclusive picker; mixing toggles, reshape modal, and the >2-D slider view (inputs-spec.md) are groundwork-laid but not yet wired. See docs/redesign/midi-gamepad-inputs-worklog.md. |
||
|---|---|---|
| .claude/worktrees/agent-ae87fe47/playground/js/shapeseq | ||
| .github/workflows | ||
| .vscode | ||
| assets/media | ||
| codegen | ||
| data | ||
| docs | ||
| firmware | ||
| manifold | ||
| nisps | ||
| playground | ||
| schemas | ||
| scripts | ||
| src | ||
| tests | ||
| vcv | ||
| .envrc | ||
| .gitignore | ||
| .gitmodules | ||
| AGENTS.md | ||
| ALIGNMENT.md | ||
| CLAUDE.md | ||
| LICENSE | ||
| MAP.md | ||
| NISPS_CORE_EXTRACTION_PLAN.md | ||
| NISPS_CORE_TASKS.md | ||
| package-lock.json | ||
| package.json | ||
| playwright.config.js | ||
| README.md | ||
| synth-midi-cc.json | ||
Neural Interactive Shaping of Parameter Spaces
https://musicallyembodiedml.github.io/memlnaut/approaches/nisps
Firmware
The hardware firmware targets the MEMLNaut RP2350 build and uses repo-local helper scripts for the known-good build configuration:
git submodule update --init --recursive
scripts/build-firmware.sh
scripts/flash-firmware.sh
scripts/build-and-flash-firmware.sh
Notes:
- The scripts build for
rp2040:rp2040:solderparty_rp2350_stamp_xlwithOptimize3. - The build forces C++20 because the firmware uses
std::spanand concepts. build-firmware.shaccepts an optional variant name such asMEMLCeliumorBreakOr. Matching remains case-insensitive, somemlceliumstill works. If you omit it in an interactive shell, the script parsesMEMLNaut-NISPS.ino, prompts for a variant, and rewrites the activeMEMLNAUT_MODE_TYPEbefore building.flash-firmware.shaccepts an optional mountpoint argument, or auto-detects common UF2 bootloader mounts such as/run/media/$USER/RP2350and/run/media/$USER/RPI-RP2.
Web Playground
Try NISPS in your browser — no hardware required:
cd playground
python3 -m http.server
# Open http://localhost:8000
Train a neural network to map joystick positions to generative visuals through interactive machine learning. Two learning modes: direct example mapping and reinforcement learning with thumbs up/down feedback.
The playground UI includes an Expand toggle on the visual surface so you can make the canvas nearly full-screen while compressing parameter/control panels into a minimal strip beneath it.