Operator-approved ABI change (P2 stop-point). The WASM MLP is now
MLPCore<DynamicStorage>:
- nisps_ml_create(input, output, hidden[3], n, seed) honours its args;
non-positive/null fall back to the historical 32→[10,14,18]→126, so
every pre-P2 caller (manifold, worker, parity harness) stays
bit-identical. Invalid/oversized dims (>4096) → null.
- NEW nisps_ml_reshape(ml, in, out, hidden, n, spread): fresh net at the
new dims, warm-started via nisps/ml/warm_start.hpp (overlapping region
copied; rest keeps spread init); feedback controller re-created (state
resets — reset-on-reshape modal is the front-end contract). Failure
leaves the old net untouched.
- nisps_ml_describe(ml, out): takes the handle; null reports defaults.
- FeedbackController got the same storage split: algorithms in
FeedbackControllerCore<FbStorage>; FixedFeedbackStorage keeps firmware/
tests source-identical via the old alias; DynamicFeedbackStorage (one
arena) sizes to the runtime net. Firmware .text unchanged (122692).
- MLHandle: per-instance scratch vectors; dropped the dead 2MB
batch_out_scratch.
- TS: types.ts decls (+_nisps_ml_reshape), wasm-iml re-describes the
created instance, worker carries a shape-contract note for P2.3.
Verified: ctest 4/4 incl. new warm-start grow/shrink test; reshape ABI
smoke (dims honoured, overlap survives, invalid rejected, outputs
bounded); parity PASS unchanged (2.4e-7); lint clean; manifold 9 unit +
20 e2e green; firmware .text 122692 (+0.30% vs pre-P2 baseline).
Algorithms (forward, backprop/SGD, init, move_weights, diagnostics) now live
once in MLPCore<Storage> (nisps/ml/mlp.hpp). Storage models:
- FixedStorage (storage.hpp): template-sized std::array, zero heap. The
classic MLP<NIn,H1,H2,H3,NOut,...> is an alias preserving kInput/kHidden*/
kOutput/kNumLayers/weight_count() constexpr — firmware + bindings + modes
compile unchanged.
- DynamicStorage (dynamic_storage.hpp): runtime dims, ONE arena allocation
at construction, nothing per-call. #error under NISPS_TARGET_EMBEDDED
(new macro in core/perf.hpp); sole lint-cpp.sh heap-allowlist entry, plus
a lint check that fails if the #error guard disappears.
Verification:
- new ctest test_mlp_storage_parity: fixed↔dynamic BIT-identical across
init/draw/inference/train(FIFO)/move_weights(pin mask)/eval_loss/
layer_stats/set_weights/infer_batch/reset; invalid+moved-from inert
- golden ML vectors (pre-refactor constants) pass → bit-stable refactor
- native↔WASM parity PASS, max delta unchanged (2.4e-7)
- chokepoint B compile: PAFSynth .text 122324→122692 (+0.30%, ±1% budget);
RAM +416B (eval scratch)
- fix: firmware-common.sh used bare 'python' (absent here) → ${PYTHON:-python3}
Part of one-core-engine-refactor P2. nisps_ml_create ABI untouched (P2.2 is
an operator stop-point).
Bring the two playground-only exploration UIs into manifold ahead of the
playground's retirement (one-core-engine refactor §P1):
- Jolt: press-and-hold continuous weight-morph, release to freeze.
- Explore: Ornstein-Uhlenbeck exploration intensity on the output vector.
Interim TS maths ported verbatim from the retired playground modules
(engine/jolt.ts, engine/ou-explore.ts). The ExplorationController
(engine/exploration.ts) is the single P3 swap boundary: it drives Jolt via the
existing EngineApi get/set-weights + process route, and OU via a new inert-by-
default output-morph hook on the spine. In §P3 only that module changes to call
nisps_ml_jolt_press/release + nisps_ml_explore_intensity.
UI lands in the Learning drawer (Jolt hold-button + Explore slider), monochrome-
consistent, British copy. Gates green: typecheck, build, Playwright smoke.
Migrate the specs worth keeping from playground/tests/e2e (retired in P1) into
manifold/tests/e2e, adapted to Manifold's probe surface:
- probe-api.spec.ts: the window.__nisps debug-probe contract (ready, bounded
outputs, example count, randomise, setInputs inference, thumbsUp/Down,
addExample, train loss non-increasing, async train, clearExamples, evalLoss,
inferBatch, getLayerStats, getWeights). Retargeted to MLP<32,10,14,18,126>
(weight_count 3148) and Manifold's direct addExample/routedOutputs surface;
dropped the playground's __init/iml-poke escape hatches and stream-pending
skips.
- spine.spec.ts: the spine invariant — setInputs -> processed -> ml -> routed
yields bounded, consistent routed outputs; the probe stays alive across dock
output-mode switches (driven via the real selector UI, replacing the
playground's localStorage-reload mode cycling).
- helpers.ts: loadProbe(?debug=1 + cleared storage + __ready wait), settleInputs
for EMA convergence, bounded/changed assertions.
Dropped playground UI specs (ui-interactions, persistence, mode-registry list)
that die with the playground chrome. No probe.ts changes needed.
Records canonical gesture trace, curve catalog samples, and input/output
pipeline outputs from the current TS implementations, plus a bun-test drift
guard that re-runs them against the fixtures within 1e-9. Serves the P4
one-core-engine gate: same pointer trace -> same routed output pre/post the
C++/WASM migration.
- Double-click the input mark to enter follow-mouse mode: a window-level
pointermove listener maps the whole viewport onto the surface's [0,1]²
space so the knob tracks the cursor across the entire UI. Esc or a second
double-click exits; a badge + hidden cursor signal the active state. Local
pan/long-press and auto-drift are suppressed while following.
- Keys 1 = thumbs-down/explore (perturb), 2 = thumbs-up (commit), on the
window keydown handler so they fire even in follow-mouse mode. Drawers
learn/inputs lose their 1/2 shortcuts (3-5 keep route/settings/help).
- Sync help keymap (Drawers) + ONBOARDING stages table.
Geometric-dislike ('Push away', Mode 1) is now the default feedback mode
instead of explore-and-place, which works poorly. Add a Clear button to
the Learning drawer that forgets all recorded examples and wipes the
on-map visuals (feedback markers + placed-anchor pins) via the existing
ctx.onClear, now also resetting pins.
The on-screen circular input disc let both the mouse and the gamepad drive
the knob outside the visible circle.
- Gamepad: each stick axis was clamped to [0,1] independently, so a full
diagonal push reached the square corner. Clamp the stick *vector* to the
unit disc (radially symmetric) before mapping to [0,1]; covers both sticks.
- Mouse: the circular variant clamped in normalised [0,1]^2 but the canvas
drew the knob across the full non-square panel against the inscribed circle,
so the disc rendered as an ellipse that spilled past the rim. Map both the
pointer and every drawn position (knob/pins/markers/trail/flash) through the
inscribed-circle geometry, and disc-clamp the auto-drift.
The painter's-algorithm sort was descending, but z2 (depth) increases
toward the camera in both the top-down and tilted-from-above views, so
nearest was painted first (underneath). This showed the last layer on
top in the default top-down view and made the stack occlude inside-out
when tilted back. Sort ascending: farthest first, nearest last.
Restores the April-2026 "uSEQ-Celium" functionality (browser → uSEQ
hardware + CV expander over USB Web Serial) as a first-class Manifold
Outputs backend, and re-vendors the RP2040 firmware into the repo.
- protocol v2 (uSEQ-CV): firmware/useq-celium/shared/protocol.h is the
single source of truth, mirrored by manifold/src/backends/useq-protocol.ts.
26-byte OUTPUT frame, 11×u16 CV (12-bit) + 3-gate bitfield + XOR; fixed
topology; host-agnostic so the MEMLNaut RP2350 can emit identical bytes.
Spec in docs/useq-celium/protocol.md.
- firmware/useq-celium/{main,expander}: PlatformIO RP2040 firmware rewritten
to v2 from the real April pin maps (expander I2C addr 0x10).
- UseqCvBackend (id cvgate): Web Serial connect/identify/disconnect, 100 Hz
stream, per-channel dead-zone, gate thresholding; modeled on midi-backend.
Per-output CvSpec (channel + gateThreshold) on MFParam; config UI in
OutputsBackendConfig + BackendAdvanced; new "CV / uSEQ" top-dock mode.
- bun-test for the protocol frame layout; MAP.md updated.
The OutputEditor popover was placed with a fixed top + an index-based
left/right heuristic and never measured the viewport, so it clipped the
screen edges when the window wasn't full-screen. It now measures its
anchored rect on layout and translates itself back inside the viewport
(8px gutter), re-checking on resize.
Single-file map of the Manifold front-end so an agent can make a tweak/fix
without re-grepping the tree or reading the 40KB design specs: run/build/
deploy/test commands, the three-layer architecture (UI / engine spine / WASM),
the convertible Stage system, the Dock + drawers, engine/inputs/feedback/
backends layout, and the non-obvious gotchas (imperative output reads,
baseURI asset URLs, hardwired WASM arch, curves↔C++ lockstep, no 'C15').
- mf-slider-input fill used calc(var(--mf-pct)*100%) but three consumers
(OutputControlRow held slider, OutputEditor, shared-ui Field) never set
--mf-pct, so the coloured track-fill stayed empty while only the native
thumb moved. Set --mf-pct inline on all three; harden the base class
(appearance:none + transparent bg + --mf-pct fallback + moz-range-progress).
- hide the two concentric noise-cap exploration rings around the input dot.
Fuse the /learn parameter-landscape sandbox into the main console:
- New SandwichStage: ports the 3D landscape renderer to React, driven by the
REAL EngineApi (inferBatch samples each output channel over the 2D input grid;
getOutputs feeds the live probe). Re-samples on engine version bumps; orbit
camera (top-down = flat 2D, tilt to reveal the stack), centred + grows to fill.
- Dock: new bottom toggle (SandwichIcon) → ctx.sandwich.
- ConsoleApp: when on, the sandwich replaces the centre stage in a 3-zone flex —
shrunken Manifold (left) · sandwich (centre, fills) · compact OutputStage
(right) — reusing pos/markers/verdicts. Opening it auto-closes the drawer so
the output zone is visible. Layers = the mode's params (capped 8).
typecheck + build green; verified headless (toggle on/off, sample, tilt).
The default (Particle) view no longer renders the 'MEMLNaut' wordmark
since the particle top bar became a heatmap strip; assert on the always-
present 'Learning' dock drawer label instead so the smoke test reflects
the current Console chrome.
Wire the modular input layer into the Console and reshape the browser
engine so input axes are genuine independent dimensions.
Inputs (manifold/src/inputs/):
- gamepad-source: emit press+release edges with standard-mapping labels
(enables hold-and-move); single/double-stick already present.
- midi-input-source: single-device selection + batch "MIDI Learn"
(every CC swept while armed becomes an axis); notes stay discrete.
- input-layer: compose() forwards each axis 1:1 (no mean-blend);
add onReducedInput so the manifold tracks gamepad/MIDI position.
- types: InputAction.phase, InputMode.
Console (manifold/src/console/):
- ConsoleApp: bind gamepad buttons to verdicts (RB up / LB down /
X randomise / Y nudge / B undo / A-hold reposition); mirror composed
position onto the manifold.
- Drawers: rebuilt Inputs drawer (source picker, gamepad legend, MIDI
device picker + batch-learn flow, learned-control meters).
Engine (nisps/wasm, manifold/src/engine):
- DefaultMLP widened MLP<2,..> -> MLP<32,..> (32 = MAX_AXES); each
active axis gets a dedicated slot, unused slots held at 0 (inert).
Rebuilt nisps.wasm (playground + manifold).
- spine/engine-api: setInputs writes the full N-D vector (was dropping
arr[2+]); primary pair keeps the 2-D pipeline; process() re-ticks the
whole vector via spine.reprocess().
Tests:
- parity_check/parity_wasm: ParityMLP -> 32 inputs, widen example bufs.
- CMakeLists: build parity binary with -ffp-contract=off so native
matches FMA-free WASM (training amplified the gap past 1e-5).
Inputs dock is still an exclusive picker; mixing toggles, reshape modal,
and the >2-D slider view (inputs-spec.md) are groundwork-laid but not
yet wired. See docs/redesign/midi-gamepad-inputs-worklog.md.
Replace the a-immersive original's clashing 20-colour palette (hot magenta,
lime, pink) with a ramp interpolated between Manifold's two brand accents —
--accent (#ff6a00) → --accent-2 (#00ccff) — softened to the app's semantic-
token saturation register so the strip reads as native chrome.
ParticleStage showed the Boldness/Memory/Precision macro sliders across the
top, but the a-immersive design this mode clones uses a thin per-output
heatmap strip (one colored bar per visual output, width = live value). The
deployed a-immersive never used the macro-axis surface.
- flow-field.ts: export VISUAL_PARAM_NAMES/COLORS + N_VISUAL_OUTPUTS (verbatim
from a-app.js:46/51)
- ParticleStage: replace macro-slider bar with the 22px heatmap strip; bar
widths updated imperatively in the existing rAF loop; hover tooltip with
live value; drop unused axes props
- ConsoleApp: update call site
Adds a device picker to the MIDI backend config: choose an external synth
(Moog Sub 37, Hydrasynth, etc.), see its parameters grouped and BY NAME with
their CC numbers, tick which ones to control, and Apply — fills the per-output
CC table (CC#/channel/name) and sets the CC count from the template. Clamps to
the engine output count. Self-contained (local state + setParam/setMidiCcCount);
the result lives in the shared params store, so the existing named-preset bar
saves/restores it. Sourced from manifold/src/midi-devices (codegen).
Add a durable, committed source of truth for external MIDI synth control:
- synth-midi-cc.json: verified CC maps + provenance/sources (8 devices researched)
- schemas/midi_device.schema.json + schemas/midi_devices/*.json: 6 CC-controllable
device templates (Moog Sub 37/Sub Phatty, Creamware Pro-12 ASB, Elektron Analog
Keys, ASM Hydrasynth, Roland JD-800), params keyed {id, cc, label, min, max,
default, group}.
- codegen/generate-midi-devices.ts (isolated from the mode golden test) emits both
nisps/midi/generated/midi_devices.hpp (no-heap constexpr, firmware+WASM) and
manifold/src/midi-devices/generated/ (typed catalogue for the browser).
- codegen/seed-midi-devices.ts: reproducible seed from the research artifact.
Lets a performer pick a device and address its parameters by name (not CC number)
on both the firmware and the Manifold browser engine.
Route onMove through the XY-pad source's pushPad; instantiate useInputLayer
per engine and expose it on ConsoleCtx. The Inputs drawer now picks the
source(s) (XY pad / MIDI / gamepad toggles), configures each (gamepad
single/double stick; MIDI learn-map with per-binding clear), and shows the
composed channel layout + the blend-to-2 reshape notice referencing the
multi-WASM TODO (inputs-spec).
Workstream F core. Pull-based InputSource interface (N axes + discrete
actions), three sources (XYPadSource push-based, WebMidiInputSource with
learn-map CCs/notes, GamepadSource single/double stick), and an InputLayer
that composes active sources into one N-dim vector at the head of the
reactive spine. Arity reduction blends >2 axes down to the fixed 2-input
WASM head; the true multi-WASM reshape is a documented TODO (inputs-spec).
useInputLayer is the React binding (enable/config/status/channel layout).
Registers a named ParticleBackend in the BackendManager + barrel in place of
the inline particles PassthroughBackend. Transport is a no-op (the flow-field
canvas reads engine outputs directly via rAF); selecting it gates synth audio
(setMuted) like the other non-synth backends and reports a clear ready status.
Particle System is the default output Mode. When active it renders the
full-bleed ParticleStage (flow field driven by engine outputs each rAF) in
place of the manifold stage, regardless of the in/split/out/composite focus,
and suppresses the duplicate corner MEMLNaut label (the stage's top axis bar
already shows it).
Faithful TypeScript port of the a-immersive visualiser
(deployments/meml-aimmersive/js/ui/visualizer.js): 400 particles, identical
value-noise + 20-output param mapping + advection/attractor/dispersion/repulsor
integration. ParticleStage renders the full-bleed canvas (driven by engine
outputs each rAF) plus the macro-axis bar and a corner input pad.
Adds the Manifold app (manifold/) — the convertible-mode React front-end on
the real NISPS ML+audio engine, served live at meml.lnfinitemonkeys.org/next/.
Console chrome trimmed per UI pass:
- drop mode label + subtitle from the top-left overlay (keep MEMLNaut wordmark)
- remove the composite split preset/ratio readout (top-centre)
- remove the OUTPUT corner tag above the bars
- remove the A/B compare toggle from the verdict cluster
- remove the follow button + input/noise readout (bottom-left)
- remove AltitudeNav focus switcher (bottom-right)
