geo_push.hpp and replay.hpp cited memllib @ 0a541cc. upstream/main pins
e291192, where the same code had been deliberately redesigned — and because
InterfaceRL was not in the tree (fixed one commit ago), we carried the
superseded version for months. Three changes, all upstream's:
kGeometricPushScale 0.5 -> 1.0 (InterfaceRL.hpp:409)
kNegLRBase 0.5 -> 1.5 (InterfaceRL.hpp:410)
/(1+len) taper deleted (InterfaceRL.tpp:724)
Upstream's own comment on the taper: "a 'no' should clearly move the mapping
away even from a sound already far from the liked region (the taper used to
kill exactly that case)". The direction is already a unit vector, so the
taper only ever shrank the push for exactly the sounds a user is most likely
to be rejecting.
Cold start is folded into the same path. Upstream's useRandom is
`!havePositives || len <= 1e-4`: with nothing liked yet there is no centroid
to push away from, so every dim goes in a random direction. Ours instead
kept the older 0a541cc fallback — train AWAY from the heard action at a
NEGATIVE lr — which was inert whenever the heard action equalled the net's
own output, i.e. in the common case. One path now, and a "no" moves the
mapping before any likes exist (ml_bench E1: 0 -> 2.3e-3). The
GeometricColdStart action is still reported so callers keep their "like a
few sounds first" prompt; only the training changed.
Measured (ml_bench, one dislike at a point):
A4 0.0157 -> 0.0533 at-point displacement (3.4x), so end to end across
this and the RMSProp fix: 5.3e-5 -> 5.3e-2, ~1000x. The gap to the
legacy Diffuse design closes from ~4100x to ~4.2x.
D1 effective_lr 4.7e-4 -> 1.5e-3; 10 presses now reach 0.34, 100 reach
the full intended push.
A5 compounding 0.87 -> 0.96 (a second press at the same spot is no
longer noticeably weaker than the first).
A7 damage_ratio essentially unchanged (0.87-1.57) — the collateral
damage to protected positives scales with the push and is NOT
addressed here; it is the negative-feedback design question.
NOT adopted, deliberately: upstream's per-tick batch retraining over all
live negatives, and its fixed kDislikeLifetimeMs=2500 in place of our
proportional decay. Both need something the core does not have — a per-tick
call site and a millisecond clock inside nisps/ml — so they change
FeedbackControllerCore's interface rather than its constants. Recorded in
ALIGNMENT's deferred-debt entry alongside the existing one-press-one-step
divergence, and filed as its own task.
test_mlp_geo_dislike.cpp: the taper test now pins its ABSENCE (equal
displacement near and far), the cold-start test pins movement where it used
to pin inertness, and a new test covers the random-direction branch.
ALIGNMENT defect 6b resolved. Gates: build-cpp-tests 139 tests / ctest 4/4,
parity-check PASS (WASM rebuilt), lint-cpp clean, firmware slpworkshop
SUCCESS.
Upstream memlp (github.com/MusicallyEmbodiedML/memlp @ ea777502, the commit
upstream/main pins) applies gradients with RMSProp everywhere: Layer.h:239
ApplyAccumulatedGradients, the m_sq_grad_avg running average at Layer.h:601,
StaticMLP.h:268. nisps/ml/training.hpp shipped SGD only and filed the
difference as an optimiser-choice research question. It was not one.
RMSProp divides each step by the running gradient magnitude, so an upstream
lr is a NORMALISED step; under SGD the same number multiplies the raw
gradient. Every learning rate ported from upstream therefore landed in an
optimiser that reads it differently — most visibly feedback.hpp's
`geo_lr_ = 0.001f // upstream InterfaceRL.hpp:312`, an RMSProp LR pasted
into a single SGD step.
rmsprop_step() ports Layer.h:239 exactly: clip at +/-10, sq = min(0.9*sq +
0.1*g^2, 1e6), adj = min(lr/(sqrt(sq)+1e-6), 1.0), w -= adj*g. The
adjusted-LR clamp stays one-sided as upstream's std::min is, so the negative
lr used by train_targets' "train away from this target" path behaves as it
does upstream. The per-weight squared-gradient average is new persistent
state and lives in the storage policies (FixedStorage arrays /
DynamicStorage arena) so nisps/ stays allocation-free and the firmware's
zero-heap contract holds. It is optimiser state, not model state: excluded
from weight_count()/get_weights()/set_weights(), matching upstream, and
cleared by MLPCore::reset_optimizer_state() (upstream ResetOptimizerState).
draw_weights() deliberately does NOT clear it — upstream's DrawWeights
doesn't either.
Measured with tests/cpp/ml_bench.cpp:
D1 one geometric dislike moves the mapping 1.6e-2, up from 5.3e-5 (~295x),
and repeated presses now CONVERGE on the intended 0.5 push (0.12 at 10,
0.56 at 100) instead of creeping linearly forever.
A4 geometric-vs-Diffuse gap narrows from ~4100x to ~14x in one press.
U4 the upstream-LR positive path actually trains now (range_util 0.71 at
100 ticks/gesture, was 0.016 — it was inert under SGD).
Not fixed by this, and now tracked as ALIGNMENT defect 6d: the dose
asymmetry. lurch_max is still ~1.08 against a [0,1] output range.
Golden vector stages 2 and 3 re-captured; stages 0 and 1 are pre-training
and did not move, which is the cross-check that only the update rule
changed. manifold/public/nisps.wasm rebuilt so parity-check compares like
with like — it FAILED at up to 5e-2 against the stale artifact and PASSES at
2.4e-7 against a fresh one. parity-check.sh only builds the WASM when it is
missing, never when it is stale; noted in MAP.md and filed separately.
ALIGNMENT defect 6 resolved (moved to Recently resolved); 6b's optimiser
cross-reference updated; new defect 6d for the positive-training dose.
Gates: build-cpp-tests 138 tests / ctest 4/4, parity-check PASS, lint-cpp
clean, manifold typecheck clean.
NISPS is a controller, not a synth: the object of study is the mapping
f: control-space -> parameter-space and how a musician's gestures deform
it. Loss measures fit to points the user dictated, which is the one thing
they never experience. So this measures geometry and gesture-response.
tests/cpp/ml_bench.cpp 61 scenarios, REPORTS never asserts (same
discipline as engine_bench.cpp). Shape-
agnostic via MLPCore<DynamicStorage>
(--shape, default 2,16,16,16,8), seeded
RNG throughout, branch points replayed
from scratch rather than snapshotted.
tests/cpp/test_ml_behaviour.cpp 20 asserting invariants, wired into
nisps_core_tests.
scripts/bench-ml.sh native + WASM from one source; --compare,
--sweep-shape, --smoke, --scenario, --seed.
Documents two contracts that fail SILENTLY (both now pinned by tests):
a thumbs-up must call BOTH mlp.add_example() and fb.store_positive(),
since dislike_geometric k-NNs the replay buffer and not the MLP dataset;
and placed_output() is valid only while state == Placing, after which an
empty span whose l2() is 0 scores a broken lifecycle as a perfect place.
ALIGNMENT defect 6 re-ranked (SGD-vs-RMSProp is not a research axis - it
silently invalidated every ported hyperparameter) and split into 6b (the
geometric dislike was ported from a superseded upstream design) and 6c
(InterfaceRL, the reference impl, is not in the tree).
Gates: build-cpp-tests (138 tests, ctest 4/4), parity-check PASS,
lint-cpp clean, bench-ml.sh --smoke runs end to end.
Four items from one workflow, committed together because their build and CI
wiring genuinely interleaves — nisps/CMakeLists.txt, run-all-tests.sh and
ci.yml each carry hunks from two of them, and the stage renumbering (1/5 ->
1/6) touches every line. Splitting would produce commits that do not build,
which is worse than a commit that does four things and says so.
S26 part 2 — the curve declaration now matches reality. params[].curve stays
the mode-wide DEFAULT; a voice_spaces entry may now be {name, curve_overrides}
declaring only the slots where THAT voice space deviates. The 6 modes with one
voice space are byte-identical. The values were derived MECHANICALLY by a new
codegen/curve-audit.ts that models the four idioms a p[N]*p[N] regex misses
(alias form, memlcelium's implicit-counter sq() lambda, loop-generated indices,
smooth_params_), inlines helpers, and RAISES rather than guessing when it
cannot reduce an expression. A drift gate cross-checks 1179 (voice space x
param) slots against engine source on every run and was proved to fail loudly
on three drift classes. Application stays in the engine: nisps/engines,
nisps/pipeline and nisps/core are untouched, generated output is pure insertion
(755 insertions, 0 deletions), and the rebuilt nisps.wasm was byte-identical.
S4 / 7.2 — firmware reads the active mode's driver config at mode start, and
mic/line is real. My brief assumed the engine owns this; the code disagreed and
the code was right. sound_analysis_midi's EngineT is NoOpEngine — the mic lives
on a separately-composed AnalysisEngine member — so engine-level wiring would
have compiled, passed every gate, and left the one mic mode on line input.
Hence a mode-level seam defaulting to engine().driver_config(). Separately,
DriverConfig's defaults (line_level 0, output_volume 1.0) had drifted from
memllib's actual 3/0.8 because nothing had ever read them; wiring them as-is
would have made every silent mode louder and its line input maximally
insensitive — a behaviour change disguised as plumbing. Now pinned by a test.
Also: GetSysClockSpeed() panic()s on unsupported sample rates and runs on the
first line of setup(), so sample_rate needed a fallback ahead of clock setup.
CI's firmware env list gains soundanalysismidi — it is the only mic variant and
nothing else compiles that path.
Plan 5e — telemetry is real. A loss_history C-API entry across the full 5-layer
chain lets the browser read the per-iteration loss the core already records.
The audit named one fabrication site; there were two — wasm-iml.ts's
synchronous train() published lossHistory: [loss] as well. A third, ctx.loss,
was not merely dead but actively synthetic (fallbacks of prev * 0.82 and a
literal 0.5, rendered by nothing) and is deleted. The firmware buffer stays
untouched, per the L25 call. EngineApi.lossHistory() reads spine state rather
than the MLP handle, because trainAsync() fits on the worker's mirror net and
the handle would give a subtly-wrong second answer.
Plan 5f — engine throughput is measurable. One source compiled twice (CMake
natively, emcc for WASM) so the targets compare directly and no WASM export is
added. Sequencers are driven into a working state, and every row prints its own
working-state evidence so a number produced by an idle engine is visible rather
than plausible. Reports, never asserts: a wall-clock threshold on shared
hardware is meaningless or flaky, same call as the firmware size job.
ALIGNMENT: the telemetry defect is deleted (built, not deferred); the
performance defect is rewritten to what is actually left — these are HOST
numbers, and nothing measures the RP2350 at 150 MHz, which is the target the
mission's constraint is about. Q4 (memllib ownership) and Q5 (legacy feedback
modes) are closed.
Corrections to my own earlier claims, both found by agents contradicting the
brief: manifold/ONBOARDING.md was NOT "now accurate" — its primitives list
still named five deleted primitives and cited a seededGradient() that does not
exist. And the parity harness misses the sequencer engines because it runs 128
frames while their sequencers evaluate every 400-500 samples, NOT because
all-params-0.5 fails to trigger them (it does trigger: 0.5 maps to ratio 2,
firing three times per bar). The fix is a longer window, not different params.
Gates: run-all-tests.sh ALL GREEN — 4/4 ctest, parity PASS, lint clean, curve
drift 1179 slots ok, 39 e2e (was 33). Firmware: 5 envs built including the mic
variant.
Phase 3 (L8). Pure statement-for-statement relocation into
dsp/ratio_seq.hpp, dsp/seq_clock.hpp and core/event_queue.hpp.
AUDIT CORRECTION: L8 says "breakor and elysiamorf duplicate ratio_seq". That is
false — ElysiamorfEngine has no ratio_seq at all; it triggers continuously via
FM operators. The real duplicate pair is BreakOrEngine and MEMLCeliumEngine,
whose copies are byte-for-byte identical. Elysiamorf did share the clock and
event-queue machinery, so it uses those. memlcelium now includes the shared
ratio_seq too, which is what actually closes this finding.
Deliberately NOT folded into core/ring_buffer.hpp: RingBuffer is an
atomics-based cross-core SPSC channel (its header says so), whereas the engines'
event queue is produced and drained on one thread. Reusing it would have meant
paying for atomics to serve a single-threaded FIFO. The distinction is now
recorded in MAP.md so the next audit does not read them as duplicates.
Bit-exactness: verified the MIDI-clock tick and bar-phasor tick preserve the
original operation order with no floating-point re-association, and that
EventQueue keeps the original `% N` indexing rather than adopting RingBuffer's
bitmask. The golden suite (nisps_golden_tests) and the native<->WASM parity blob
both pass unchanged — they are the check, and they were not re-baselined.
Phase 2 (S30, L52, L51). These are the gates that are supposed to protect the
core's headline constraints, so the fix is demonstrated rather than asserted.
- S30: lint-cpp.sh's heap audit hardcoded dsp/engines/ml/modes, so nisps/core/
and nisps/pipeline/ — the P4 control-rate hot path — were NEVER scanned. Its
comment handling also post-filtered with `grep -v ' *//'`, which misses
trailing comments. Coverage is now exclusion-based (everything under nisps/
except wasm/, tests/, build/), comments are STRIPPED before matching using the
same perl strip audit_float_suffix already used, and the pattern set is
extended. dynamic_storage.hpp remains deliberately allowlisted — it is a real,
documented heap user that #errors on RP2350.
PROOF (run by me, not just reported): planted `new float[4]` in
nisps/pipeline/input_chain.hpp plus a std::malloc in the trailing-comment form
the old filter skipped.
old lint: "[lint-cpp] clean", exit 0 <- the false negative, live
new lint: FAIL, both lines named, exit 1
Plants removed; lint clean and exit 0 again; tree verified unmodified.
- L52: parity-check.sh's FAIL branch was unreachable — under `set -e` the script
died at the diff command before it could print anything. A mismatch therefore
failed silently. The diff now runs inside the `if` condition, where set -e does
not apply. Verified by forcing NISPS_PARITY_TOL=0: old script exited silently,
new one prints FAIL with the exit code.
- L51: parity_check.cpp and parity_wasm.mjs headers documented 4 stages / blob
v1 against a real 7-stage / v5 implementation. Comment-only rewrite; parity
behaviour unchanged (re-ran the gate: PASS, 1273 floats within 1e-5).
Note for future work: the lint now permanently scans nisps/core/ and
nisps/pipeline/, so changes there are enforced that were not before.
Phase 2, S35. Two real defects from one root cause, both confirmed by trace
rather than taken from the audit:
1. Divergence. WasmIML built its TS Dataset mirror with a cap of 100 while
every addExample() ALSO pushed into the C++ FIFO ring, capped at 128. Since
train() reads the C++ ring and trainAsync() reads the TS mirror, past 100
examples the two trained on different datasets — silently.
2. Latent OOB read. nisps_ml_train sizes its sample-weight span by the C++
side's example_count() (up to 128), but wasm-iml.ts allocates that heap
buffer from the TS dataset's size (<=100). Once the ring exceeds the mirror,
the span reads past the end of the caller's allocation.
Fix: name the capacity ONCE as nisps::ml::kDefaultMaxExamples = 128, used by
FixedStorage's default template arg, DynamicStorage's default ctor arg, and the
MLP<> alias (which is the only real FixedStorage instantiation path and carried
its own independent 128 literal — the last copy of this dual truth). Expose it
through nisps_ml_describe and have the TS side read it instead of hardcoding.
Dataset's constructor default is removed entirely: a default was what invited
this bug class, and the sole call site now always supplies the describe() value.
ABI NOTE: this extends nisps_ml_describe from a 6-int to a 7-int descriptor.
nisps_ml_describe always writes 7 ints regardless of the caller's buffer, so
every call site had to grow in the same change or it would overflow the WASM
heap by 4 bytes per call. All five sites updated: three in wasm-iml.ts (init
defaults, init per-instance, reshape re-describe — the finding said there were
two), one in wasm-worker.ts, one in tests/cpp/parity_wasm.mjs. The parity
harness's expected-dims check now also pins the new max_examples slot.
Regression test: tests/cpp/test_mlp_storage_defaults.cpp — pins the two storage
policies to one constant, and drives MLPCore<DynamicStorage> exactly as
bindings.cpp does past the old TS cap, asserting it saturates at 128 and not at
100. Fail-before/pass-after confirmed by temporarily setting the constant to
100: 2 failures, named. Reverted: green.
Audit correction: the cited dataset.ts:81 is the FIFO eviction check; the
hardcoded default was at dataset.ts:45.
Gates: run-all-tests.sh ALL GREEN, parity PASS.
Phase 1 group 4 (S33, S34, L54).
- S33: removed 12 dead entries across the full 5-layer registration chain
(bindings.cpp KEEPALIVE -> EXPORTED_FUNCTIONS -> the NispsModule declaration
table -> the WasmIML wrapper -> the EngineApi facade): nisps_ml_reset,
example_count, move_weights, feedback_learning_paused, feedback_drag,
jolt_lr_scale, jolt_tick_lr_ramp, pipeline_state_size/save_state/load_state,
feedback_placing and feedback_state. Each was grepped against manifold/src,
manifold/tests, the e2e specs, manifold/tests/wasm-load.ts and
tests/cpp/parity_wasm.mjs — the parity gate builds its own API via cwrap and
is a real consumer, so it counts.
KEPT deliberately: EXPORTED_RUNTIME's heap views + ccall/cwrap (the parity
harness and wasm-load.ts depend on them), and nisps_ml_feedback_static_output,
whose C export IS called directly by parity_wasm.mjs even though no TS
wrapper reaches it. Also dropped parity_wasm.mjs's moveWeights cwrap, which
was declared but never invoked.
- S34: deleted the publishWeights_ channel — EngineSink.setWeights,
Spine.setWeights/weights()/liveWeights and every call site. It ran a C->heap
copy plus a fresh Float32Array allocation at up to 200 Hz into a field
nothing read. getWeights survives for persistence and the debug probe.
- L54: worklet loader — deleted the unused imports object, the 'c' branch,
exMap and the duplicate second loop, and replaced the silent `() => 0` stub
with one that throws, so a missing import fails loudly instead of returning
plausible zeros into the audio path.
manifold/public/nisps.{js,wasm} rebuilt with the trimmed export list (emcc
3.1.69, the CI-pinned version) and committed — the freshness gate added in
Phase 0 requires it, and the webhook ships this artifact to production.
Gates: run-all-tests.sh ALL GREEN, parity 1273 floats within 1e-5.
Phase 1 group 2 (L27, L26, L28, S21, L13, ST6, S20).
- L27: fixed_buffer.hpp + its test + the CMake entry — no consumers.
- L26: dislike_multiplier_ and its doubling/halving bookkeeping — upstream
InterfaceRL residue that drove nothing. The audit pointed at the wrong test
file for the surviving reference; the actual assert was in
test_mlp_geo_dislike.cpp:211, removed here.
- L28: added copy_weights_to(std::span<float>) to FixedStorage and
DynamicStorage and switched feedback.hpp's take_snapshot/push_undo/nudge to
it. Drops the permanent whole-net flat_ scratch buffer from FixedStorage and
the per-gesture double copy. Behaviour-identical: same source values, same
write order, same RNG draw order in nudge().
- S21 + L13: deleted NISPS_AUDIO_MEM / NISPS_APP_SRAM / NISPS_AUDIO_FUNC —
zero use sites outside perf.hpp and comments — and rewrote midi_io.hpp's one
misshapen NISPS_AUDIO_FUNC use as a plain `inline void`. perf.hpp now
documents only the inlining/hotness macros that actually exist, and
audio_driver.hpp no longer claims an SRAM discipline the code never had.
- ST6: feedback.hpp's header now describes the four current modes and the
Geometric default, dropping the retracted "geometric push NOT ported" claim.
S20 — OPERATOR DECISION (§7.1): the four legacy feedback behaviours
(RandomiseOutputs, RandomiseMlp, AvoidStyle::Diffuse, the RandomiseMlp branch of
on_drag) are KEPT, not deleted. They are wanted as building blocks for
experimenting with how different instruments feel under different behaviours.
Each is now marked at its definition as deliberately-retained research reserve
so future audits stop flagging it as dead code.
L25 (the 16 KB firmware loss-history buffer) is NOT done here — see the phase
report; it turned out to be coupled into the shared mlp.hpp, and its fate
belongs with the browser telemetry build (§7.3 / plan §6.5e).
Gates: run-all-tests.sh ALL GREEN.
Phase 1 of the 2026-07 simplification audit, group 1. All verified dead by
independent grep across scripts/, .github/workflows/, codegen/, manifold/,
firmware/, vcv/, nisps/ and README.md before removal.
- S23 / L55: playground/ — only dist/ was tracked (1.2 MB of bundled JS,
sourcemaps and a second copy of nisps.wasm). The app itself is preserved on
branch archive/playground-solidjs and tag playground-solidjs-final.
- S29: the root Playwright rig — playwright.config.js, package.json,
package-lock.json and tests/e2e/. It served ./playground, a page that no
longer exists anywhere in the tree, and was invoked by nothing. NOTE: the
live suite is manifold/tests/e2e/ with manifold/playwright.config.ts, which
is untouched; every CI Playwright step runs with working-directory: manifold.
- L48: NISPS_CORE_EXTRACTION_PLAN.md + NISPS_CORE_TASKS.md (818 lines of
extraction relics; docs/specs/plans/ is the sanctioned home for plans).
- L36: data/ — two CSVs from a retired era, plus a tracked LibreOffice lock file.
- L32: .claude/worktrees/agent-ae87fe47/ only. The audit's wording invites
deleting .claude/worktrees/ wholesale; that would have been destructive —
the directory holds 12 LIVE registered git worktrees (332 MB), four of them
on branches with unpushed commits. Verified agent-ae87fe47 is NOT registered
before removing it; every live worktree is left intact. Cleaning up the rest
is a separate operator decision.
- ST10 fallout: dropped the dead `port-solidjs` branch trigger from ci.yml and
the dead `feat/manifold-mission` / `feat/vcv-dist` triggers from
vcv-plugin.yml — those branches are stale and the SolidJS target is retired.
tests/cpp/ is untouched. Gates: run-all-tests.sh ALL GREEN.
Replace the vendored runtime MLP in vcv/src/iml.hpp (DetRng + 3D-weight-store
MLP + Dataset + IML) with a THIN, Rack-free adapter over the shared core:
nisps::ml::MLPCore<nisps::ml::DynamicStorage> (8->[16,24,16]->16, the P2 dynamic
case), nisps::Rng, and the core MLP's own FIFO dataset. Behaviour changes from
the vendored approximation to core-exact firmware/WASM semantics.
- MEMLNaut.cpp: staged/pending weight buffers and patch JSON now use the core's
flat [weights..][biases..] vector (nisps::IML<float>::Weights); patch version
bumped to 3. Double-buffer / single-writer threading discipline unchanged.
- New ctest tests/cpp/test_vcv_iml_parity.cpp: seeded train/infer/move_weights
session through the adapter is memcmp-equal to a bare MLPCore<DynamicStorage>.
- Docs: vcv-module.md delta #5 marked CLOSED (2026-07-18); MAP.md vcv/ updated.
Closes vcv-module.md delta #5.
Operator-approved ABI change (P2 stop-point). The WASM MLP is now
MLPCore<DynamicStorage>:
- nisps_ml_create(input, output, hidden[3], n, seed) honours its args;
non-positive/null fall back to the historical 32→[10,14,18]→126, so
every pre-P2 caller (manifold, worker, parity harness) stays
bit-identical. Invalid/oversized dims (>4096) → null.
- NEW nisps_ml_reshape(ml, in, out, hidden, n, spread): fresh net at the
new dims, warm-started via nisps/ml/warm_start.hpp (overlapping region
copied; rest keeps spread init); feedback controller re-created (state
resets — reset-on-reshape modal is the front-end contract). Failure
leaves the old net untouched.
- nisps_ml_describe(ml, out): takes the handle; null reports defaults.
- FeedbackController got the same storage split: algorithms in
FeedbackControllerCore<FbStorage>; FixedFeedbackStorage keeps firmware/
tests source-identical via the old alias; DynamicFeedbackStorage (one
arena) sizes to the runtime net. Firmware .text unchanged (122692).
- MLHandle: per-instance scratch vectors; dropped the dead 2MB
batch_out_scratch.
- TS: types.ts decls (+_nisps_ml_reshape), wasm-iml re-describes the
created instance, worker carries a shape-contract note for P2.3.
Verified: ctest 4/4 incl. new warm-start grow/shrink test; reshape ABI
smoke (dims honoured, overlap survives, invalid rejected, outputs
bounded); parity PASS unchanged (2.4e-7); lint clean; manifold 9 unit +
20 e2e green; firmware .text 122692 (+0.30% vs pre-P2 baseline).
Algorithms (forward, backprop/SGD, init, move_weights, diagnostics) now live
once in MLPCore<Storage> (nisps/ml/mlp.hpp). Storage models:
- FixedStorage (storage.hpp): template-sized std::array, zero heap. The
classic MLP<NIn,H1,H2,H3,NOut,...> is an alias preserving kInput/kHidden*/
kOutput/kNumLayers/weight_count() constexpr — firmware + bindings + modes
compile unchanged.
- DynamicStorage (dynamic_storage.hpp): runtime dims, ONE arena allocation
at construction, nothing per-call. #error under NISPS_TARGET_EMBEDDED
(new macro in core/perf.hpp); sole lint-cpp.sh heap-allowlist entry, plus
a lint check that fails if the #error guard disappears.
Verification:
- new ctest test_mlp_storage_parity: fixed↔dynamic BIT-identical across
init/draw/inference/train(FIFO)/move_weights(pin mask)/eval_loss/
layer_stats/set_weights/infer_batch/reset; invalid+moved-from inert
- golden ML vectors (pre-refactor constants) pass → bit-stable refactor
- native↔WASM parity PASS, max delta unchanged (2.4e-7)
- chokepoint B compile: PAFSynth .text 122324→122692 (+0.30%, ±1% budget);
RAM +416B (eval scratch)
- fix: firmware-common.sh used bare 'python' (absent here) → ${PYTHON:-python3}
Part of one-core-engine-refactor P2. nisps_ml_create ABI untouched (P2.2 is
an operator stop-point).
Add a "grab → move → drop" gesture that moves an existing positive
example's output to a new input position, preserving the output. This is
the new core's home for upstream InterfaceRL's drag-store/reposition-commit,
distinct from Explore→Place (which places newly-auditioned scratchpad sounds).
- nisps/ml/feedback.hpp: begin_reposition()/commit_reposition()/repositioning().
Reuses the Placing state (static_output holds the carried vector) but a
reposition_ flag makes commit AND the mode-switch teardown SKIP the weight
restore — the real net is never set aside here, so restoring snapshot_ would
clobber the live trained net. Guards cancel_place + abort_explore_place.
- firmware glue: state-gate Toggle B. Exploring → reroll/nudge (unchanged);
Idle → MomB1 grab, MomB2 drop (commit + add_example + train). The 4D variant
has no joystick button, so the gesture lives on the momentary toggle. Also
fix a stale top-of-file control-map comment that contradicted the bindings.
- tests: 4 reposition cases (hold without snapshot; commit stores carried
output with no restore; mode-switch aborts without clobber; begin-only-Idle).
- MAP.md: document the full ExploreAndPlace lifecycle + reposition wiring.
Audio-hold (carrying the sound audibly during the move) remains the existing
unwired static_output() TODO and affects Explore→Place identically.
Firmware compile unverified (no arduino-cli); host tests + lint pass.
New SLP-Workshop firmware variant (Synth Library Portland), built on the
MEMLCelium engine + MLP shape. Ports the two post-fork learning-algorithm
changes from upstream memllib InterfaceRL into the shared nisps/ml core,
runtime-configurable (no compile-time switch), inert by default:
- nisps/ml/jolt.hpp: Jolt — held continuous weight morph over the flat
weight buffer + post-release LR ramp (kJolt* constants verbatim).
- nisps/ml/ou_noise.hpp: OUNoise<N> — Ornstein-Uhlenbeck exploration walk
on the output vector (theta=0.02, dt=0.001, kMaxAmplitude=0.65).
Both wired into ModeBase so every mode gains jolt_press/jolt_release/
jolt_lr_scale + set_explore_intensity; gated so existing modes stay
bit-identical (parity + golden tests green). Firmware surfaces them on
TogB1 (Jolt) and RVX1 (explore). New SLPWorkshopMode mode + schema +
codegen; firmware alias + .ino variant; playground mode registration.
Tests: jolt + OU unit tests, ModeBase learning integration incl. an
inert-parity test proving SLP-Workshop == MEMLCelium with features off.
Verified: cpp tests, wasm build, native↔wasm parity, lint, codegen
golden, playground typecheck. Firmware compile/e2e/hardware are
environment-bound (no arduino-cli/submodules/browser here).
Refs ergo 019f0fca.
Platform-agnostic neutral-pin mechanism (set_input_pinned/pin_value) on
ModeBase, lifted from ebb953a. Consumed by the firmware Joystick Dual/Single
settings menu; the browser fixed-MLP<4> half of ebb953a is superseded by main's
MLP<32> N-D input work and intentionally dropped.
Wire the modular input layer into the Console and reshape the browser
engine so input axes are genuine independent dimensions.
Inputs (manifold/src/inputs/):
- gamepad-source: emit press+release edges with standard-mapping labels
(enables hold-and-move); single/double-stick already present.
- midi-input-source: single-device selection + batch "MIDI Learn"
(every CC swept while armed becomes an axis); notes stay discrete.
- input-layer: compose() forwards each axis 1:1 (no mean-blend);
add onReducedInput so the manifold tracks gamepad/MIDI position.
- types: InputAction.phase, InputMode.
Console (manifold/src/console/):
- ConsoleApp: bind gamepad buttons to verdicts (RB up / LB down /
X randomise / Y nudge / B undo / A-hold reposition); mirror composed
position onto the manifold.
- Drawers: rebuilt Inputs drawer (source picker, gamepad legend, MIDI
device picker + batch-learn flow, learned-control meters).
Engine (nisps/wasm, manifold/src/engine):
- DefaultMLP widened MLP<2,..> -> MLP<32,..> (32 = MAX_AXES); each
active axis gets a dedicated slot, unused slots held at 0 (inert).
Rebuilt nisps.wasm (playground + manifold).
- spine/engine-api: setInputs writes the full N-D vector (was dropping
arr[2+]); primary pair keeps the 2-D pipeline; process() re-ticks the
whole vector via spine.reprocess().
Tests:
- parity_check/parity_wasm: ParityMLP -> 32 inputs, widen example bufs.
- CMakeLists: build parity binary with -ffp-contract=off so native
matches FMA-free WASM (training amplified the gap past 1e-5).
Inputs dock is still an exclusive picker; mixing toggles, reshape modal,
and the >2-D slider view (inputs-spec.md) are groundwork-laid but not
yet wired. See docs/redesign/midi-gamepad-inputs-worklog.md.
CMakeLists.txt:
- Native host build by default; Emscripten-target detection plumbed but
WASM emit deferred to stream 7 (playground build script).
- Header-only INTERFACE library `nisps_core`.
- Host test executable `nisps_core_tests` compiled with -Wall -Wextra
-Werror -Wpedantic (Chris's rules: clean build is non-negotiable).
tests/cpp/test_helpers.hpp:
- Minimal NISPS_TEST / NISPS_EXPECT / NISPS_EXPECT_NEAR macros, no external
deps. Rationale documented in-file: Catch2/doctest would add ~10MB and 30s
for what is currently <100 LOC of test runtime.
22 unit tests covering FixedBuffer (5), RingBuffer (5), Rng (7), math (5).
All green; verified via `cmake --build nisps/build && ./nisps/build/nisps_core_tests`.
b290144 made matrix cells opt-in to prevent joystick-silences-voice,
but that broke modular-ui.updateLive(): the matrix DOM stopped
reflecting live MLP outputs because matrixIndexCache was empty when
buildMatrixIndex() walked paramMeta. This was the same regression
6072fe8 had previously fixed.
Fix it structurally at the DSP layer instead: amp_val now computes
as `clamp(base_amp + max(0, mod_amp)) * level * vel_gain`, so matrix
d08_amp cells can only boost the amp floor — never cut it. base_amp
defaults to 1.0 (always audible), and presets that want classic
envelope-gated voices (slow pad, plucky bass, crystal, morphing
drone) drop base_amp to 0 and layer a positive ADSR→amp route on top.
With the DSP guard in place, all 480 matrix cells can safely live
in paramMeta again, and updateLive() gets its live visual feedback
back. Revert the opt-in gate in _rebuildParamMeta and the 32-param
test counts, and add a regression test asserting that every matrix
destination has 48 cells in paramMeta — that's what updateLive needs.
Modular sub-engines computed amp_val as a pure function of mod_amp (the
matrix d08_amp destination sum), so once the MLP drove the matrix cells
every joystick movement had a chance to silence the voice: matrix cells
have signed range [-1, 1], sigmoid outputs near 0.5 denormalise to 0,
and the amp gate collapsed. Additive survived in scattered regions
because it only has one kill-switch (d08_amp); subtractive and fm were
almost always dead because they also have d05_cutoff and d01_op1_level.
Two changes:
1. DSP: each sub-engine gets a base_amp hslider (default 1.0) so
amp_val = clamp(base_amp + mod_amp) * level * vel_gain. At the
default the voice is always fully open and d08_amp modulation is
purely additive decoration; drop base_amp to 0 for classic
ADSR-gated VCA behaviour.
2. ModularEngine._rebuildParamMeta: restore the _exposedMatrixCells
gate (default empty). paramCount drops from 512 to 32 (4 ADSR * 4
+ 8 LFO * 2); matrix cells are opt-in via setExposeMatrixCell.
_applyDefaultPatch no longer writes s00_d08_amp since base_amp
keeps the voice audible without routing.
Tests updated for the new 32-param baseline; matrix-cell persistence
test now calls setExposeMatrixCell(1, 5, true) before asserting the
cell lands in paramMeta. Drive-by: engine-switching test bumped from
3 to 4 engine cards (stale since the modular engine was added).
New "Modular" engine in a-immersive with three hot-swappable Faust
sub-engines (subtractive/additive/fm) sharing a common modulation pool:
16 ADSR slots + 32 LFO slots (single-knob sine->tri->square->saw
wavemorph) routed through a 48-source x 10-destination matrix per
engine. Per-connection scalar amounts in [-1, 1], summed at each
destination. Default MLP output count is 512 (32 mod-source params +
480 matrix cells); model reinits on sub-engine swap, count change, or
engine-param exposure toggle.
Faust layer:
- mod-pool.lib: shared ADSR/LFO/source-bus library
- gen-modular-dsp.py: byte-reproducible generator (source of truth)
- modular-subtractive: faithful Minimoog (3 osc, ladder filter, no envs)
- modular-additive: 64-partial, spectral shape + formants, no envs/LFOs
- modular-fm: 4-op matrix + self-feedback, no envs
- All three share d08=amp, d09=pan conventions
- MODULAR_DESTINATIONS.md: authoritative destination table
JS layer:
- ModularEngine: self-contained SynthEngine with getState/setState,
setSubEngine, setModSourceCount, setExposeEngineParam
- modular-ui: drawer with sub-engine toggle, ADSR/LFO count steppers,
per-slot enable switches, matrix grid editor (tap-cycle, long-press
precise, right-click menu, negative amounts), preset overlay
- modular-presets: 6 named presets (Slow pad, Plucky bass, Crystal,
DX bell, Morphing drone + default)
- a-app.js: Modular mode registered, paramMeta:change -> resizeMLP,
modular DSP state persisted under modularDspState, window.__nisps
debug hooks for programmatic control
Tests: tests/e2e/modular-mode.spec.js (11 Playwright tests, all passing
including DSP state survives reload, sub-engine swap keeps paramCount,
preset apply verification).
Also fixes a pre-existing build.sh bug where the -e flag caused faust
to overwrite .wasm outputs with expanded DSP source text, leaving
additive/fm-matrix/eoc-* committed as invalid WebAssembly. Rebuilt all
affected engines with the corrected script. Added an early-message
buffer to faust-worklet-processor.js so setParam calls arriving before
wasm instantiation are queued rather than dropped (needed when the user
configures modular state before clicking Start Audio).
Tests cover:
- Default state (C15, 126 params, heatmap cells)
- Engine switcher UI (3 cards, active state)
- Switch to Additive (48 params, outputs bounded, training works)
- Switch to FM Matrix (55 params, outputs bounded, training works)
- Round-trip switching (C15→Add→C15, C15→FM→Add→C15)
- Warm-start weight preservation across resize
- EOC chain accessibility across engines
- SynthVisualizer visibility across engines
- No console errors during any switch
Also fixes: faustJsonToParamMeta now filters [hidden:1] params,
giving correct counts (48 additive, 55 FM) instead of including
freq/gate/_vel control inputs.
- Add window.__nisps debug probe (gated on ?debug=1) exposing iml state,
getOutputs/getLoss/getWeights/getExampleCount, and action triggers
(thumbsUp/thumbsDown/train/randomise/clearExamples/saveState)
- Fix WasmIML bug: this.dataset was a plain object; import Dataset and
use new Dataset(100) so computeWeights() is available for training
- Fix WasmIML.addExample/clearDataset to use Dataset API methods
- 44 Playwright e2e tests across 4 spec files:
- ml-engine.spec.js: WASM inference bounds, training loss, thumbs
up/down behavior, async training, example capture semantics
- ui-interactions.spec.js: drawer open/close, mode switching,
heatmap bar counts, preset chips, keyboard shortcuts (1/2/Z)
- input-pipeline.spec.js: input→output variation, clamping, joystick
drag, post-training output bounds across the full input space
- persistence.spec.js: URL params (?preset, ?spread), localStorage
round-trip, saveState probe
