geo_push.hpp and replay.hpp cited memllib @ 0a541cc. upstream/main pins
e291192, where the same code had been deliberately redesigned — and because
InterfaceRL was not in the tree (fixed one commit ago), we carried the
superseded version for months. Three changes, all upstream's:
kGeometricPushScale 0.5 -> 1.0 (InterfaceRL.hpp:409)
kNegLRBase 0.5 -> 1.5 (InterfaceRL.hpp:410)
/(1+len) taper deleted (InterfaceRL.tpp:724)
Upstream's own comment on the taper: "a 'no' should clearly move the mapping
away even from a sound already far from the liked region (the taper used to
kill exactly that case)". The direction is already a unit vector, so the
taper only ever shrank the push for exactly the sounds a user is most likely
to be rejecting.
Cold start is folded into the same path. Upstream's useRandom is
`!havePositives || len <= 1e-4`: with nothing liked yet there is no centroid
to push away from, so every dim goes in a random direction. Ours instead
kept the older 0a541cc fallback — train AWAY from the heard action at a
NEGATIVE lr — which was inert whenever the heard action equalled the net's
own output, i.e. in the common case. One path now, and a "no" moves the
mapping before any likes exist (ml_bench E1: 0 -> 2.3e-3). The
GeometricColdStart action is still reported so callers keep their "like a
few sounds first" prompt; only the training changed.
Measured (ml_bench, one dislike at a point):
A4 0.0157 -> 0.0533 at-point displacement (3.4x), so end to end across
this and the RMSProp fix: 5.3e-5 -> 5.3e-2, ~1000x. The gap to the
legacy Diffuse design closes from ~4100x to ~4.2x.
D1 effective_lr 4.7e-4 -> 1.5e-3; 10 presses now reach 0.34, 100 reach
the full intended push.
A5 compounding 0.87 -> 0.96 (a second press at the same spot is no
longer noticeably weaker than the first).
A7 damage_ratio essentially unchanged (0.87-1.57) — the collateral
damage to protected positives scales with the push and is NOT
addressed here; it is the negative-feedback design question.
NOT adopted, deliberately: upstream's per-tick batch retraining over all
live negatives, and its fixed kDislikeLifetimeMs=2500 in place of our
proportional decay. Both need something the core does not have — a per-tick
call site and a millisecond clock inside nisps/ml — so they change
FeedbackControllerCore's interface rather than its constants. Recorded in
ALIGNMENT's deferred-debt entry alongside the existing one-press-one-step
divergence, and filed as its own task.
test_mlp_geo_dislike.cpp: the taper test now pins its ABSENCE (equal
displacement near and far), the cold-start test pins movement where it used
to pin inertness, and a new test covers the random-direction branch.
ALIGNMENT defect 6b resolved. Gates: build-cpp-tests 139 tests / ctest 4/4,
parity-check PASS (WASM rebuilt), lint-cpp clean, firmware slpworkshop
SUCCESS.
InterfaceRL.{hpp,cpp,tpp} + InterfaceRLFileFormat.hpp are the upstream
reference implementation of the entire NISPS feedback subsystem —
nisps/ml/{geo_push,replay,feedback,jolt,ou_noise}.hpp are all ports of it,
several still carrying `// upstream InterfaceRL.hpp:NNN` line references.
The Phase-4 vendoring dropped examples/ because nothing compiled it. That
was correct for the build and wrong for the codebase: with the source of
truth out of tree, upstream redesigned the geometric dislike (deleted the
/(1+len) taper, doubled kGeometricPushScale, tripled the negative-LR base,
moved to batch training over all negatives every tick) and we did not notice
for months.
Copied verbatim from memllib @ e291192 — the same commit the rest of the
vendored tree pins — into lib/memllib/reference/, which sits OUTSIDE src/
and is therefore never compiled: PlatformIO's LDF only recursively builds an
Arduino-format library's src/ folder. Verified: slpworkshop still builds
(RAM 28.8%, flash 2.1%).
reference/README.md states the two rules (never compiled, never edited — a
divergence from upstream is a recorded decision, not an edit here) and
VENDORED.md's "what was dropped" section now tells the truth.
Resolves ALIGNMENT defect 6c.
Upstream memlp (github.com/MusicallyEmbodiedML/memlp @ ea777502, the commit
upstream/main pins) applies gradients with RMSProp everywhere: Layer.h:239
ApplyAccumulatedGradients, the m_sq_grad_avg running average at Layer.h:601,
StaticMLP.h:268. nisps/ml/training.hpp shipped SGD only and filed the
difference as an optimiser-choice research question. It was not one.
RMSProp divides each step by the running gradient magnitude, so an upstream
lr is a NORMALISED step; under SGD the same number multiplies the raw
gradient. Every learning rate ported from upstream therefore landed in an
optimiser that reads it differently — most visibly feedback.hpp's
`geo_lr_ = 0.001f // upstream InterfaceRL.hpp:312`, an RMSProp LR pasted
into a single SGD step.
rmsprop_step() ports Layer.h:239 exactly: clip at +/-10, sq = min(0.9*sq +
0.1*g^2, 1e6), adj = min(lr/(sqrt(sq)+1e-6), 1.0), w -= adj*g. The
adjusted-LR clamp stays one-sided as upstream's std::min is, so the negative
lr used by train_targets' "train away from this target" path behaves as it
does upstream. The per-weight squared-gradient average is new persistent
state and lives in the storage policies (FixedStorage arrays /
DynamicStorage arena) so nisps/ stays allocation-free and the firmware's
zero-heap contract holds. It is optimiser state, not model state: excluded
from weight_count()/get_weights()/set_weights(), matching upstream, and
cleared by MLPCore::reset_optimizer_state() (upstream ResetOptimizerState).
draw_weights() deliberately does NOT clear it — upstream's DrawWeights
doesn't either.
Measured with tests/cpp/ml_bench.cpp:
D1 one geometric dislike moves the mapping 1.6e-2, up from 5.3e-5 (~295x),
and repeated presses now CONVERGE on the intended 0.5 push (0.12 at 10,
0.56 at 100) instead of creeping linearly forever.
A4 geometric-vs-Diffuse gap narrows from ~4100x to ~14x in one press.
U4 the upstream-LR positive path actually trains now (range_util 0.71 at
100 ticks/gesture, was 0.016 — it was inert under SGD).
Not fixed by this, and now tracked as ALIGNMENT defect 6d: the dose
asymmetry. lurch_max is still ~1.08 against a [0,1] output range.
Golden vector stages 2 and 3 re-captured; stages 0 and 1 are pre-training
and did not move, which is the cross-check that only the update rule
changed. manifold/public/nisps.wasm rebuilt so parity-check compares like
with like — it FAILED at up to 5e-2 against the stale artifact and PASSES at
2.4e-7 against a fresh one. parity-check.sh only builds the WASM when it is
missing, never when it is stale; noted in MAP.md and filed separately.
ALIGNMENT defect 6 resolved (moved to Recently resolved); 6b's optimiser
cross-reference updated; new defect 6d for the positive-training dose.
Gates: build-cpp-tests 138 tests / ctest 4/4, parity-check PASS, lint-cpp
clean, manifold typecheck clean.
NISPS is a controller, not a synth: the object of study is the mapping
f: control-space -> parameter-space and how a musician's gestures deform
it. Loss measures fit to points the user dictated, which is the one thing
they never experience. So this measures geometry and gesture-response.
tests/cpp/ml_bench.cpp 61 scenarios, REPORTS never asserts (same
discipline as engine_bench.cpp). Shape-
agnostic via MLPCore<DynamicStorage>
(--shape, default 2,16,16,16,8), seeded
RNG throughout, branch points replayed
from scratch rather than snapshotted.
tests/cpp/test_ml_behaviour.cpp 20 asserting invariants, wired into
nisps_core_tests.
scripts/bench-ml.sh native + WASM from one source; --compare,
--sweep-shape, --smoke, --scenario, --seed.
Documents two contracts that fail SILENTLY (both now pinned by tests):
a thumbs-up must call BOTH mlp.add_example() and fb.store_positive(),
since dislike_geometric k-NNs the replay buffer and not the MLP dataset;
and placed_output() is valid only while state == Placing, after which an
empty span whose l2() is 0 scores a broken lifecycle as a perfect place.
ALIGNMENT defect 6 re-ranked (SGD-vs-RMSProp is not a research axis - it
silently invalidated every ported hyperparameter) and split into 6b (the
geometric dislike was ported from a superseded upstream design) and 6c
(InterfaceRL, the reference impl, is not in the tree).
Gates: build-cpp-tests (138 tests, ctest 4/4), parity-check PASS,
lint-cpp clean, bench-ml.sh --smoke runs end to end.
Four items from one workflow, committed together because their build and CI
wiring genuinely interleaves — nisps/CMakeLists.txt, run-all-tests.sh and
ci.yml each carry hunks from two of them, and the stage renumbering (1/5 ->
1/6) touches every line. Splitting would produce commits that do not build,
which is worse than a commit that does four things and says so.
S26 part 2 — the curve declaration now matches reality. params[].curve stays
the mode-wide DEFAULT; a voice_spaces entry may now be {name, curve_overrides}
declaring only the slots where THAT voice space deviates. The 6 modes with one
voice space are byte-identical. The values were derived MECHANICALLY by a new
codegen/curve-audit.ts that models the four idioms a p[N]*p[N] regex misses
(alias form, memlcelium's implicit-counter sq() lambda, loop-generated indices,
smooth_params_), inlines helpers, and RAISES rather than guessing when it
cannot reduce an expression. A drift gate cross-checks 1179 (voice space x
param) slots against engine source on every run and was proved to fail loudly
on three drift classes. Application stays in the engine: nisps/engines,
nisps/pipeline and nisps/core are untouched, generated output is pure insertion
(755 insertions, 0 deletions), and the rebuilt nisps.wasm was byte-identical.
S4 / 7.2 — firmware reads the active mode's driver config at mode start, and
mic/line is real. My brief assumed the engine owns this; the code disagreed and
the code was right. sound_analysis_midi's EngineT is NoOpEngine — the mic lives
on a separately-composed AnalysisEngine member — so engine-level wiring would
have compiled, passed every gate, and left the one mic mode on line input.
Hence a mode-level seam defaulting to engine().driver_config(). Separately,
DriverConfig's defaults (line_level 0, output_volume 1.0) had drifted from
memllib's actual 3/0.8 because nothing had ever read them; wiring them as-is
would have made every silent mode louder and its line input maximally
insensitive — a behaviour change disguised as plumbing. Now pinned by a test.
Also: GetSysClockSpeed() panic()s on unsupported sample rates and runs on the
first line of setup(), so sample_rate needed a fallback ahead of clock setup.
CI's firmware env list gains soundanalysismidi — it is the only mic variant and
nothing else compiles that path.
Plan 5e — telemetry is real. A loss_history C-API entry across the full 5-layer
chain lets the browser read the per-iteration loss the core already records.
The audit named one fabrication site; there were two — wasm-iml.ts's
synchronous train() published lossHistory: [loss] as well. A third, ctx.loss,
was not merely dead but actively synthetic (fallbacks of prev * 0.82 and a
literal 0.5, rendered by nothing) and is deleted. The firmware buffer stays
untouched, per the L25 call. EngineApi.lossHistory() reads spine state rather
than the MLP handle, because trainAsync() fits on the worker's mirror net and
the handle would give a subtly-wrong second answer.
Plan 5f — engine throughput is measurable. One source compiled twice (CMake
natively, emcc for WASM) so the targets compare directly and no WASM export is
added. Sequencers are driven into a working state, and every row prints its own
working-state evidence so a number produced by an idle engine is visible rather
than plausible. Reports, never asserts: a wall-clock threshold on shared
hardware is meaningless or flaky, same call as the firmware size job.
ALIGNMENT: the telemetry defect is deleted (built, not deferred); the
performance defect is rewritten to what is actually left — these are HOST
numbers, and nothing measures the RP2350 at 150 MHz, which is the target the
mission's constraint is about. Q4 (memllib ownership) and Q5 (legacy feedback
modes) are closed.
Corrections to my own earlier claims, both found by agents contradicting the
brief: manifold/ONBOARDING.md was NOT "now accurate" — its primitives list
still named five deleted primitives and cited a seededGradient() that does not
exist. And the parity harness misses the sequencer engines because it runs 128
frames while their sequencers evaluate every 400-500 samples, NOT because
all-params-0.5 fails to trigger them (it does trigger: 0.5 maps to ratio 2,
firing three times per bar). The fix is a longer window, not different params.
Gates: run-all-tests.sh ALL GREEN — 4/4 ctest, parity PASS, lint clean, curve
drift 1179 slots ok, 39 e2e (was 33). Firmware: 5 envs built including the mic
variant.
One cut, no dual path. Closes ALIGNMENT defect 3 ("Arduino-CLI build
machinery is actively hostile") and vision bullet 4.
platformio.ini carries 16 [env:], one per variant, each passing
-DMEMLNAUT_MODE_TYPE; selftest passes -DNISPS_SELFTEST=1 instead. The env list
IS the registry now — the .ino comment-registry and the NISPS_ST_* token-paste
table are deleted rather than migrated. L12 noted that table was already
silently missing the currently-shipped SLPWorkshop variant, which is the whole
argument against having a second list.
Also deleted: the Python/sed machinery that rewrote the COMMITTED .ino on every
build, the sketch symlink forest, the global TFT_eSPI User_Setup.h mutation
(now -D flags — TFT_eSPI's own documented PlatformIO recipe), the UF2
boot-mount detection stack (upload_protocol=picotool talks to the bootloader
directly), and build-firmware-arch.sh entirely. Scripts 683 -> 435 lines.
memllib is vendored at lib/memllib/ from upstream e291192; no submodules
remain. VENDORED.md records provenance and the re-sync procedure.
S9: a firmware-build CI job compiles three representative envs against a cached
toolchain and reports per-variant flash/RAM. Firmware is in an automated gate
for the FIRST time. The old ci.yml comment justified excluding it as "low
verification value" — an assessment that did not survive contact, since the
SelfTest variant sat broken for an unknown period calling a DisplayDriver
method that did not exist at the pinned memllib commit, and nothing noticed
because nothing built it.
Verified: all 16 envs build from an empty cache, each within ~520 bytes of the
arduino-cli binary it replaces, flash and RAM. Measured as .text+.rodata /
.data+.bss+vector+uninitialized — NOT PlatformIO's console line, which
double-counts .data on this board. This does not prove the hardware boots; no
flash+smoke test was possible and that stays an operator chokepoint.
slpworkshop 248232/145028 pafsynth 256880/149716 selftest 216228/17960
(all 16 in the CI log format; none exceeds 2% of a 16 MB flash)
Two traps recorded so nobody rediscovers them: vendoring memllib's subdirs
without a src/ wrapper makes PlatformIO's library builder silently compile
NOTHING while still linking; and project build_flags land BEFORE the
framework's own -std=gnu++17 -Os, so build_unflags is required.
CORRECTION carried in this commit: the firmware sizes in c19d846's message and
the first version of the memllib recon doc were wrong — SLPWorkshop 145348,
PAFSynth 145300, SelfTest 141840. They came from building variants in sequence
through a SHARED incremental arduino-cli build directory, which reused stale
objects and under-reported by ~75 KB. Clean-cache rebuilds of the identical
commit give 216736/18492 for SelfTest. The real cost of the memllib upstream
bump is +216 bytes flash, not +316. Never measure firmware size through a
reused build dir.
HISTORY NOTE: this commit and the docs commit before it were rebuilt (force-push,
2026-07-21) so that each contains only what its message describes. The first
versions had the firmware deletions stranded in the docs commit by a shared-index
race between concurrent agents; content is byte-identical to the originals.
Gates: run-all-tests.sh ALL GREEN (nisps/ untouched by this change beyond
include paths); 16/16 pio envs build.
Roughly 20k lines were deleted from this repo in the last week and much of the
corpus still described the pre-deletion world in the present tense. Executes
the §8 table: archive the retired, reclassify the executed, prune the stale.
aimmersive-clone-spec -> _archive/ with a deprecated-by note
feedback-modes-port-spec -> plans/, kind: plan, status: executed
manifold-parity-features -> plans/, kind: plan, status: active
playground-2.0-rewrite -> status: superseded
engine-architecture 434 -> ~120 lines; seam + spine kept, rewritten
present-tense against the shipped engine/
MAIN.md six contradicted claims fixed; registry resynced
vcv-module.md pruned to the current 8->16 contract and made the
single .nisps format spec
vcv/NISPS-FORMAT.md DELETED — documented a v1 format that no longer loads
vcv/README.md, BUILDING.md rewritten to the real contract, menu, OSC table
inputs/backends/dock trio grounding sections marked historical, dead cites fixed
Two rows of the §8 table were themselves wrong, corrected here: the deleted
full-state sync lives in backends-spec.md §6.3, not vcv-module.md (which has no
§6.3), and codegen/README.md was already a MAP pointer with no port-solidjs
trigger left to remove.
Beyond the table — found by sweeping every backticked path in the changed docs
against `git ls-files`, which is how these should have been caught before:
manifold/ONBOARDING.md documented a UI that Phase 1 deleted, as if current:
SplitStage, ReadoutStrip, InputMini, BackendAdvanced, AltitudeNav, and a
shot.spec.ts that does not exist. The whole stage table was keyed on a `focus`
axis that no longer exists — selection is now sandwich > particles >
composite. This matters more than the rest: CLAUDE.md tells every agent to
read ONBOARDING.md first for Manifold work, so it was actively teaching a
fiction. Rewritten against ConsoleApp.tsx.
MAP.md claimed the input layer reduces axes to the engine arity with an
"even/odd blend". input-layer.ts says the opposite in its own header: one
dedicated slot per axis, 1:1, into a 32-input over-provisioned head, and
mean-blending was removed deliberately because it diluted every source.
AGENT-REFERENCE.md still promised TS emission "returns at P5" (landed),
per-mode dims "become schema-real at P5" (landed at P5.3), and pointed at
nisps::FixedBuffer (deleted).
Doc-right/code-suspect, filed rather than fixed: VCV computes derivedMean/Std/
Delta and cachedNovelty behind a live context-menu toggle that nothing reads;
vcv/plugin.json points at the MusicallyEmbodiedML org rather than this repo's
origin; and the module defaults to UDP 7001+id%64 while bridge.ts defaults to
9000, so out of the box they do not meet.
Firmware-build docs are deliberately untouched — the PlatformIO migration
lands next and rewrites all of them.
The operator's call: "there should be one default learning rate and one
default max iterations and they should both be configurable at runtime."
There were SIX copies, not the four the audit described, and they did not
agree:
nisps/ml/mlp.hpp no-arg train() hardcoding 1.f / 1000u / 0.001f —
and firmware's ONLY training path calls exactly
this, so firmware had no runtime knob at all
wasm-iml.ts train() and trainAsync() TS default params (x2)
engine-api.ts learningRate ?? 1.0, with no maxIterations knob
vcv/src/iml.hpp 200 / 0.1 / 0.00001 — silently divergent
external_synth_midi.hpp its own kDefaultLearningRate/kDefaultMaxIterations
schemas/modes/*.json x9, identical, read by nobody at runtime
Now: schemas/ml_defaults.json is the single declaration (validated against a
sibling meta-schema, matching the midi_device.schema.json convention), codegen
emits it to C++ and TS in the same run, and MLPCore carries a TrainConfig whose
default member initialisers read the generated constant.
set_train_config()/nisps_ml_set_train_config() make it runtime-overridable on
every target; the explicit-argument train() overload is untouched. min_error
joins the tuple — it was duplicated identically and belongs with the other two.
The per-mode ml block loses default_learning_rate/default_max_iterations.
default_spread stays (genuinely wired on both targets) and input_channels stays
(codegen-time validated, real information for sound_analysis_midi).
VCV BEHAVIOUR CHANGE, deliberate: MEMLNaut.cpp constructs IML positionally and
relies on those defaults, so the module moves to 1000/1.0/0.001 — 5x the max
iterations, 10x the learning rate, and a 100x looser early-stop threshold. The
old values were never justified anywhere; they arrived with fbc68eb alongside
an unrelated module rewrite and no tuning rationale. Firmware and WASM have
shipped 1.0/1000 all along. It is now runtime-settable if this turns out worse.
The generated header lands in nisps/ml/generated/, not nisps/modes/generated/
where the rest of codegen output lives: training hyperparameters are an ML
fact, and nisps/ml sits below nisps/modes, so emitting them there would make
mlp.hpp include upward. The agent that built this flagged the directory-crossing
rather than hiding it; this is the fix. CI's generated-freshness gate learns the
new directory.
Gates: run-all-tests.sh ALL GREEN — 4/4 ctest, parity PASS (max delta 2.38e-7),
lint clean, manifold typecheck + 17 unit + 33 e2e (which exercise train() and
trainAsync() through a real browser).
MAP.md: codegen's generated/ now owns ParamSchema and the per-mode MLP aliases;
new shared headers (dsp/ratio_seq.hpp, dsp/seq_clock.hpp, core/event_queue.hpp
with a note on why it is deliberately not RingBuffer); backends/base-backend.ts;
the seed script and codegen/templates bullets removed; the synth-midi-cc.json
path corrected to its sources/ subdir.
Plan §4 burned down. S26 is NOT done — it is now an operator decision with the
per-field inventory that makes it cheap, recorded in the plan: default_spread is
already wired (the audit's "zero consumers" was a quarter wrong);
default_learning_rate/default_max_iterations are unread but every schema carries
exactly the values already hardcoded, so wiring them is numerically a no-op
today; input_channels is codegen-time-validated and carries real meaning for
sound_analysis_midi; and per-param curve is a trap — it is descriptive of
squaring the engines already do internally, so wiring it would double-apply on
35 params.
The three top-level orienting docs are each falsified by several of the eight
preceding commits, so they land here as one sync rather than being split
across commits that would each leave them half-true. Same phase, same push.
MAP.md — removed fixed_buffer.hpp, voice_space.hpp, the src/daisysp entry and
its symlink from the sketch-tree list, input_router.hpp/wire_inputs (inputs are
wired by bind_peripherals now), test_fixed_buffer.cpp, SplitStage/ReadoutStrip/
InputMini, BackendAdvanced.tsx, and feedback/rng.ts; corrected the primitives
count 12 -> 7; replaced the "engine LIFTED from playground/src" provenance;
rewrote the perf-attribute convention and deleted the NISPS_AUDIO_FUNC gotcha
(both macros and gotcha are gone).
docs/AGENT-REFERENCE.md — the memory-section-attribute instruction now names
only the macros that exist; input_router.hpp dropped from the firmware tree;
symlink list and the submodule-init note no longer mention daisysp.
ALIGNMENT.md — defect 5 (dead mass and registry sprawl) rewritten: the deletion
half is done, so the entry now scopes to what actually remains, which is the
registry/dual-truth half (Phase 3) plus the stale specs (docs pass). Per the
ALIGNMENT convention this is a rewrite-to-current, not a checkbox.
Phase 0 of the 2026-07 simplification audit (plan §1). CI has been 100% red on
main since 2026-07-13 and every "gates green" claim since rested on local runs.
- S7 / critic gap 2: push memllib `feat/nisps-core-swap` (3 commits incl. the
pin b37fc53) to monkey-w1n5t0n/memllib and repoint .gitmodules at the fork.
Those commits existed on exactly one disk; `git ls-remote` now resolves the
pin, so `submodules: recursive` checkout and fresh clones work again. Drops
the compensating unreachable-pin error paragraph in build-firmware-arch.sh.
- S24 / S31: the manifold-tests job regenerates from schemas/, runs the codegen
golden test, and fails on a dirty diff — the "schema changes ship with both
generated outputs" rule is now enforced rather than assumed.
- S32: a WASM freshness gate runs the parity harness against the *committed*
manifold/public/nisps.{js,wasm} before the CI rebuild overwrites it. That
artifact is what the webhook ships to production, so a stale commit now fails
loudly instead of shipping.
- critic gap 3 / operator decision §7.4: the VPS webhook
(~/.config/webhooks/meml-deploy.sh, not in this repo) waits for the `CI`
workflow to conclude success on the pushed SHA before building. Fail-closed;
MEML_SKIP_CI_GATE=1 for an emergency hand-deploy. Verified the gate query
returns `failure` for fa37047, i.e. it would have blocked that deploy.
- S31: corrected run-all-tests.sh's false "single command CI invokes" header.
Docs moved with the code: ALIGNMENT defect 1 deleted (resolved) and the rest
renumbered; MAP.md's unreachable-pin warning replaced with the fork pin and a
pointer to the §7.5 vendoring decision; ONBOARDING documents the deploy gate
and the tracked-WASM-ships-to-prod hazard; plan §1 marked burned down.
Gates: scripts/run-all-tests.sh ALL GREEN (ctest 4/4, parity 1273 floats within
1e-5, lint, typecheck, 33 Playwright specs).
Replace the vendored runtime MLP in vcv/src/iml.hpp (DetRng + 3D-weight-store
MLP + Dataset + IML) with a THIN, Rack-free adapter over the shared core:
nisps::ml::MLPCore<nisps::ml::DynamicStorage> (8->[16,24,16]->16, the P2 dynamic
case), nisps::Rng, and the core MLP's own FIFO dataset. Behaviour changes from
the vendored approximation to core-exact firmware/WASM semantics.
- MEMLNaut.cpp: staged/pending weight buffers and patch JSON now use the core's
flat [weights..][biases..] vector (nisps::IML<float>::Weights); patch version
bumped to 3. Double-buffer / single-writer threading discipline unchanged.
- New ctest tests/cpp/test_vcv_iml_parity.cpp: seeded train/infer/move_weights
session through the adapter is memcmp-equal to a bare MLPCore<DynamicStorage>.
- Docs: vcv-module.md delta #5 marked CLOSED (2026-07-18); MAP.md vcv/ updated.
Closes vcv-module.md delta #5.
Schema-backed modes in console/model.ts are now DERIVED from the codegen
schemas in src/modes/generated/ (source of truth): real param names/groups/
count, plus each mode's ml net shape (MFMode.ml) and schema engine_id. A thin
manifold OVERLAY supplies only label/glyph/ModeClass/input/ordering. New
browser-viable modes xiasri + slp_workshop get derived entries; schema-less
visualizer + c15 stay hand-written on DEFAULT_MODE_ML. Schema min/max/default/
label/curve surface as engine-unit metadata (schemaMin/... on MFParam) without
touching the 0..1 routing semantics.
Switching instrument mode reshapes the runtime-shaped WASM net to the mode's
schema ml config (ConsoleApp effect keyed on [engine, modeId]; no confirm
modal). Boot lands paf_synth dims (4->[10,10,14]->33) once WASM is ready. The
P2.3 axis-count reshape offer still reads the engine's live inputSize and does
not spuriously prompt on a mode switch.
Adds schema-modes.spec.ts (P5 gate): drives switches via a new window.__mf
debug seam and asserts describe() dims, getWeights count, output length/bounds,
and UI param count FROM the imported schemas; spot-checks trainAsync after a
switch. Updates reshape/probe-api/geo-dislike specs to assert from the boot
mode schema instead of the retired fixed 32/126 shape.
All gates green: typecheck, unit (9), build, e2e (33).
- plan P2 marked landed (hardware timing spot-check deferred to the
chokepoint-B session on physical MEMLNaut)
- ALIGNMENT: defect #3 resolved by the runtime-shaped browser MLP
- MAP/AGENT-REFERENCE/nisps-wasm-README: describe the honoured-dims
create + reshape ABI; note per-mode dims become schema-real at P5
- fresh wasm artifacts from the gate run
Algorithms (forward, backprop/SGD, init, move_weights, diagnostics) now live
once in MLPCore<Storage> (nisps/ml/mlp.hpp). Storage models:
- FixedStorage (storage.hpp): template-sized std::array, zero heap. The
classic MLP<NIn,H1,H2,H3,NOut,...> is an alias preserving kInput/kHidden*/
kOutput/kNumLayers/weight_count() constexpr — firmware + bindings + modes
compile unchanged.
- DynamicStorage (dynamic_storage.hpp): runtime dims, ONE arena allocation
at construction, nothing per-call. #error under NISPS_TARGET_EMBEDDED
(new macro in core/perf.hpp); sole lint-cpp.sh heap-allowlist entry, plus
a lint check that fails if the #error guard disappears.
Verification:
- new ctest test_mlp_storage_parity: fixed↔dynamic BIT-identical across
init/draw/inference/train(FIFO)/move_weights(pin mask)/eval_loss/
layer_stats/set_weights/infer_batch/reset; invalid+moved-from inert
- golden ML vectors (pre-refactor constants) pass → bit-stable refactor
- native↔WASM parity PASS, max delta unchanged (2.4e-7)
- chokepoint B compile: PAFSynth .text 122324→122692 (+0.30%, ±1% budget);
RAM +416B (eval scratch)
- fix: firmware-common.sh used bare 'python' (absent here) → ${PYTHON:-python3}
Part of one-core-engine-refactor P2. nisps_ml_create ABI untouched (P2.2 is
an operator stop-point).
Add a "grab → move → drop" gesture that moves an existing positive
example's output to a new input position, preserving the output. This is
the new core's home for upstream InterfaceRL's drag-store/reposition-commit,
distinct from Explore→Place (which places newly-auditioned scratchpad sounds).
- nisps/ml/feedback.hpp: begin_reposition()/commit_reposition()/repositioning().
Reuses the Placing state (static_output holds the carried vector) but a
reposition_ flag makes commit AND the mode-switch teardown SKIP the weight
restore — the real net is never set aside here, so restoring snapshot_ would
clobber the live trained net. Guards cancel_place + abort_explore_place.
- firmware glue: state-gate Toggle B. Exploring → reroll/nudge (unchanged);
Idle → MomB1 grab, MomB2 drop (commit + add_example + train). The 4D variant
has no joystick button, so the gesture lives on the momentary toggle. Also
fix a stale top-of-file control-map comment that contradicted the bindings.
- tests: 4 reposition cases (hold without snapshot; commit stores carried
output with no restore; mode-switch aborts without clobber; begin-only-Idle).
- MAP.md: document the full ExploreAndPlace lifecycle + reposition wiring.
Audio-hold (carrying the sound audibly during the move) remains the existing
unwired static_output() TODO and affects Explore→Place identically.
Firmware compile unverified (no arduino-cli); host tests + lint pass.
Consolidate docs/slp-workshop/SPEC.md into docs/specs/slp-workshop-
firmware.md so the project has a single SLP-Workshop spec. The unified
spec is now two parts:
- Part I (§1-8, shipped/stable): the mode + Jolt & OU-explore gestures.
- Part II (§9-13, planned): the output-mode evolution — Continuous/Rhythm
stream model, gate sequences, CV/MIDI output config, Manifold UX, and
the locked operator decisions (2026-06-28).
Frontmatter bumped stable -> evolving (the doc now spans shipped +
planned). Donor docs/slp-workshop/SPEC.md removed; README.md row and
MAP.md ## Specs note updated. No code change.
Self-contained stable reference spec crystallizing the SLP-Workshop
firmware: the MEMLCelium-based mode, the two adaptive-learning gestures
ported from upstream InterfaceRL (Jolt held weight-morph, OU output
walk) with exact constants, the runtime-not-compile-time + inert-by-
default design, the ModeBase integration incl. the GCC -Wstringop-
overflow workaround, control mappings, schema/codegen, and the
browser-parity caveat. Adds the spec to docs/specs/README.md and a
## Specs section to MAP.md per the specs-skill config format.
Refs commits 4e60d01, 57c9ede (merged at 527b8fc).
Restores the April-2026 "uSEQ-Celium" functionality (browser → uSEQ
hardware + CV expander over USB Web Serial) as a first-class Manifold
Outputs backend, and re-vendors the RP2040 firmware into the repo.
- protocol v2 (uSEQ-CV): firmware/useq-celium/shared/protocol.h is the
single source of truth, mirrored by manifold/src/backends/useq-protocol.ts.
26-byte OUTPUT frame, 11×u16 CV (12-bit) + 3-gate bitfield + XOR; fixed
topology; host-agnostic so the MEMLNaut RP2350 can emit identical bytes.
Spec in docs/useq-celium/protocol.md.
- firmware/useq-celium/{main,expander}: PlatformIO RP2040 firmware rewritten
to v2 from the real April pin maps (expander I2C addr 0x10).
- UseqCvBackend (id cvgate): Web Serial connect/identify/disconnect, 100 Hz
stream, per-channel dead-zone, gate thresholding; modeled on midi-backend.
Per-output CvSpec (channel + gateThreshold) on MFParam; config UI in
OutputsBackendConfig + BackendAdvanced; new "CV / uSEQ" top-dock mode.
- bun-test for the protocol frame layout; MAP.md updated.
New SLP-Workshop firmware variant (Synth Library Portland), built on the
MEMLCelium engine + MLP shape. Ports the two post-fork learning-algorithm
changes from upstream memllib InterfaceRL into the shared nisps/ml core,
runtime-configurable (no compile-time switch), inert by default:
- nisps/ml/jolt.hpp: Jolt — held continuous weight morph over the flat
weight buffer + post-release LR ramp (kJolt* constants verbatim).
- nisps/ml/ou_noise.hpp: OUNoise<N> — Ornstein-Uhlenbeck exploration walk
on the output vector (theta=0.02, dt=0.001, kMaxAmplitude=0.65).
Both wired into ModeBase so every mode gains jolt_press/jolt_release/
jolt_lr_scale + set_explore_intensity; gated so existing modes stay
bit-identical (parity + golden tests green). Firmware surfaces them on
TogB1 (Jolt) and RVX1 (explore). New SLPWorkshopMode mode + schema +
codegen; firmware alias + .ino variant; playground mode registration.
Tests: jolt + OU unit tests, ModeBase learning integration incl. an
inert-parity test proving SLP-Workshop == MEMLCelium with features off.
Verified: cpp tests, wasm build, native↔wasm parity, lint, codegen
golden, playground typecheck. Firmware compile/e2e/hardware are
environment-bound (no arduino-cli/submodules/browser here).
Refs ergo 019f0fca.
Add glue/settings_view.hpp: wire_settings(mode) registers a "Joystick"
SingleSelectView on the MEMLNaut display carousel (TFT + rotary encoder, same
DisplayDriver as SystemView/SelfTest). For the 4-input two-joystick modes,
"Single" pins ML input channels 2,3 (the second joystick) to neutral via
ModeBase::set_input_pinned — the network is never rebuilt and trained state
survives toggling. Default is Dual. Only registered when input arity == 4.
Called from MEMLNaut-NISPS.ino setup() after addSystemInfoView(). MAP.md +
CLAUDE.md glue listings updated.
NOTE: compile-unverified — no arduino-cli/RP2350 toolchain on this host and no
hardware; needs scripts/build-firmware.sh + a flash test (chokepoint A).
Wire the modular input layer into the Console and reshape the browser
engine so input axes are genuine independent dimensions.
Inputs (manifold/src/inputs/):
- gamepad-source: emit press+release edges with standard-mapping labels
(enables hold-and-move); single/double-stick already present.
- midi-input-source: single-device selection + batch "MIDI Learn"
(every CC swept while armed becomes an axis); notes stay discrete.
- input-layer: compose() forwards each axis 1:1 (no mean-blend);
add onReducedInput so the manifold tracks gamepad/MIDI position.
- types: InputAction.phase, InputMode.
Console (manifold/src/console/):
- ConsoleApp: bind gamepad buttons to verdicts (RB up / LB down /
X randomise / Y nudge / B undo / A-hold reposition); mirror composed
position onto the manifold.
- Drawers: rebuilt Inputs drawer (source picker, gamepad legend, MIDI
device picker + batch-learn flow, learned-control meters).
Engine (nisps/wasm, manifold/src/engine):
- DefaultMLP widened MLP<2,..> -> MLP<32,..> (32 = MAX_AXES); each
active axis gets a dedicated slot, unused slots held at 0 (inert).
Rebuilt nisps.wasm (playground + manifold).
- spine/engine-api: setInputs writes the full N-D vector (was dropping
arr[2+]); primary pair keeps the 2-D pipeline; process() re-ticks the
whole vector via spine.reprocess().
Tests:
- parity_check/parity_wasm: ParityMLP -> 32 inputs, widen example bufs.
- CMakeLists: build parity binary with -ffp-contract=off so native
matches FMA-free WASM (training amplified the gap past 1e-5).
Inputs dock is still an exclusive picker; mixing toggles, reshape modal,
and the >2-D slider view (inputs-spec.md) are groundwork-laid but not
yet wired. See docs/redesign/midi-gamepad-inputs-worklog.md.
Joystick -> MLP -> MIDI CC for an external hardware synth, using the
compile-time device templates from nisps/midi/generated. ExternalSynthMIDIMode
<const MidiDevice&, NOut> mirrors SoundAnalysisMIDIMode (NoOpEngine,
kRouteOutputsToEngine=false, pushes ControlChange events). A consteval
pick_cc_slots curates which NOut params fill the output slots (prefers musical
params over Bank Select/global). Adds six flashable variants
(MEMLNautModeExtSynth{Sub37,SubPhatty,Pro12,AnalogKeys,Hydrasynth,JD800}) wired
into mode_select.hpp + the .ino variant list + NISPS_ST guards, and a src/nisps/
midi symlink so the sketch tree can reach nisps/midi.
Host-compile-verified under C++20 (incl. via the sketch include path) and
lint-clean; full arduino-cli build + flash is on the hardware (no toolchain here.
Add a durable, committed source of truth for external MIDI synth control:
- synth-midi-cc.json: verified CC maps + provenance/sources (8 devices researched)
- schemas/midi_device.schema.json + schemas/midi_devices/*.json: 6 CC-controllable
device templates (Moog Sub 37/Sub Phatty, Creamware Pro-12 ASB, Elektron Analog
Keys, ASM Hydrasynth, Roland JD-800), params keyed {id, cc, label, min, max,
default, group}.
- codegen/generate-midi-devices.ts (isolated from the mode golden test) emits both
nisps/midi/generated/midi_devices.hpp (no-heap constexpr, firmware+WASM) and
manifold/src/midi-devices/generated/ (typed catalogue for the browser).
- codegen/seed-midi-devices.ts: reproducible seed from the research artifact.
Lets a performer pick a device and address its parameters by name (not CC number)
on both the firmware and the Manifold browser engine.
Move the Arduino sketch into firmware/MEMLNaut-NISPS/ and bridge the
hardware (memllib) to the platform-agnostic nisps/ library through a
slim glue layer. Delete the legacy root-level *AudioApp.hpp,
modes/MEMLNautMode*.hpp, voicespaces/, IMLInterface.hpp, XiasriAnalysis,
and the src/memlp submodule.
Glue layout (firmware/MEMLNaut-NISPS/glue/):
audio_driver.hpp - bridge memllib block callback to Mode::process
via per-Mode templated trampoline (no virtual dispatch)
peripherals.hpp - joystick/pots/buttons -> Mode::set_input + ML primitives
midi_io.hpp - MIDI in -> mode.note_on/update_bpm/set_playing,
drains mode ControlEvent ring -> MIDI UART
mode_select.hpp - using-aliases mapping MEMLNautMode<Name> to
nisps::modes::*Mode (build script rewrites the
#define MEMLNAUT_MODE_TYPE line)
input_router.hpp / output_router.hpp - top-level wire/drain entry points
The sketch tree uses src/{memllib,daisysp,nisps} symlinks because
Arduino-CLI rejects ".." in include paths from sketch-tree headers.
mode_select.hpp #undefs Arduino's sq/min/max/abs/round macros before
including nisps headers (some nisps engines use those identifiers as
method names). The audio bridge struct is extern in the header and
defined in the .ino because inline + __not_in_flash section attribute
collide at link time.
Verification: arduino-cli compile succeeds for PAFSynth, ChannelStrip,
and BreakOr (rp2040:rp2040:solderparty_rp2350_stamp_xl:opt=Optimize3,
-std=gnu++20). Host C++ tests under nisps/build still pass (3 binaries,
110+ tests). Build script (scripts/build-firmware.sh) updated to point
at the new sketch path; mode-rewrite logic unchanged.
Closes meml-gkm.
