ci: restore verification — reachable submodule pin, codegen + WASM freshness gates
Phase 0 of the 2026-07 simplification audit (plan §1). CI has been 100% red on
main since 2026-07-13 and every "gates green" claim since rested on local runs.
- S7 / critic gap 2: push memllib `feat/nisps-core-swap` (3 commits incl. the
pin b37fc53) to monkey-w1n5t0n/memllib and repoint .gitmodules at the fork.
Those commits existed on exactly one disk; `git ls-remote` now resolves the
pin, so `submodules: recursive` checkout and fresh clones work again. Drops
the compensating unreachable-pin error paragraph in build-firmware-arch.sh.
- S24 / S31: the manifold-tests job regenerates from schemas/, runs the codegen
golden test, and fails on a dirty diff — the "schema changes ship with both
generated outputs" rule is now enforced rather than assumed.
- S32: a WASM freshness gate runs the parity harness against the *committed*
manifold/public/nisps.{js,wasm} before the CI rebuild overwrites it. That
artifact is what the webhook ships to production, so a stale commit now fails
loudly instead of shipping.
- critic gap 3 / operator decision §7.4: the VPS webhook
(~/.config/webhooks/meml-deploy.sh, not in this repo) waits for the `CI`
workflow to conclude success on the pushed SHA before building. Fail-closed;
MEML_SKIP_CI_GATE=1 for an emergency hand-deploy. Verified the gate query
returns `failure` for fa37047, i.e. it would have blocked that deploy.
- S31: corrected run-all-tests.sh's false "single command CI invokes" header.
Docs moved with the code: ALIGNMENT defect 1 deleted (resolved) and the rest
renumbered; MAP.md's unreachable-pin warning replaced with the fork pin and a
pointer to the §7.5 vendoring decision; ONBOARDING documents the deploy gate
and the tracked-WASM-ships-to-prod hazard; plan §1 marked burned down.
Gates: scripts/run-all-tests.sh ALL GREEN (ctest 4/4, parity 1273 floats within
1e-5, lint, typecheck, 33 Playwright specs).
This commit is contained in:
monkey-w1n5t0n
parent
fd0aee2354
commit
8c249ea8af
8 changed files with 66 additions and 32 deletions
29
.github/workflows/ci.yml
vendored
29
.github/workflows/ci.yml
vendored
|
|
@ -59,6 +59,16 @@ jobs:
|
|||
CMAKE_BUILD_PARALLEL_LEVEL: '4'
|
||||
run: bash scripts/build-cpp-tests.sh
|
||||
|
||||
# Freshness gate. manifold/public/nisps.{js,wasm} are TRACKED artifacts:
|
||||
# the VPS webhook builds only manifold/, so vite copies whatever is
|
||||
# committed into the production bundle. Prove the committed artifact
|
||||
# still agrees with the C++ we just built, BEFORE the rebuild below
|
||||
# overwrites it. A stale commit fails here instead of silently shipping.
|
||||
- name: WASM freshness gate (committed artifact vs native)
|
||||
env:
|
||||
NISPS_PARITY_NO_BUILD: '1'
|
||||
run: bash scripts/parity-check.sh
|
||||
|
||||
- name: Build WASM
|
||||
env:
|
||||
# The script defaults to /usr/lib/emscripten/emcc; the runner gets
|
||||
|
|
@ -97,6 +107,25 @@ jobs:
|
|||
with:
|
||||
bun-version: latest
|
||||
|
||||
# Enforces the "schema changes ship with regenerated C++ AND TypeScript
|
||||
# in the same change" rule. Regenerate from schemas/ and fail if the
|
||||
# committed outputs differ; then run the codegen golden test.
|
||||
- name: Codegen is committed and idempotent
|
||||
run: |
|
||||
cd codegen
|
||||
bun install --frozen-lockfile
|
||||
bun run generate.ts
|
||||
bun run generate-midi-devices.ts
|
||||
bun run tests/golden_test.ts
|
||||
cd ..
|
||||
GEN_DIRS="nisps/modes/generated nisps/midi/generated \
|
||||
manifold/src/modes/generated manifold/src/midi-devices/generated"
|
||||
git add -N $GEN_DIRS
|
||||
if ! git diff --exit-code -- $GEN_DIRS; then
|
||||
echo "::error::Generated outputs are stale. Run codegen and commit the result." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Install manifold deps
|
||||
working-directory: manifold
|
||||
run: bun install --frozen-lockfile
|
||||
|
|
|
|||
2
.gitmodules
vendored
2
.gitmodules
vendored
|
|
@ -1,3 +1,3 @@
|
|||
[submodule "src/memllib"]
|
||||
path = src/memllib
|
||||
url = https://github.com/MusicallyEmbodiedML/memllib.git
|
||||
url = https://github.com/monkey-w1n5t0n/memllib.git
|
||||
|
|
|
|||
24
ALIGNMENT.md
24
ALIGNMENT.md
|
|
@ -12,15 +12,7 @@ The clean-slate rewrite (2026-04-29) consolidated everything into one C++20 code
|
|||
|
||||
## Top defects (ranked by mission impact)
|
||||
|
||||
### 1. The verification story is void: CI red for a month, unpushed load-bearing commits, ungated deploys (2026-07-21)
|
||||
|
||||
**What.** CI has been 100% failing on main since 2026-07-13: the memllib submodule pin (`b37fc53`, local branch `feat/nisps-core-swap`) is reachable from no remote, so GitHub checkout dies before any gate runs — and those three firmware-critical commits exist only on this one disk. Meanwhile push-to-main deploys straight to `meml.lnfinitemonkeys.org/next/` via webhook with no gate, shipping the *committed* `manifold/public/nisps.wasm`, which has no freshness check. All P4–P6 "gates green" claims rest on local runs.
|
||||
|
||||
**Why it blocks the mission.** "Verifiable without hardware" and "confident agentic changes" are the mission's operating premises; both are currently fiction at the remote/deploy boundary. One disk failure loses firmware-critical code.
|
||||
|
||||
**Rough cost.** Half a day (plan §1): push the branch to the `monkey-w1n5t0n/memllib` fork, repoint `.gitmodules`, confirm green, add codegen + WASM-freshness steps; deploy gating is an operator decision (plan §7.4).
|
||||
|
||||
### 2. The mode layer is not shared: WASM re-orchestrates modes by hand (2026-07-21)
|
||||
### 1. The mode layer is not shared: WASM re-orchestrates modes by hand (2026-07-21)
|
||||
|
||||
**What.** `nisps/modes/` — the CRTP layer binding ML config, engine, voice-space and I/O — compiles only into firmware. `nisps/wasm/bindings.cpp` includes engines and ML primitives but zero mode headers, and Manifold re-assembles mode behaviour (jolt stepping, OU, routing) in TS. "Firmware and WASM share the same modes" is true only at the engine level; every ModeBase behaviour must be mirrored browser-side by hand.
|
||||
|
||||
|
|
@ -28,7 +20,7 @@ The clean-slate rewrite (2026-04-29) consolidated everything into one C++20 code
|
|||
|
||||
**Rough cost.** Spec first, then ~a week: storage-policy the ModeBase orchestration the way P2 did MLPCore (verified shape in plan §6.5a — *not* binding monolithic mode objects, which would contradict the locked two-instance RT architecture). Related honesty gap: Manifold currently catalogues 4 modes that structurally cannot run in the browser (no mic input, event-only engines) — plan §6.5b (absorbs the old C15/mic-input defect; C15 itself lives on `archive/playground-solidjs`).
|
||||
|
||||
### 3. No curated/advanced split and no in-UI mode picker — the UI fights vision 3 (2026-07-21)
|
||||
### 2. No curated/advanced split and no in-UI mode picker — the UI fights vision 3 (2026-07-21)
|
||||
|
||||
**What.** Manifold is 100% dev-maximalist: five drawers of everything, no preset data model to author against, and mode switching exists only via the debug hook — there is no instrument picker in the UI at all (the plumbing, `ctx.modes`/`setModeId`, already exists unused). A stratum of decorative controls (training-param sliders, master volume, bpm, A/B, snapshots, fabricated gradient health) renders real-looking UI that drives nothing.
|
||||
|
||||
|
|
@ -36,7 +28,7 @@ The clean-slate rewrite (2026-04-29) consolidated everything into one C++20 code
|
|||
|
||||
**Rough cost.** Product-model decision first (plan §7.6), then incremental: picker is days; the curated-preset model seeds from `backends/presets.ts` + schemas; disclosure via per-drawer depth levels. Deleting the decorative stratum is part of the Phase-1 sweep.
|
||||
|
||||
### 4. Arduino-CLI build machinery is actively hostile — vision 4 unstarted (2026-07-21)
|
||||
### 3. Arduino-CLI build machinery is actively hostile — vision 4 unstarted (2026-07-21)
|
||||
|
||||
**What.** The build script sed-mutates the committed `.ino` to select variants (polluting history), the mode list is triple-bookkept (a `NISPS_ST_*` token-paste table is already silently missing the currently-active SLPWorkshop variant), a symlink forest works around Arduino's include rules, and the toolchain globally mutates the installed TFT_eSPI library. Firmware compilation is in no automated gate anywhere.
|
||||
|
||||
|
|
@ -44,7 +36,7 @@ The clean-slate rewrite (2026-04-29) consolidated everything into one C++20 code
|
|||
|
||||
**Rough cost.** 2–3 days, one cut (plan §5): env-per-variant `platformio.ini`, delete ~400 lines of hackery, then a firmware CI job. Gated on the memllib ownership decision (plan §7.5).
|
||||
|
||||
### 5. Manifold-as-hardware-editor is a facade (2026-07-21)
|
||||
### 4. Manifold-as-hardware-editor is a facade (2026-07-21)
|
||||
|
||||
**What.** Vision bullet 5 exists as a 237-line Web Serial shell: sound connect lifecycle, zero protocol (`saveModel`/`restoreModel`/`getSettings` are literal stubs), and firmware has no serial command surface or on-device persistence to talk to.
|
||||
|
||||
|
|
@ -52,7 +44,7 @@ The clean-slate rewrite (2026-04-29) consolidated everything into one C++20 code
|
|||
|
||||
**Rough cost.** Week+, spec-first (plan §6.5d). The right discipline already exists in-repo: useq-celium's C-header wire truth + TS mirror + parity test; settings payloads should derive from schema codegen.
|
||||
|
||||
### 6. Dead mass and registry sprawl across every layer (2026-07-21)
|
||||
### 5. Dead mass and registry sprawl across every layer (2026-07-21)
|
||||
|
||||
**What.** The audit's aggregate: ~40% of `feedback.hpp` is legacy modes nothing reaches; a dead four-way focus/altitude UI system; ~25 unconsumed ConsoleCtx fields; a dozen dead WASM API entries threaded through a 5-file registration chain; daisysp compiled into every firmware build with zero consumers; retired-playground artifacts and dead planning relics tracked at root; mode identity spread across ~6 hand-maintained registries with demonstrated drift; assorted stale specs presenting a deleted world as present tense.
|
||||
|
||||
|
|
@ -60,19 +52,19 @@ The clean-slate rewrite (2026-04-29) consolidated everything into one C++20 code
|
|||
|
||||
**Rough cost.** Plan phases 1–3 (~a week total, mostly mechanical deletions with green gates). Behaviour bugs found en route (dataset-cap divergence 100 vs 128 + OOB read, VCV 2-D input truncation, VCV audio-thread race + JSON) are plan §3.
|
||||
|
||||
### 7. No performance measurement despite a performance-defined mission (2026-07-21)
|
||||
### 6. No performance measurement despite a performance-defined mission (2026-07-21)
|
||||
|
||||
**What.** The "super performance-sensitive" constraint is enforced only by static discipline (no-heap lint — itself with proven false negatives — and section attrs, 3/5 of which are dead macros). No benchmark, no CPU-load assertion, no flash/RAM size report on either target; the 16 KB dead buffer was found by reading, not by any gate.
|
||||
|
||||
**Rough cost.** ~A day for a host-side blocks-per-second benchmark + a per-variant size report in `build-firmware.sh` (plan §6.5f).
|
||||
|
||||
### 8. Training-health telemetry: one product decision fragmented into four half-features (2026-07-21)
|
||||
### 7. Training-health telemetry: one product decision fragmented into four half-features (2026-07-21)
|
||||
|
||||
**What.** A 16 KB loss-history buffer in every firmware MLP that nothing reads; a WASM worker faking a 1-element loss history; decorative gradient-health UI; and a real `get_layer_stats` API plumbed end-to-end and consumed by nobody.
|
||||
|
||||
**Why it blocks the mission.** "Is the network learning?" is a core research affordance — currently it *looks* answered while being fake. Decide feature-or-delete once (plan §7.3) and collapse all four limbs accordingly.
|
||||
|
||||
### 9. RMSProp still deferred from `nisps/ml/` (2026-04-29; reaffirmed 2026-07-21)
|
||||
### 8. RMSProp still deferred from `nisps/ml/` (2026-04-29; reaffirmed 2026-07-21)
|
||||
|
||||
**What.** `training.hpp` ships SGD only; the legacy firmware used RMSProp for `TrainBatch`. Optimizer choice is a research axis. Not blocking current fits; will matter for harder loss landscapes. Port target: upstream MusicallyEmbodiedML `memlp` (the in-repo `src/memlp` copy is deleted; use the GitHub remote or archive branch).
|
||||
|
||||
|
|
|
|||
2
MAP.md
2
MAP.md
|
|
@ -122,7 +122,7 @@ includes; no `nisps-core`.
|
|||
- `ci.yml` — GitHub Actions: cmake build + ctest + WASM build + parity check + lint + Playwright (cpp-tests + manifold-tests jobs). Firmware compile is documented as manual.
|
||||
|
||||
### `src/` — submodule + vendored trees
|
||||
- `src/memllib/` — hardware abstraction (audio driver, peripherals, MIDI), the only true submodule. **Not auto-initialized** — fresh clones need `git submodule update --init --recursive`. ⚠ The pinned commit currently lives on no remote (ALIGNMENT defect 1 / plan §1).
|
||||
- `src/memllib/` — hardware abstraction (audio driver, peripherals, MIDI), the only true submodule. **Not auto-initialized** — fresh clones need `git submodule update --init --recursive`. Pinned to `monkey-w1n5t0n/memllib` branch `feat/nisps-core-swap` (the operator's fork; upstream is `MusicallyEmbodiedML/memllib`). Ownership decision — vendor the load-bearing subset into this repo — lands with the PlatformIO migration (plan §5, §7.5).
|
||||
- `src/daisysp/` — vendored plain files (NOT a submodule). Zero remaining consumers — nisps replaced its PitchShifter with a custom granular impl; deletion planned (plan S8).
|
||||
|
||||
### Top-level docs
|
||||
|
|
|
|||
|
|
@ -17,6 +17,14 @@ status: active
|
|||
|
||||
## §1 Phase 0 — Restore verification (URGENT, ~half day) — S7, S24, S31, S32; critic gaps 1–3
|
||||
|
||||
**BURNED DOWN 2026-07-21.** All five items landed: memllib `feat/nisps-core-swap` pushed to
|
||||
`monkey-w1n5t0n/memllib` and `.gitmodules` repointed (pin `b37fc53` now reachable — data-loss risk
|
||||
closed); the compensating error paragraph in `build-firmware-arch.sh` deleted; codegen dirty-diff +
|
||||
golden test added to the `manifold-tests` CI job; a WASM freshness gate (parity harness vs the
|
||||
*committed* artifact) added to `cpp-tests` before the rebuild; and the VPS webhook
|
||||
(`~/.config/webhooks/meml-deploy.sh`, not in this repo) now blocks the deploy unless the `CI`
|
||||
workflow concluded `success` on that exact SHA — fail-closed, `MEML_SKIP_CI_GATE=1` to override.
|
||||
|
||||
CI has been 100% red on main since 2026-07-13; the cause and the data-loss risk are the same object.
|
||||
|
||||
1. **Push `feat/nisps-core-swap` to the `monkey-w1n5t0n/memllib` fork** (3 commits incl. pin `b37fc53` currently existing only on this disk); repoint `.gitmodules` to the fork URL; verify a fresh clone + `submodules: recursive` checkout succeeds; delete the compensating error-message paragraph in `scripts/build-firmware-arch.sh` (S7).
|
||||
|
|
|
|||
|
|
@ -40,9 +40,17 @@ bun run test:e2e # Playwright smoke (needs `bun run build` first; runs again
|
|||
Chromium): `PLAYWRIGHT_BROWSERS_PATH=/home/w1n5t0n/snap/bun-js/87/.cache/ms-playwright node node_modules/.bin/playwright test`. Preview via bun is fine. The smoke spec (`tests/e2e/smoke.spec.ts`)
|
||||
asserts: engine WASM loads, spine invariant (setInputs → outputs change), feedback runs, console
|
||||
renders, **no "C15" in the bundle**, no console errors. `shot.spec.ts` takes screenshots.
|
||||
- **Deploy is automatic on push to GitHub `main`** → webhook → builds `manifold/` → rsyncs to the
|
||||
live `/next/` subdir. See the `manifold-deploy-pipeline` memory for the full chain and gotchas
|
||||
(the `cp index.html a-immersive.html` 403 workaround; git-ignored `bun.lock`).
|
||||
- **Deploy is automatic on push to GitHub `main`, but gated on CI** → webhook → waits for the
|
||||
`CI` workflow to conclude `success` on that exact SHA → builds `manifold/` → rsyncs to the live
|
||||
`/next/` subdir. A red or missing CI run aborts the deploy (fail-closed, 20 min timeout);
|
||||
`MEML_SKIP_CI_GATE=1` bypasses it for an emergency hand-deploy. The gate lives VPS-side in
|
||||
`~/.config/webhooks/meml-deploy.sh` (not in this repo) — added 2026-07-21 per the simplification
|
||||
audit. See the `manifold-deploy-pipeline` memory for the full chain and gotchas (the
|
||||
`cp index.html a-immersive.html` 403 workaround; git-ignored `bun.lock`).
|
||||
- **`manifold/public/nisps.{js,wasm}` are tracked artifacts that ship to production** — the webhook
|
||||
builds only `manifold/`, so vite copies whatever is committed. CI's *WASM freshness gate* runs
|
||||
the parity harness against the committed artifact before rebuilding it, so a stale commit fails
|
||||
loudly. Rebuild with `scripts/build-wasm.sh` and commit it whenever `nisps/` changes.
|
||||
- **`?debug=1`** installs `window.__nisps` (synchronous probe for Playwright/console — see
|
||||
`src/debug/probe.ts`). **`base: './'`** in `vite.config.ts` keeps asset URLs relative so one
|
||||
`dist/` mounts at both `/` and `/next/`. **WASM URLs must resolve via `document.baseURI`**, never
|
||||
|
|
|
|||
|
|
@ -66,17 +66,12 @@ if ! command -v arduino-cli >/dev/null 2>&1; then
|
|||
fi
|
||||
fi
|
||||
|
||||
# ---- 3. submodule sanity (the orphan-pin footgun) -----------------------
|
||||
# memllib provides the audio/synth/hardware tree; if its pin is unreachable the
|
||||
# build cannot be assembled. Fail loudly with the fix rather than 100 cryptic
|
||||
# missing-header errors deep in the compile.
|
||||
# ---- 3. submodule bring-up ----------------------------------------------
|
||||
# memllib provides the audio/synth/hardware tree the sketch compiles against.
|
||||
log "Checking the memllib submodule"
|
||||
if ! git -C "$REPO_ROOT" submodule update --init --recursive 2>/tmp/mf-submod.err; then
|
||||
cat /tmp/mf-submod.err >&2 || true
|
||||
die "git submodule update failed. If this is the 'not our ref' / unreachable-pin error, the recorded memllib commit is on an unpushed branch (feat/nisps-core-swap). Push that memllib branch to a reachable remote, or bump the submodule pointer to a published commit, then re-run."
|
||||
fi
|
||||
git -C "$REPO_ROOT" submodule update --init --recursive
|
||||
if [[ ! -e "$REPO_ROOT/firmware/MEMLNaut-NISPS/src/memllib/hardware/memlnaut" ]]; then
|
||||
die "src/memllib looks empty after submodule init — the memllib pin is unreachable. See the message above."
|
||||
die "src/memllib looks empty after submodule init — check the submodule pin and remote."
|
||||
fi
|
||||
|
||||
# ---- 4. toolchain bring-up (delegates the heavy lifting) ----------------
|
||||
|
|
|
|||
|
|
@ -1,6 +1,8 @@
|
|||
#!/usr/bin/env bash
|
||||
# scripts/run-all-tests.sh — master entrypoint that exercises every check
|
||||
# stream 11 owns. Designed to be the single command CI invokes.
|
||||
# scripts/run-all-tests.sh — master local entrypoint that exercises every
|
||||
# check the repo owns. CI (.github/workflows/ci.yml) does NOT invoke this
|
||||
# script; it re-lists the same stages as discrete steps so failures are
|
||||
# attributable per-step. Keep the two in sync when adding a stage.
|
||||
#
|
||||
# Stages (each fails fast):
|
||||
# 1. C++ build + ctest → scripts/build-cpp-tests.sh
|
||||
|
|
|
|||
Loading…
Reference in a new issue