build(firmware): migrate to PlatformIO and vendor memllib (plan §5)
One cut, no dual path. Closes ALIGNMENT defect 3 ("Arduino-CLI build
machinery is actively hostile") and vision bullet 4.
platformio.ini carries 16 [env:], one per variant, each passing
-DMEMLNAUT_MODE_TYPE; selftest passes -DNISPS_SELFTEST=1 instead. The env list
IS the registry now — the .ino comment-registry and the NISPS_ST_* token-paste
table are deleted rather than migrated. L12 noted that table was already
silently missing the currently-shipped SLPWorkshop variant, which is the whole
argument against having a second list.
Also deleted: the Python/sed machinery that rewrote the COMMITTED .ino on every
build, the sketch symlink forest, the global TFT_eSPI User_Setup.h mutation
(now -D flags — TFT_eSPI's own documented PlatformIO recipe), the UF2
boot-mount detection stack (upload_protocol=picotool talks to the bootloader
directly), and build-firmware-arch.sh entirely. Scripts 683 -> 435 lines.
memllib is vendored at lib/memllib/ from upstream e291192; no submodules
remain. VENDORED.md records provenance and the re-sync procedure.
S9: a firmware-build CI job compiles three representative envs against a cached
toolchain and reports per-variant flash/RAM. Firmware is in an automated gate
for the FIRST time. The old ci.yml comment justified excluding it as "low
verification value" — an assessment that did not survive contact, since the
SelfTest variant sat broken for an unknown period calling a DisplayDriver
method that did not exist at the pinned memllib commit, and nothing noticed
because nothing built it.
Verified: all 16 envs build from an empty cache, each within ~520 bytes of the
arduino-cli binary it replaces, flash and RAM. Measured as .text+.rodata /
.data+.bss+vector+uninitialized — NOT PlatformIO's console line, which
double-counts .data on this board. This does not prove the hardware boots; no
flash+smoke test was possible and that stays an operator chokepoint.
slpworkshop 248232/145028 pafsynth 256880/149716 selftest 216228/17960
(all 16 in the CI log format; none exceeds 2% of a 16 MB flash)
Two traps recorded so nobody rediscovers them: vendoring memllib's subdirs
without a src/ wrapper makes PlatformIO's library builder silently compile
NOTHING while still linking; and project build_flags land BEFORE the
framework's own -std=gnu++17 -Os, so build_unflags is required.
CORRECTION carried in this commit: the firmware sizes in c19d846's message and
the first version of the memllib recon doc were wrong — SLPWorkshop 145348,
PAFSynth 145300, SelfTest 141840. They came from building variants in sequence
through a SHARED incremental arduino-cli build directory, which reused stale
objects and under-reported by ~75 KB. Clean-cache rebuilds of the identical
commit give 216736/18492 for SelfTest. The real cost of the memllib upstream
bump is +216 bytes flash, not +316. Never measure firmware size through a
reused build dir.
HISTORY NOTE: this commit and the docs commit before it were rebuilt (force-push,
2026-07-21) so that each contains only what its message describes. The first
versions had the firmware deletions stranded in the docs commit by a shared-index
race between concurrent agents; content is byte-identical to the originals.
Gates: run-all-tests.sh ALL GREEN (nisps/ untouched by this change beyond
include paths); 16/16 pio envs build.
2026-07-21 20:17:58 +02:00
|
|
|
# memllib — vendored, not a submodule
|
|
|
|
|
|
|
|
|
|
This directory is a **vendored copy** of the load-bearing subset of
|
|
|
|
|
[`MusicallyEmbodiedML/memllib`](https://github.com/MusicallyEmbodiedML/memllib), the
|
|
|
|
|
hardware-abstraction library for the MEMLNaut board (audio driver, TFT display, MIDI
|
|
|
|
|
I/O, peripherals). It replaced the `src/memllib` git submodule during the Phase 4
|
|
|
|
|
PlatformIO migration (2026-07-21).
|
|
|
|
|
|
|
|
|
|
## Provenance
|
|
|
|
|
|
|
|
|
|
- **Upstream repo**: `https://github.com/MusicallyEmbodiedML/memllib.git`
|
|
|
|
|
- **Vendored at commit**: `e291192d8e4f2fca7b79670c4df9c2ec8bdf03cd` (upstream `main`,
|
|
|
|
|
"l r input swap")
|
|
|
|
|
- **License**: MPL-2.0 (`LICENSE` in this directory, copied verbatim from upstream)
|
|
|
|
|
|
|
|
|
|
## What was copied, what was dropped
|
|
|
|
|
|
|
|
|
|
Copied verbatim, directory structure unchanged, under `src/`: `audio/`, `hardware/`,
|
|
|
|
|
`interface/`, `synth/`, `utils/`, `PicoDefs.hpp`. `LICENSE` sits at this directory's
|
|
|
|
|
root (metadata, not source). These are exactly the subdirectories the firmware sketch
|
|
|
|
|
used to reach via its symlink forest (`firmware/MEMLNaut-NISPS/src/memllib` before this
|
|
|
|
|
migration) — only the wrapping `src/` folder and the `library.properties` manifest are
|
|
|
|
|
new, both required for PlatformIO to discover and recursively compile this tree (see
|
|
|
|
|
below).
|
|
|
|
|
|
docs(firmware): vendor InterfaceRL as read-only reference
InterfaceRL.{hpp,cpp,tpp} + InterfaceRLFileFormat.hpp are the upstream
reference implementation of the entire NISPS feedback subsystem —
nisps/ml/{geo_push,replay,feedback,jolt,ou_noise}.hpp are all ports of it,
several still carrying `// upstream InterfaceRL.hpp:NNN` line references.
The Phase-4 vendoring dropped examples/ because nothing compiled it. That
was correct for the build and wrong for the codebase: with the source of
truth out of tree, upstream redesigned the geometric dislike (deleted the
/(1+len) taper, doubled kGeometricPushScale, tripled the negative-LR base,
moved to batch training over all negatives every tick) and we did not notice
for months.
Copied verbatim from memllib @ e291192 — the same commit the rest of the
vendored tree pins — into lib/memllib/reference/, which sits OUTSIDE src/
and is therefore never compiled: PlatformIO's LDF only recursively builds an
Arduino-format library's src/ folder. Verified: slpworkshop still builds
(RAM 28.8%, flash 2.1%).
reference/README.md states the two rules (never compiled, never edited — a
divergence from upstream is a recorded decision, not an edit here) and
VENDORED.md's "what was dropped" section now tells the truth.
Resolves ALIGNMENT defect 6c.
2026-07-25 11:14:35 +02:00
|
|
|
Dropped: `examples/` (never compiled; the firmware never referenced it, and its content
|
|
|
|
|
that mattered was already ported into `nisps/ml/{jolt,ou_noise,feedback,geo_push}.hpp`
|
|
|
|
|
per the pre-Phase-4 submodule-bump decision) — **except** `InterfaceRL.{hpp,cpp,tpp}` and
|
|
|
|
|
`InterfaceRLFileFormat.hpp`, which were added back on 2026-07-25 under `reference/`.
|
|
|
|
|
Dropping them was correct for the build and wrong for the codebase: `InterfaceRL` is the
|
|
|
|
|
source of truth for the whole feedback subsystem we ported, and with it out of tree we
|
|
|
|
|
missed upstream's redesign of the geometric dislike for months. `reference/` sits outside
|
|
|
|
|
`src/`, so PlatformIO does not compile it; see `reference/README.md`. Also dropped:
|
|
|
|
|
`.git` (submodule gitlink),
|
build(firmware): migrate to PlatformIO and vendor memllib (plan §5)
One cut, no dual path. Closes ALIGNMENT defect 3 ("Arduino-CLI build
machinery is actively hostile") and vision bullet 4.
platformio.ini carries 16 [env:], one per variant, each passing
-DMEMLNAUT_MODE_TYPE; selftest passes -DNISPS_SELFTEST=1 instead. The env list
IS the registry now — the .ino comment-registry and the NISPS_ST_* token-paste
table are deleted rather than migrated. L12 noted that table was already
silently missing the currently-shipped SLPWorkshop variant, which is the whole
argument against having a second list.
Also deleted: the Python/sed machinery that rewrote the COMMITTED .ino on every
build, the sketch symlink forest, the global TFT_eSPI User_Setup.h mutation
(now -D flags — TFT_eSPI's own documented PlatformIO recipe), the UF2
boot-mount detection stack (upload_protocol=picotool talks to the bootloader
directly), and build-firmware-arch.sh entirely. Scripts 683 -> 435 lines.
memllib is vendored at lib/memllib/ from upstream e291192; no submodules
remain. VENDORED.md records provenance and the re-sync procedure.
S9: a firmware-build CI job compiles three representative envs against a cached
toolchain and reports per-variant flash/RAM. Firmware is in an automated gate
for the FIRST time. The old ci.yml comment justified excluding it as "low
verification value" — an assessment that did not survive contact, since the
SelfTest variant sat broken for an unknown period calling a DisplayDriver
method that did not exist at the pinned memllib commit, and nothing noticed
because nothing built it.
Verified: all 16 envs build from an empty cache, each within ~520 bytes of the
arduino-cli binary it replaces, flash and RAM. Measured as .text+.rodata /
.data+.bss+vector+uninitialized — NOT PlatformIO's console line, which
double-counts .data on this board. This does not prove the hardware boots; no
flash+smoke test was possible and that stays an operator chokepoint.
slpworkshop 248232/145028 pafsynth 256880/149716 selftest 216228/17960
(all 16 in the CI log format; none exceeds 2% of a 16 MB flash)
Two traps recorded so nobody rediscovers them: vendoring memllib's subdirs
without a src/ wrapper makes PlatformIO's library builder silently compile
NOTHING while still linking; and project build_flags land BEFORE the
framework's own -std=gnu++17 -Os, so build_unflags is required.
CORRECTION carried in this commit: the firmware sizes in c19d846's message and
the first version of the memllib recon doc were wrong — SLPWorkshop 145348,
PAFSynth 145300, SelfTest 141840. They came from building variants in sequence
through a SHARED incremental arduino-cli build directory, which reused stale
objects and under-reported by ~75 KB. Clean-cache rebuilds of the identical
commit give 216736/18492 for SelfTest. The real cost of the memllib upstream
bump is +216 bytes flash, not +316. Never measure firmware size through a
reused build dir.
HISTORY NOTE: this commit and the docs commit before it were rebuilt (force-push,
2026-07-21) so that each contains only what its message describes. The first
versions had the firmware deletions stranded in the docs commit by a shared-index
race between concurrent agents; content is byte-identical to the originals.
Gates: run-all-tests.sh ALL GREEN (nisps/ untouched by this change beyond
include paths); 16/16 pio envs build.
2026-07-21 20:17:58 +02:00
|
|
|
`.gitignore` (build-artifact patterns, meaningless once vendored — this repo's own
|
|
|
|
|
`.gitignore` covers it), `README.md` (described the old Arduino-IDE TFT_eSPI
|
|
|
|
|
`User_Setup_Select.h` copy-paste workflow, which PlatformIO replaces with
|
|
|
|
|
`-D USER_SETUP_LOADED=1` + explicit build flags in `platformio.ini` — see there).
|
|
|
|
|
|
|
|
|
|
98 files, ~1.9 MB total — all of memllib bar `examples/`; there is no smaller subset to
|
|
|
|
|
lift (every one of the 24 `.cpp` translation units here is reached by at least one
|
|
|
|
|
compiled firmware variant).
|
|
|
|
|
|
|
|
|
|
## Internal include convention (do not break)
|
|
|
|
|
|
|
|
|
|
Files inside this tree include each other with paths relative to `src/` as the root
|
|
|
|
|
(e.g. `src/hardware/memlnaut/MEMLNaut.cpp` does `#include "../PicoDefs.hpp"`,
|
|
|
|
|
`src/hardware/memlnaut/display/View.cpp` does `#include "../../PicoDefs.hpp"`).
|
|
|
|
|
|
|
|
|
|
This directory is consumed as a PlatformIO Arduino-format library (`lib/memllib/`, with
|
|
|
|
|
`library.properties` + a `src/` subfolder — the standard 1.5 Arduino library layout).
|
|
|
|
|
PlatformIO's Library Dependency Finder therefore adds `lib/memllib/src` (not
|
|
|
|
|
`lib/memllib` itself) to the include search path and recursively compiles every source
|
|
|
|
|
file under `src/`.
|
|
|
|
|
|
|
|
|
|
**Do not vendor these five subdirectories directly under `lib/memllib/`** (i.e. without
|
|
|
|
|
the `src/` wrapper) — that was tried first and silently compiles nothing: PlatformIO's
|
|
|
|
|
`ArduinoLibBuilder`, when it finds no `src/` subfolder, falls back to a *non-recursive*
|
|
|
|
|
"files directly in the library root" scan (the historical Arduino 1.0 library format,
|
|
|
|
|
which only special-cases a `utility/` subfolder). Nested folders like `audio/` or
|
|
|
|
|
`hardware/` are silently invisible to the build under that fallback — it links, or
|
|
|
|
|
rather fails to link, with `undefined reference to MEMLNaut::...` for every symbol in
|
|
|
|
|
this library. The `src/` subfolder switches PlatformIO onto the recursive path.
|
|
|
|
|
|
|
|
|
|
Firmware code outside this tree (`../../src/main.cpp`, `../../glue/*.hpp`) includes
|
|
|
|
|
headers here relative to `src/` as the root, e.g. `#include "audio/AudioDriver.hpp"`,
|
|
|
|
|
`#include "hardware/memlnaut/MEMLNaut.hpp"` — no `memllib/` or `src/` prefix, because
|
|
|
|
|
`lib/memllib/src/` *is* the include root PlatformIO adds.
|
|
|
|
|
|
|
|
|
|
## Re-syncing with upstream
|
|
|
|
|
|
|
|
|
|
There is no submodule to bump anymore, so a re-sync is a manual, documented diff:
|
|
|
|
|
|
|
|
|
|
1. Clone upstream at the desired commit: `git clone
|
|
|
|
|
https://github.com/MusicallyEmbodiedML/memllib.git /tmp/memllib-upstream`
|
|
|
|
|
2. Diff the five subdirs + `PicoDefs.hpp` against this directory's `src/`, e.g.:
|
|
|
|
|
```
|
|
|
|
|
diff -ru /tmp/memllib-upstream/audio firmware/MEMLNaut-NISPS/lib/memllib/src/audio
|
|
|
|
|
# ...repeat for hardware/ interface/ synth/ utils/ PicoDefs.hpp
|
|
|
|
|
```
|
|
|
|
|
3. Copy over the changed files (`cp -a`), re-run `diff -ru` both ways to confirm nothing
|
|
|
|
|
outside the tracked subset leaked in and nothing was silently dropped.
|
|
|
|
|
4. Update the "Vendored at commit" line above to the new upstream SHA + its subject
|
|
|
|
|
line.
|
|
|
|
|
5. Rebuild every `platformio.ini` env (`pio run`) and diff flash/RAM sizes against the
|
|
|
|
|
previous vendored commit's numbers — a size jump with no corresponding upstream
|
|
|
|
|
feature is a signal something unexpected changed.
|
|
|
|
|
6. Commit the vendored-file changes and this doc update together.
|
|
|
|
|
|
|
|
|
|
If upstream ever restructures these directories (renames, new cross-subdir relative
|
|
|
|
|
includes), the internal-include convention above may need re-verification — grep for
|
|
|
|
|
`#include "\.\./` inside this tree and confirm every relative path still resolves.
|