fix(gates): close the no-heap lint's false negatives and parity's silent FAIL
Phase 2 (S30, L52, L51). These are the gates that are supposed to protect the
core's headline constraints, so the fix is demonstrated rather than asserted.
- S30: lint-cpp.sh's heap audit hardcoded dsp/engines/ml/modes, so nisps/core/
and nisps/pipeline/ — the P4 control-rate hot path — were NEVER scanned. Its
comment handling also post-filtered with `grep -v ' *//'`, which misses
trailing comments. Coverage is now exclusion-based (everything under nisps/
except wasm/, tests/, build/), comments are STRIPPED before matching using the
same perl strip audit_float_suffix already used, and the pattern set is
extended. dynamic_storage.hpp remains deliberately allowlisted — it is a real,
documented heap user that #errors on RP2350.
PROOF (run by me, not just reported): planted `new float[4]` in
nisps/pipeline/input_chain.hpp plus a std::malloc in the trailing-comment form
the old filter skipped.
old lint: "[lint-cpp] clean", exit 0 <- the false negative, live
new lint: FAIL, both lines named, exit 1
Plants removed; lint clean and exit 0 again; tree verified unmodified.
- L52: parity-check.sh's FAIL branch was unreachable — under `set -e` the script
died at the diff command before it could print anything. A mismatch therefore
failed silently. The diff now runs inside the `if` condition, where set -e does
not apply. Verified by forcing NISPS_PARITY_TOL=0: old script exited silently,
new one prints FAIL with the exit code.
- L51: parity_check.cpp and parity_wasm.mjs headers documented 4 stages / blob
v1 against a real 7-stage / v5 implementation. Comment-only rewrite; parity
behaviour unchanged (re-ran the gate: PASS, 1273 floats within 1e-5).
Note for future work: the lint now permanently scans nisps/core/ and
nisps/pipeline/, so changes there are enforced that were not before.
This commit is contained in:
monkey-w1n5t0n
parent
1a78ed9597
commit
8c44a51220
3 changed files with 101 additions and 41 deletions
|
|
@ -10,11 +10,19 @@
|
|||
# applies to decimal literals consumed at runtime.
|
||||
# Warns; non-zero only if NISPS_LINT_STRICT=1.
|
||||
#
|
||||
# 2. FAIL: heap allocation primitives in audio paths (nisps/dsp/, nisps/engines/,
|
||||
# nisps/ml/, nisps/modes/). Forbidden patterns:
|
||||
# - std::vector
|
||||
# - bare `new ` / `new(`
|
||||
# - malloc(
|
||||
# 2. FAIL: heap allocation primitives anywhere under nisps/ EXCEPT
|
||||
# nisps/wasm/ (the host/browser binding layer, where heap is legitimate).
|
||||
# Exclusion-based on purpose: a newly added nisps/ subdirectory is
|
||||
# scanned by default instead of silently skipped (an older include-list
|
||||
# missed nisps/pipeline/ and nisps/core/ entirely). Forbidden patterns:
|
||||
# - allocating STL containers (std::vector/string/deque/list/map/set/
|
||||
# unordered_*/function; std::string_view is fine and not matched)
|
||||
# - std::make_unique / std::make_shared
|
||||
# - `new` in any spelling (`new T`, `new(...)`, nothrow, placement)
|
||||
# - C allocators: malloc/calloc/realloc/aligned_alloc/strdup/strndup
|
||||
# Comments and string literals are STRIPPED before matching (incl.
|
||||
# multi-line /* */ blocks), so a real allocation with a trailing comment
|
||||
# cannot hide, and prose mentioning std::vector cannot false-flag.
|
||||
# Files matching */tests/* are exempt — they are host-only.
|
||||
# SOLE allowlisted file: nisps/ml/dynamic_storage.hpp — the runtime-shaped
|
||||
# MLP storage (one arena allocation at construction). It is compile-time
|
||||
|
|
@ -118,18 +126,65 @@ audit_float_suffix() {
|
|||
# 2. Heap-alloc audit — fail.
|
||||
# ---------------------------------------------------------------------------
|
||||
audit_heap_alloc() {
|
||||
local subdirs=("$NISPS_DIR/dsp" "$NISPS_DIR/engines" "$NISPS_DIR/ml" "$NISPS_DIR/modes")
|
||||
local pat='\bstd::vector\b|\bnew[ \t]*\(|\bnew[ \t]+[A-Za-z_]|\bmalloc[ \t]*\('
|
||||
local hits
|
||||
hits=$(grep -REn "$pat" \
|
||||
--include='*.hpp' --include='*.cpp' \
|
||||
--exclude-dir=build --exclude-dir=tests \
|
||||
--exclude='dynamic_storage.hpp' \
|
||||
"${subdirs[@]}" 2>/dev/null \
|
||||
| grep -v ' *//' \
|
||||
|| true)
|
||||
# Heap primitives forbidden in the platform-neutral core (see header §2).
|
||||
# Perl regex, applied AFTER comment/string stripping below.
|
||||
local pat='\bstd::(vector|string|deque|list|forward_list|map|multimap|set|multiset|unordered_map|unordered_set|function)\b|\bmake_(unique|shared)\b|\bnew[ \t]*\(|\bnew[ \t]+[A-Za-z_:]|\b(malloc|calloc|realloc|aligned_alloc|strdup|strndup)[ \t]*\('
|
||||
|
||||
# Everything under nisps/ except the wasm/ binding layer, tests, and
|
||||
# build artifacts. Exclusion-based so new subdirectories are covered by
|
||||
# default.
|
||||
local files
|
||||
mapfile -t files < <(find "$NISPS_DIR" -type f \( -name '*.hpp' -o -name '*.cpp' \) \
|
||||
-not -path '*/wasm/*' \
|
||||
-not -path '*/tests/*' \
|
||||
-not -path '*/build/*' | sort)
|
||||
|
||||
local hits="" file file_hits
|
||||
for file in "${files[@]}"; do
|
||||
# SOLE allowlisted file (see header §2); guarded by the
|
||||
# NISPS_TARGET_EMBEDDED check below.
|
||||
if [[ "$file" == "$NISPS_DIR/ml/dynamic_storage.hpp" ]]; then
|
||||
continue
|
||||
fi
|
||||
# Strip string literals, then // and /* */ comments (with cross-line
|
||||
# block-comment state), THEN match — a trailing `// grow buffer`
|
||||
# comment can no longer hide a real allocation on the same line.
|
||||
file_hits=$(NISPS_HEAP_PAT="$pat" perl -ne '
|
||||
my $line = $_;
|
||||
chomp $line;
|
||||
$line =~ s{"(?:[^"\\]|\\.)*"}{""}g; # string literals
|
||||
if ($in_block) {
|
||||
if ($line =~ s{^.*?\*/}{}) { $in_block = 0; } else { next; }
|
||||
}
|
||||
while (1) {
|
||||
my $pl = index($line, "//");
|
||||
my $pb = index($line, "/*");
|
||||
last if $pl < 0 && $pb < 0;
|
||||
if ($pb < 0 || ($pl >= 0 && $pl < $pb)) {
|
||||
$line = substr($line, 0, $pl); # // line comment
|
||||
last;
|
||||
}
|
||||
my $pe = index($line, "*/", $pb + 2);
|
||||
if ($pe < 0) { # /* opens a block
|
||||
$line = substr($line, 0, $pb);
|
||||
$in_block = 1;
|
||||
last;
|
||||
}
|
||||
$line = substr($line, 0, $pb) . " " . substr($line, $pe + 2);
|
||||
}
|
||||
if ($line =~ /$ENV{NISPS_HEAP_PAT}/o) {
|
||||
$line =~ s/^\s+//;
|
||||
print "$ARGV:$.: $line\n";
|
||||
}
|
||||
' "$file" 2>/dev/null || true)
|
||||
if [[ -n "$file_hits" ]]; then
|
||||
hits+="$file_hits"$'\n'
|
||||
fi
|
||||
done
|
||||
hits="${hits%$'\n'}"
|
||||
|
||||
if [[ -n "$hits" ]]; then
|
||||
echo "[lint-cpp] FAIL: heap allocation in audio path:"
|
||||
echo "[lint-cpp] FAIL: heap allocation in nisps/ core:"
|
||||
echo "$hits" | sed 's/^/ /'
|
||||
fails=$((fails + 1))
|
||||
fi
|
||||
|
|
|
|||
|
|
@ -64,12 +64,13 @@ echo "[parity-check] running WASM..."
|
|||
node "$TESTS_DIR/parity_wasm.mjs" "$WASM_OUT"
|
||||
|
||||
echo "[parity-check] diffing (tolerance=$TOL)..."
|
||||
node "$TESTS_DIR/parity_diff.mjs" "$NATIVE_OUT" "$WASM_OUT" "$TOL"
|
||||
status=$?
|
||||
|
||||
if [[ $status -eq 0 ]]; then
|
||||
# The node invocation must live in the `if` condition: under `set -e` a bare
|
||||
# failing command would kill the script before any FAIL line could print.
|
||||
if node "$TESTS_DIR/parity_diff.mjs" "$NATIVE_OUT" "$WASM_OUT" "$TOL"; then
|
||||
echo "[parity-check] PASS"
|
||||
else
|
||||
status=$?
|
||||
echo "[parity-check] FAIL (exit=$status)" >&2
|
||||
exit "$status"
|
||||
fi
|
||||
exit $status
|
||||
exit 0
|
||||
|
|
|
|||
|
|
@ -11,32 +11,36 @@
|
|||
// script `scripts/parity-check.sh` then runs both and float32-diffs the
|
||||
// outputs with a 1e-5 tolerance.
|
||||
//
|
||||
// What we cover
|
||||
// -------------
|
||||
// 1. ML: seed=42, draw_weights(0.5), set_input(0.25, 0.75), process.
|
||||
// → 126 outputs + 12 weights sampled at known offsets.
|
||||
// 2. ML training: 3 examples added, train(0.3, 50, 0), capture loss + outputs.
|
||||
// 3. PAFSynth engine: seed-equivalent setup (params=0.5), 128-sample run on
|
||||
// silence, capture L+R averages.
|
||||
// 4. ChannelStrip engine: identical methodology.
|
||||
// What we cover — seven stages (this file is the AUTHORITATIVE stage list)
|
||||
// ------------------------------------------------------------------------
|
||||
// The `---- Stage N ----` sections in main() below are the source of truth
|
||||
// for the payload layout; parity_wasm.mjs replays the identical sequence via
|
||||
// the C ABI, and parity_diff.mjs names the leading payload sections in its
|
||||
// error context. Summary:
|
||||
// 1. ML inference: seed=42 (widened as in bindings.cpp), draw_weights(0.5),
|
||||
// process at (0.25, 0.75) → 126 outputs + 12 probed weights (kProbeIdx).
|
||||
// 2. ML training: 3 examples, train(0.3, 50, 0) → 126 outputs + final loss.
|
||||
// 3. PAFSynth engine: params=0.5, 128-sample run on silence → L+R means.
|
||||
// 4. ChannelStrip engine: same methodology on a 0.25 step input.
|
||||
// 5. FeedbackController ("Down Action"): RandomiseOutputs static-output
|
||||
// draws, RandomiseMlp snapshot/restore, and the ExploreAndPlace
|
||||
// lifecycle (explore→reroll→nudge→undo→place→commit).
|
||||
// 6. Geometric dislike (Avoid mode): scripted likes + dislikes drive the
|
||||
// push-away training path → counts, outputs, probed weights.
|
||||
// 7. Pipelines + curves (one-core-engine P4): InputChain (2 configs),
|
||||
// OutputChain (2 configs, freeze mask), and the curve catalog, all on
|
||||
// deterministic rational traces.
|
||||
//
|
||||
// We use the EXACT SAME compile-time MLP architecture as the WASM build:
|
||||
// MLP<32, 10, 14, 18, 126> (32-input max for mix-and-match; see bindings.cpp)
|
||||
//
|
||||
// Output blob format
|
||||
// ------------------
|
||||
// Output blob format (v5 — see kVersion below; bump BOTH drivers together)
|
||||
// ------------------------------------------------------------------------
|
||||
// uint32 magic = 'NPRT' = 0x5450524E
|
||||
// uint32 version = 1
|
||||
// uint32 version = 5
|
||||
// uint32 n_floats
|
||||
// float32[n_floats] payload
|
||||
//
|
||||
// Stable order of payload (concatenated):
|
||||
// * 126 floats: outputs after stage 1 (post-process at (0.25, 0.75))
|
||||
// * 12 floats: weights sampled at fixed indices (see kProbeIdx below)
|
||||
// * 126 floats: outputs after stage 2 (post-train, re-process)
|
||||
// * 1 float : final training loss
|
||||
// * 2 floats: PAFSynth L mean, R mean (over 128 samples)
|
||||
// * 2 floats: ChannelStrip L mean, R mean
|
||||
// float32[n_floats] payload — the per-stage pushes in main(), concatenated
|
||||
// in order.
|
||||
//
|
||||
// Why not bit-perfect
|
||||
// -------------------
|
||||
|
|
|
|||
Loading…
Reference in a new issue