fix(gates): close the no-heap lint's false negatives and parity's silent FAIL
Phase 2 (S30, L52, L51). These are the gates that are supposed to protect the
core's headline constraints, so the fix is demonstrated rather than asserted.
- S30: lint-cpp.sh's heap audit hardcoded dsp/engines/ml/modes, so nisps/core/
and nisps/pipeline/ — the P4 control-rate hot path — were NEVER scanned. Its
comment handling also post-filtered with `grep -v ' *//'`, which misses
trailing comments. Coverage is now exclusion-based (everything under nisps/
except wasm/, tests/, build/), comments are STRIPPED before matching using the
same perl strip audit_float_suffix already used, and the pattern set is
extended. dynamic_storage.hpp remains deliberately allowlisted — it is a real,
documented heap user that #errors on RP2350.
PROOF (run by me, not just reported): planted `new float[4]` in
nisps/pipeline/input_chain.hpp plus a std::malloc in the trailing-comment form
the old filter skipped.
old lint: "[lint-cpp] clean", exit 0 <- the false negative, live
new lint: FAIL, both lines named, exit 1
Plants removed; lint clean and exit 0 again; tree verified unmodified.
- L52: parity-check.sh's FAIL branch was unreachable — under `set -e` the script
died at the diff command before it could print anything. A mismatch therefore
failed silently. The diff now runs inside the `if` condition, where set -e does
not apply. Verified by forcing NISPS_PARITY_TOL=0: old script exited silently,
new one prints FAIL with the exit code.
- L51: parity_check.cpp and parity_wasm.mjs headers documented 4 stages / blob
v1 against a real 7-stage / v5 implementation. Comment-only rewrite; parity
behaviour unchanged (re-ran the gate: PASS, 1273 floats within 1e-5).
Note for future work: the lint now permanently scans nisps/core/ and
nisps/pipeline/, so changes there are enforced that were not before.
This commit is contained in:
parent
1a78ed9597
commit
8c44a51220
3 changed files with 101 additions and 41 deletions
|
|
@ -10,11 +10,19 @@
|
||||||
# applies to decimal literals consumed at runtime.
|
# applies to decimal literals consumed at runtime.
|
||||||
# Warns; non-zero only if NISPS_LINT_STRICT=1.
|
# Warns; non-zero only if NISPS_LINT_STRICT=1.
|
||||||
#
|
#
|
||||||
# 2. FAIL: heap allocation primitives in audio paths (nisps/dsp/, nisps/engines/,
|
# 2. FAIL: heap allocation primitives anywhere under nisps/ EXCEPT
|
||||||
# nisps/ml/, nisps/modes/). Forbidden patterns:
|
# nisps/wasm/ (the host/browser binding layer, where heap is legitimate).
|
||||||
# - std::vector
|
# Exclusion-based on purpose: a newly added nisps/ subdirectory is
|
||||||
# - bare `new ` / `new(`
|
# scanned by default instead of silently skipped (an older include-list
|
||||||
# - malloc(
|
# missed nisps/pipeline/ and nisps/core/ entirely). Forbidden patterns:
|
||||||
|
# - allocating STL containers (std::vector/string/deque/list/map/set/
|
||||||
|
# unordered_*/function; std::string_view is fine and not matched)
|
||||||
|
# - std::make_unique / std::make_shared
|
||||||
|
# - `new` in any spelling (`new T`, `new(...)`, nothrow, placement)
|
||||||
|
# - C allocators: malloc/calloc/realloc/aligned_alloc/strdup/strndup
|
||||||
|
# Comments and string literals are STRIPPED before matching (incl.
|
||||||
|
# multi-line /* */ blocks), so a real allocation with a trailing comment
|
||||||
|
# cannot hide, and prose mentioning std::vector cannot false-flag.
|
||||||
# Files matching */tests/* are exempt — they are host-only.
|
# Files matching */tests/* are exempt — they are host-only.
|
||||||
# SOLE allowlisted file: nisps/ml/dynamic_storage.hpp — the runtime-shaped
|
# SOLE allowlisted file: nisps/ml/dynamic_storage.hpp — the runtime-shaped
|
||||||
# MLP storage (one arena allocation at construction). It is compile-time
|
# MLP storage (one arena allocation at construction). It is compile-time
|
||||||
|
|
@ -118,18 +126,65 @@ audit_float_suffix() {
|
||||||
# 2. Heap-alloc audit — fail.
|
# 2. Heap-alloc audit — fail.
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
audit_heap_alloc() {
|
audit_heap_alloc() {
|
||||||
local subdirs=("$NISPS_DIR/dsp" "$NISPS_DIR/engines" "$NISPS_DIR/ml" "$NISPS_DIR/modes")
|
# Heap primitives forbidden in the platform-neutral core (see header §2).
|
||||||
local pat='\bstd::vector\b|\bnew[ \t]*\(|\bnew[ \t]+[A-Za-z_]|\bmalloc[ \t]*\('
|
# Perl regex, applied AFTER comment/string stripping below.
|
||||||
local hits
|
local pat='\bstd::(vector|string|deque|list|forward_list|map|multimap|set|multiset|unordered_map|unordered_set|function)\b|\bmake_(unique|shared)\b|\bnew[ \t]*\(|\bnew[ \t]+[A-Za-z_:]|\b(malloc|calloc|realloc|aligned_alloc|strdup|strndup)[ \t]*\('
|
||||||
hits=$(grep -REn "$pat" \
|
|
||||||
--include='*.hpp' --include='*.cpp' \
|
# Everything under nisps/ except the wasm/ binding layer, tests, and
|
||||||
--exclude-dir=build --exclude-dir=tests \
|
# build artifacts. Exclusion-based so new subdirectories are covered by
|
||||||
--exclude='dynamic_storage.hpp' \
|
# default.
|
||||||
"${subdirs[@]}" 2>/dev/null \
|
local files
|
||||||
| grep -v ' *//' \
|
mapfile -t files < <(find "$NISPS_DIR" -type f \( -name '*.hpp' -o -name '*.cpp' \) \
|
||||||
|| true)
|
-not -path '*/wasm/*' \
|
||||||
|
-not -path '*/tests/*' \
|
||||||
|
-not -path '*/build/*' | sort)
|
||||||
|
|
||||||
|
local hits="" file file_hits
|
||||||
|
for file in "${files[@]}"; do
|
||||||
|
# SOLE allowlisted file (see header §2); guarded by the
|
||||||
|
# NISPS_TARGET_EMBEDDED check below.
|
||||||
|
if [[ "$file" == "$NISPS_DIR/ml/dynamic_storage.hpp" ]]; then
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
# Strip string literals, then // and /* */ comments (with cross-line
|
||||||
|
# block-comment state), THEN match — a trailing `// grow buffer`
|
||||||
|
# comment can no longer hide a real allocation on the same line.
|
||||||
|
file_hits=$(NISPS_HEAP_PAT="$pat" perl -ne '
|
||||||
|
my $line = $_;
|
||||||
|
chomp $line;
|
||||||
|
$line =~ s{"(?:[^"\\]|\\.)*"}{""}g; # string literals
|
||||||
|
if ($in_block) {
|
||||||
|
if ($line =~ s{^.*?\*/}{}) { $in_block = 0; } else { next; }
|
||||||
|
}
|
||||||
|
while (1) {
|
||||||
|
my $pl = index($line, "//");
|
||||||
|
my $pb = index($line, "/*");
|
||||||
|
last if $pl < 0 && $pb < 0;
|
||||||
|
if ($pb < 0 || ($pl >= 0 && $pl < $pb)) {
|
||||||
|
$line = substr($line, 0, $pl); # // line comment
|
||||||
|
last;
|
||||||
|
}
|
||||||
|
my $pe = index($line, "*/", $pb + 2);
|
||||||
|
if ($pe < 0) { # /* opens a block
|
||||||
|
$line = substr($line, 0, $pb);
|
||||||
|
$in_block = 1;
|
||||||
|
last;
|
||||||
|
}
|
||||||
|
$line = substr($line, 0, $pb) . " " . substr($line, $pe + 2);
|
||||||
|
}
|
||||||
|
if ($line =~ /$ENV{NISPS_HEAP_PAT}/o) {
|
||||||
|
$line =~ s/^\s+//;
|
||||||
|
print "$ARGV:$.: $line\n";
|
||||||
|
}
|
||||||
|
' "$file" 2>/dev/null || true)
|
||||||
|
if [[ -n "$file_hits" ]]; then
|
||||||
|
hits+="$file_hits"$'\n'
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
hits="${hits%$'\n'}"
|
||||||
|
|
||||||
if [[ -n "$hits" ]]; then
|
if [[ -n "$hits" ]]; then
|
||||||
echo "[lint-cpp] FAIL: heap allocation in audio path:"
|
echo "[lint-cpp] FAIL: heap allocation in nisps/ core:"
|
||||||
echo "$hits" | sed 's/^/ /'
|
echo "$hits" | sed 's/^/ /'
|
||||||
fails=$((fails + 1))
|
fails=$((fails + 1))
|
||||||
fi
|
fi
|
||||||
|
|
|
||||||
|
|
@ -64,12 +64,13 @@ echo "[parity-check] running WASM..."
|
||||||
node "$TESTS_DIR/parity_wasm.mjs" "$WASM_OUT"
|
node "$TESTS_DIR/parity_wasm.mjs" "$WASM_OUT"
|
||||||
|
|
||||||
echo "[parity-check] diffing (tolerance=$TOL)..."
|
echo "[parity-check] diffing (tolerance=$TOL)..."
|
||||||
node "$TESTS_DIR/parity_diff.mjs" "$NATIVE_OUT" "$WASM_OUT" "$TOL"
|
# The node invocation must live in the `if` condition: under `set -e` a bare
|
||||||
status=$?
|
# failing command would kill the script before any FAIL line could print.
|
||||||
|
if node "$TESTS_DIR/parity_diff.mjs" "$NATIVE_OUT" "$WASM_OUT" "$TOL"; then
|
||||||
if [[ $status -eq 0 ]]; then
|
|
||||||
echo "[parity-check] PASS"
|
echo "[parity-check] PASS"
|
||||||
else
|
else
|
||||||
|
status=$?
|
||||||
echo "[parity-check] FAIL (exit=$status)" >&2
|
echo "[parity-check] FAIL (exit=$status)" >&2
|
||||||
|
exit "$status"
|
||||||
fi
|
fi
|
||||||
exit $status
|
exit 0
|
||||||
|
|
|
||||||
|
|
@ -11,32 +11,36 @@
|
||||||
// script `scripts/parity-check.sh` then runs both and float32-diffs the
|
// script `scripts/parity-check.sh` then runs both and float32-diffs the
|
||||||
// outputs with a 1e-5 tolerance.
|
// outputs with a 1e-5 tolerance.
|
||||||
//
|
//
|
||||||
// What we cover
|
// What we cover — seven stages (this file is the AUTHORITATIVE stage list)
|
||||||
// -------------
|
// ------------------------------------------------------------------------
|
||||||
// 1. ML: seed=42, draw_weights(0.5), set_input(0.25, 0.75), process.
|
// The `---- Stage N ----` sections in main() below are the source of truth
|
||||||
// → 126 outputs + 12 weights sampled at known offsets.
|
// for the payload layout; parity_wasm.mjs replays the identical sequence via
|
||||||
// 2. ML training: 3 examples added, train(0.3, 50, 0), capture loss + outputs.
|
// the C ABI, and parity_diff.mjs names the leading payload sections in its
|
||||||
// 3. PAFSynth engine: seed-equivalent setup (params=0.5), 128-sample run on
|
// error context. Summary:
|
||||||
// silence, capture L+R averages.
|
// 1. ML inference: seed=42 (widened as in bindings.cpp), draw_weights(0.5),
|
||||||
// 4. ChannelStrip engine: identical methodology.
|
// process at (0.25, 0.75) → 126 outputs + 12 probed weights (kProbeIdx).
|
||||||
|
// 2. ML training: 3 examples, train(0.3, 50, 0) → 126 outputs + final loss.
|
||||||
|
// 3. PAFSynth engine: params=0.5, 128-sample run on silence → L+R means.
|
||||||
|
// 4. ChannelStrip engine: same methodology on a 0.25 step input.
|
||||||
|
// 5. FeedbackController ("Down Action"): RandomiseOutputs static-output
|
||||||
|
// draws, RandomiseMlp snapshot/restore, and the ExploreAndPlace
|
||||||
|
// lifecycle (explore→reroll→nudge→undo→place→commit).
|
||||||
|
// 6. Geometric dislike (Avoid mode): scripted likes + dislikes drive the
|
||||||
|
// push-away training path → counts, outputs, probed weights.
|
||||||
|
// 7. Pipelines + curves (one-core-engine P4): InputChain (2 configs),
|
||||||
|
// OutputChain (2 configs, freeze mask), and the curve catalog, all on
|
||||||
|
// deterministic rational traces.
|
||||||
//
|
//
|
||||||
// We use the EXACT SAME compile-time MLP architecture as the WASM build:
|
// We use the EXACT SAME compile-time MLP architecture as the WASM build:
|
||||||
// MLP<32, 10, 14, 18, 126> (32-input max for mix-and-match; see bindings.cpp)
|
// MLP<32, 10, 14, 18, 126> (32-input max for mix-and-match; see bindings.cpp)
|
||||||
//
|
//
|
||||||
// Output blob format
|
// Output blob format (v5 — see kVersion below; bump BOTH drivers together)
|
||||||
// ------------------
|
// ------------------------------------------------------------------------
|
||||||
// uint32 magic = 'NPRT' = 0x5450524E
|
// uint32 magic = 'NPRT' = 0x5450524E
|
||||||
// uint32 version = 1
|
// uint32 version = 5
|
||||||
// uint32 n_floats
|
// uint32 n_floats
|
||||||
// float32[n_floats] payload
|
// float32[n_floats] payload — the per-stage pushes in main(), concatenated
|
||||||
//
|
// in order.
|
||||||
// Stable order of payload (concatenated):
|
|
||||||
// * 126 floats: outputs after stage 1 (post-process at (0.25, 0.75))
|
|
||||||
// * 12 floats: weights sampled at fixed indices (see kProbeIdx below)
|
|
||||||
// * 126 floats: outputs after stage 2 (post-train, re-process)
|
|
||||||
// * 1 float : final training loss
|
|
||||||
// * 2 floats: PAFSynth L mean, R mean (over 128 samples)
|
|
||||||
// * 2 floats: ChannelStrip L mean, R mean
|
|
||||||
//
|
//
|
||||||
// Why not bit-perfect
|
// Why not bit-perfect
|
||||||
// -------------------
|
// -------------------
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue