w1n5t0n/memlnaut-nisps
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat(playground/wasm): WASM bridge between C++ core and SolidJS playground (meml-tgm)

Browse source 
Stream 7 wires nisps/ml + nisps/engines into the playground via Emscripten.

Highlights:
- nisps/wasm/bindings.cpp: flat C API per architecture.md §6.2. Fixed-arch
  MLP<2, 10, 14, 18, 126>; engine string→type dispatch table with NoOp
  fallback.
- scripts/build-wasm.sh: emcc invocation, MODULARIZE=1, exports listed
  explicitly; produces playground/public/nisps.{js,wasm}.
- playground/src/ml/wasm-iml.ts: main-thread MLP host (sync inference,
  sync training, RL ops, weights I/O, layer stats, localStorage).
- playground/src/ml/wasm-worker.ts: disposable Web Worker for off-thread
  async training, owns its own WASM instance.
- playground/src/ml/dataset.ts: Float32Array-backed FIFO with sample-weight
  modes (uniform/global/local/combined). Port of legacy dataset.js.
- playground/src/audio/engine-host.ts: AudioContext + AudioWorkletNode
  lifecycle, with start/stop/setEngine/setParams.
- playground/src/audio/worklet/nisps-processor.ts: WASM-loading
  AudioWorkletProcessor that runs engine.process_block per 128-sample
  block. Loads its own WASM instance from main-thread-supplied bytes
  (no fetch in worklet).
- playground/src/stores/ml-store.ts: wired stub methods to WasmIML
  singleton; lazy initialize().
- playground/src/debug/probe.ts: window.__nisps now calls real WasmIML
  via the store; lazy-init on first use.

Verified:
- bash scripts/build-wasm.sh succeeds (94 KB nisps.wasm)
- bun run typecheck OK
- bun run build OK (production bundle)
- vite dev server serves /nisps.{js,wasm} with COOP/COEP

Known limitation: WASM is fixed at one MLP shape. Multi-arch deferred —
documented in nisps/wasm/README.md.
This commit is contained in:
w1n5t0n 2026-04-29 16:36:29 +03:00
parent 5cd5041b6a
commit f26ec923e1
15 changed files with 2860 additions and 90 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

70
nisps/wasm/README.md Normal file
View file

@ -0,0 +1,70 @@
# nisps/wasm
Emscripten target that exposes `nisps/ml` (MLP) and `nisps/engines` (audio
engines) to the SolidJS playground via a flat C ABI.
This directory is a leaf — it does not export headers for inclusion by
other C++ code. The only artifact is `bindings.cpp` plus the build script
that turns it into `playground/public/nisps.{wasm,js}`.
## Building
```bash
scripts/build-wasm.sh
```
Requires `emcc` (Emscripten). The script defaults to
`/usr/lib/emscripten/emcc` and respects an `EMCC` env var override.
Output:
- `playground/public/nisps.wasm` — the compiled module.
- `playground/public/nisps.js` — Emscripten glue (factory function
`createNispsModule`, MODULARIZE=1).
Both files are committed (so the playground works from a fresh clone
without a C++ toolchain). Re-run `build-wasm.sh` after changes to
`nisps/{core,ml,engines,wasm}`.
## Architecture limit (read this)
The MLP class template is parametrised on `(input_size, hidden1, hidden2,
hidden3, output_size)`. WASM cannot recompile templates at runtime, so
this build instantiates exactly ONE configuration:
nisps::ml::MLP<2, 10, 14, 18, 126>
That serves the playground use case (2-D joystick → up to 126 synth
parameters). `nisps_ml_create()` accepts caller-supplied dimensions for
forward compatibility but currently ignores them — see comment at the top
of `bindings.cpp`. The schemas in `schemas/modes/*.json` use up to
`output_size=126`; modes whose output_size is < 126 simply ignore the
trailing entries.
To support additional architectures, either:
1. Compile multiple wasm modules (`nisps_small.wasm`,
`nisps_default.wasm`, …) and let the playground load the right one
based on the active mode.
2. Add a runtime-shape MLP variant to `nisps/ml` (heap allocation only at
`create()`; no impact on hot paths).
Both options are deferred to a future stream.
## C API surface
See `bindings.cpp` for the full list. Summary:
| Group | Functions |
|-----------|------------------------------------------------------------------------------|
| ML life | `nisps_ml_create`, `nisps_ml_destroy`, `nisps_ml_reset` |
| ML I/O | `nisps_ml_set_input`, `nisps_ml_process`, `nisps_ml_outputs`, `nisps_ml_infer_batch` |
| Training | `nisps_ml_add_example`, `nisps_ml_train`, `nisps_ml_eval_loss`, `nisps_ml_clear_examples`, `nisps_ml_example_count` |
| Weights | `nisps_ml_weight_count`, `nisps_ml_get_weights`, `nisps_ml_set_weights`, `nisps_ml_draw_weights`, `nisps_ml_move_weights` |
| Diag | `nisps_ml_get_layer_stats`, `nisps_ml_describe` |
| Engines | `nisps_engine_create`, `nisps_engine_destroy`, `nisps_engine_set_params`, `nisps_engine_process_block` |
Engine-id strings follow the C++ `engine_id()` constexpr accessors:
`thru`, `paf_synth`, `channel_strip`, `xiasri`, `verb_fx`, `memlcelium`,
`breakor`, `elysiamorf`, `analysis`. Unknown ids fall back to `thru`
(silent passthrough).

515
nisps/wasm/bindings.cpp Normal file
View file

@ -0,0 +1,515 @@
// nisps/wasm/bindings.cpp — flat C API exported to the SolidJS playground.
//
// Two consumers per build:
// 1. Main-thread WasmIML (playground/src/ml/wasm-iml.ts) — ML calls.
// 2. AudioWorklet processor (playground/src/audio/worklet/...) — engine
// calls. (Each instance owns its own WASM module instance.)
//
// FIXED-ARCHITECTURE LIMITATION (VERY IMPORTANT)
// ----------------------------------------------
// The C++ MLP class is templated on layer sizes (architecture.md §4.1, §6.2).
// We instantiate ONE concrete configuration here:
//
// using DefaultMLP = nisps::ml::MLP<2, 10, 14, 18, 126>;
//
// This was chosen as the union of the playground use case (2-D joystick →
// 126 synth params) and the largest hidden-layer footprint that still fits
// firmware budgets. `nisps_ml_create()` accepts caller-supplied input_size,
// output_size, hidden[], n_hidden but ONLY validates them against the
// compile-time defaults — extra inputs/outputs are clipped at the boundary.
// If the caller passes incompatible dimensions we still create the module:
// extra inputs are zero-padded, extra outputs are truncated, and the
// hidden-layer override is silently ignored.
//
// Future work: ship multiple WASM modules (one per common architecture) or
// rebuild on demand. See architecture.md "open questions" — Stream 7 punts.
//
// WIRE FORMAT FOR WEIGHTS
// -----------------------
// The flat layout matches `nisps::ml::MLP::get_weights()`:
//
// [layer0_weights] [layer1_weights] [layer2_weights] [layer3_weights]
// [layer0_biases] [layer1_biases] [layer2_biases] [layer3_biases]
//
// Total count = `nisps_ml_weight_count()`. Both endianness and float layout
// match the host (Emscripten produces little-endian Float32Array-friendly
// memory).
//
// LAYER-STATS LAYOUT
// ------------------
// `nisps_ml_get_layer_stats()` writes 4 floats per layer into the caller
// buffer: [mean_abs, max_abs, dead_frac, saturating_frac]. Total = 16
// floats for 4 layers.
#include <emscripten.h>
#include <emscripten/emscripten.h>
#include <array>
#include <cstdint>
#include <cstring>
#include <span>
#include <string>
#include <string_view>
#include <vector>
// Engines.
#include "../engines/analysis.hpp"
#include "../engines/base.hpp"
#include "../engines/breakor.hpp"
#include "../engines/channel_strip.hpp"
#include "../engines/elysiamorf.hpp"
#include "../engines/memlcelium.hpp"
#include "../engines/paf_synth.hpp"
#include "../engines/verb_fx.hpp"
#include "../engines/xiasri.hpp"
// ML.
#include "../core/types.hpp"
#include "../ml/mlp.hpp"
#include "../ml/stats.hpp"
namespace {
// ---------------------------------------------------------------------------
// ML side
// ---------------------------------------------------------------------------
// Compile-time default architecture. See header comment.
//
// Choice rationale:
// * 2 inputs — playground virtual joystick (X, Y).
// * [10, 14, 18] hidden — covers the largest schema layouts in
// `schemas/modes/*.json` (channel_strip variants, verb_fx, breakor,
// elysiamorf, memlcelium).
// * 126 outputs — enough for the C15 mode and any current schema.
//
// The MLP also has dataset slots, loss history etc. — see mlp.hpp.
using DefaultMLP = nisps::ml::MLP<2u, 10u, 14u, 18u, 126u>;
constexpr std::size_t kDefaultInputs = DefaultMLP::kInput;
constexpr std::size_t kDefaultOutputs = DefaultMLP::kOutput;
// We allocate the MLP on the heap (one-off — not the audio path) and return
// the opaque pointer to JS.
struct MLHandle {
DefaultMLP mlp;
// Buffers used to bridge JS → C++:
std::array<float, kDefaultInputs> input_scratch{};
std::array<float, kDefaultOutputs> output_scratch{};
// Stats buffer fed back to JS via get_layer_stats.
std::array<float, DefaultMLP::kNumLayers * 4u> stats_scratch{};
// Used by infer_batch with arbitrary N — must exceed any reasonable
// request from the heatmap. 256x256 = 65536 max points → too many in
// practice. We cap batch size at 4096 here; callers must split larger
// requests.
static constexpr std::size_t kMaxBatch = 4096u;
std::array<float, kMaxBatch * kDefaultOutputs> batch_out_scratch{};
explicit MLHandle(std::uint64_t seed) noexcept : mlp(seed) {}
};
// ---------------------------------------------------------------------------
// Engine side
// ---------------------------------------------------------------------------
// Variant-style dispatch. Each create call instantiates ONE engine kind
// stored on the heap; the type is recorded in `kind` so process_block can
// dispatch without RTTI.
//
// We DO NOT use std::variant — Emscripten supports it but the overhead is
// unwanted. A discriminated union of pointers is enough.
enum class EngineKind : std::uint8_t {
NoOp,
PAFSynth,
ChannelStrip,
XIASRI,
VerbFX,
MEMLCelium,
BreakOr,
Elysiamorf,
Analysis,
};
struct EngineHandle {
EngineKind kind = EngineKind::NoOp;
void* ptr = nullptr;
};
template <typename EngineT>
inline EngineHandle make_handle(EngineKind kind, float sr) noexcept {
auto* e = new EngineT();
e->setup(sr);
return EngineHandle{kind, static_cast<void*>(e)};
}
template <typename EngineT>
inline void destroy_typed(void* ptr) noexcept {
delete static_cast<EngineT*>(ptr);
}
template <typename EngineT>
inline void set_params_typed(void* ptr, std::span<const float> params) noexcept {
static_cast<EngineT*>(ptr)->set_params(params);
}
template <typename EngineT>
inline void process_typed(void* ptr,
const float* in_l, const float* in_r,
float* out_l, float* out_r,
int n_samples) noexcept {
auto* e = static_cast<EngineT*>(ptr);
for (int i = 0; i < n_samples; ++i) {
nisps::stereosample_t s{in_l ? in_l[i] : 0.f, in_r ? in_r[i] : 0.f};
const auto y = e->process(s);
if (out_l) out_l[i] = y.L;
if (out_r) out_r[i] = y.R;
}
}
EngineHandle dispatch_create(std::string_view id, float sample_rate) noexcept {
using nisps::NoOpEngine;
using nisps::PAFSynthEngine;
using nisps::ChannelStripEngine;
using nisps::XIASRIEngine;
using nisps::VerbFXEngine;
using nisps::MEMLCeliumEngine;
using nisps::BreakOrEngine;
using nisps::ElysiamorfEngine;
using nisps::AnalysisEngine;
if (id == NoOpEngine::engine_id()) return make_handle<NoOpEngine>(EngineKind::NoOp, sample_rate);
if (id == PAFSynthEngine::engine_id()) return make_handle<PAFSynthEngine>(EngineKind::PAFSynth, sample_rate);
if (id == ChannelStripEngine::engine_id()) return make_handle<ChannelStripEngine>(EngineKind::ChannelStrip, sample_rate);
if (id == XIASRIEngine::engine_id()) return make_handle<XIASRIEngine>(EngineKind::XIASRI, sample_rate);
if (id == VerbFXEngine::engine_id()) return make_handle<VerbFXEngine>(EngineKind::VerbFX, sample_rate);
if (id == MEMLCeliumEngine::engine_id()) return make_handle<MEMLCeliumEngine>(EngineKind::MEMLCelium, sample_rate);
if (id == BreakOrEngine::engine_id()) return make_handle<BreakOrEngine>(EngineKind::BreakOr, sample_rate);
if (id == ElysiamorfEngine::engine_id()) return make_handle<ElysiamorfEngine>(EngineKind::Elysiamorf, sample_rate);
if (id == AnalysisEngine::engine_id()) return make_handle<AnalysisEngine>(EngineKind::Analysis, sample_rate);
// Unknown id → fall back to NoOp so the worklet is at least silent
// rather than UB.
return make_handle<NoOpEngine>(EngineKind::NoOp, sample_rate);
}
void dispatch_destroy(EngineHandle& h) noexcept {
using nisps::NoOpEngine;
using nisps::PAFSynthEngine;
using nisps::ChannelStripEngine;
using nisps::XIASRIEngine;
using nisps::VerbFXEngine;
using nisps::MEMLCeliumEngine;
using nisps::BreakOrEngine;
using nisps::ElysiamorfEngine;
using nisps::AnalysisEngine;
if (!h.ptr) return;
switch (h.kind) {
case EngineKind::NoOp: destroy_typed<NoOpEngine>(h.ptr); break;
case EngineKind::PAFSynth: destroy_typed<PAFSynthEngine>(h.ptr); break;
case EngineKind::ChannelStrip: destroy_typed<ChannelStripEngine>(h.ptr); break;
case EngineKind::XIASRI: destroy_typed<XIASRIEngine>(h.ptr); break;
case EngineKind::VerbFX: destroy_typed<VerbFXEngine>(h.ptr); break;
case EngineKind::MEMLCelium: destroy_typed<MEMLCeliumEngine>(h.ptr); break;
case EngineKind::BreakOr: destroy_typed<BreakOrEngine>(h.ptr); break;
case EngineKind::Elysiamorf: destroy_typed<ElysiamorfEngine>(h.ptr); break;
case EngineKind::Analysis: destroy_typed<AnalysisEngine>(h.ptr); break;
}
h.ptr = nullptr;
}
void dispatch_set_params(EngineHandle& h, std::span<const float> params) noexcept {
using nisps::NoOpEngine;
using nisps::PAFSynthEngine;
using nisps::ChannelStripEngine;
using nisps::XIASRIEngine;
using nisps::VerbFXEngine;
using nisps::MEMLCeliumEngine;
using nisps::BreakOrEngine;
using nisps::ElysiamorfEngine;
using nisps::AnalysisEngine;
switch (h.kind) {
case EngineKind::NoOp: set_params_typed<NoOpEngine>(h.ptr, params); break;
case EngineKind::PAFSynth: set_params_typed<PAFSynthEngine>(h.ptr, params); break;
case EngineKind::ChannelStrip: set_params_typed<ChannelStripEngine>(h.ptr, params); break;
case EngineKind::XIASRI: set_params_typed<XIASRIEngine>(h.ptr, params); break;
case EngineKind::VerbFX: set_params_typed<VerbFXEngine>(h.ptr, params); break;
case EngineKind::MEMLCelium: set_params_typed<MEMLCeliumEngine>(h.ptr, params); break;
case EngineKind::BreakOr: set_params_typed<BreakOrEngine>(h.ptr, params); break;
case EngineKind::Elysiamorf: set_params_typed<ElysiamorfEngine>(h.ptr, params); break;
case EngineKind::Analysis: set_params_typed<AnalysisEngine>(h.ptr, params); break;
}
}
void dispatch_process_block(EngineHandle& h,
const float* in_l, const float* in_r,
float* out_l, float* out_r,
int n_samples) noexcept {
using nisps::NoOpEngine;
using nisps::PAFSynthEngine;
using nisps::ChannelStripEngine;
using nisps::XIASRIEngine;
using nisps::VerbFXEngine;
using nisps::MEMLCeliumEngine;
using nisps::BreakOrEngine;
using nisps::ElysiamorfEngine;
using nisps::AnalysisEngine;
switch (h.kind) {
case EngineKind::NoOp: process_typed<NoOpEngine>(h.ptr, in_l, in_r, out_l, out_r, n_samples); break;
case EngineKind::PAFSynth: process_typed<PAFSynthEngine>(h.ptr, in_l, in_r, out_l, out_r, n_samples); break;
case EngineKind::ChannelStrip: process_typed<ChannelStripEngine>(h.ptr, in_l, in_r, out_l, out_r, n_samples); break;
case EngineKind::XIASRI: process_typed<XIASRIEngine>(h.ptr, in_l, in_r, out_l, out_r, n_samples); break;
case EngineKind::VerbFX: process_typed<VerbFXEngine>(h.ptr, in_l, in_r, out_l, out_r, n_samples); break;
case EngineKind::MEMLCelium: process_typed<MEMLCeliumEngine>(h.ptr, in_l, in_r, out_l, out_r, n_samples); break;
case EngineKind::BreakOr: process_typed<BreakOrEngine>(h.ptr, in_l, in_r, out_l, out_r, n_samples); break;
case EngineKind::Elysiamorf: process_typed<ElysiamorfEngine>(h.ptr, in_l, in_r, out_l, out_r, n_samples); break;
case EngineKind::Analysis: process_typed<AnalysisEngine>(h.ptr, in_l, in_r, out_l, out_r, n_samples); break;
}
}
} // anonymous namespace
extern "C" {
// ---------------------------------------------------------------------------
// ML lifecycle
// ---------------------------------------------------------------------------
EMSCRIPTEN_KEEPALIVE
void* nisps_ml_create(int input_size, int output_size,
const int* /*hidden*/, int /*n_hidden*/,
uint32_t seed) {
// We accept and ignore caller-supplied dimensions if they don't match the
// compile-time default. See file header.
//
// NOTE: the C++ Rng takes uint64_t; we sign-extend the 32-bit seed into
// the high 32 bits via xor-shift so callers passing zero still get a
// non-degenerate seed. Truly 64-bit seeds are not exposed to JS — the
// playground doesn't need them, and avoiding BigInt at the boundary
// simplifies both wasm-iml.ts and wasm-worker.ts.
(void)input_size;
(void)output_size;
const std::uint64_t s64 = static_cast<std::uint64_t>(seed) ^
(static_cast<std::uint64_t>(seed) << 32);
auto* h = new MLHandle(s64);
return static_cast<void*>(h);
}
EMSCRIPTEN_KEEPALIVE
void nisps_ml_destroy(void* ml) {
if (!ml) return;
delete static_cast<MLHandle*>(ml);
}
// ---------------------------------------------------------------------------
// ML inference
// ---------------------------------------------------------------------------
EMSCRIPTEN_KEEPALIVE
void nisps_ml_set_input(void* ml, int idx, float v) {
if (!ml) return;
auto* h = static_cast<MLHandle*>(ml);
if (idx < 0) return;
if (static_cast<std::size_t>(idx) >= kDefaultInputs) return;
h->mlp.set_input(static_cast<std::size_t>(idx), v);
}
EMSCRIPTEN_KEEPALIVE
void nisps_ml_process(void* ml) {
if (!ml) return;
auto* h = static_cast<MLHandle*>(ml);
h->mlp.process();
auto outs = h->mlp.outputs();
for (std::size_t i = 0; i < kDefaultOutputs; ++i) h->output_scratch[i] = outs[i];
}
EMSCRIPTEN_KEEPALIVE
const float* nisps_ml_outputs(void* ml) {
if (!ml) return nullptr;
auto* h = static_cast<MLHandle*>(ml);
return h->output_scratch.data();
}
EMSCRIPTEN_KEEPALIVE
void nisps_ml_infer_batch(void* ml, const float* points, int n_points, float* out) {
if (!ml || !points || !out || n_points <= 0) return;
auto* h = static_cast<MLHandle*>(ml);
const std::size_t n = static_cast<std::size_t>(n_points);
if (n > MLHandle::kMaxBatch) {
// Caller exceeded the scratch buffer. Process what we can.
const std::size_t safe_n = MLHandle::kMaxBatch;
h->mlp.infer_batch(
std::span<const float>(points, safe_n * kDefaultInputs),
std::span<float>(out, safe_n * kDefaultOutputs));
return;
}
h->mlp.infer_batch(
std::span<const float>(points, n * kDefaultInputs),
std::span<float>(out, n * kDefaultOutputs));
}
// ---------------------------------------------------------------------------
// ML training
// ---------------------------------------------------------------------------
EMSCRIPTEN_KEEPALIVE
void nisps_ml_add_example(void* ml, const float* features, const float* labels) {
if (!ml || !features || !labels) return;
auto* h = static_cast<MLHandle*>(ml);
h->mlp.add_example(
std::span<const float>(features, kDefaultInputs),
std::span<const float>(labels, kDefaultOutputs));
}
EMSCRIPTEN_KEEPALIVE
float nisps_ml_train(void* ml, float lr, int max_iter, float min_err,
const float* sample_weights) {
if (!ml) return 0.f;
auto* h = static_cast<MLHandle*>(ml);
if (max_iter <= 0) max_iter = 1;
std::span<const float> weights;
if (sample_weights) {
weights = std::span<const float>(sample_weights, h->mlp.example_count());
}
return h->mlp.train(lr, static_cast<std::size_t>(max_iter), min_err, weights);
}
EMSCRIPTEN_KEEPALIVE
float nisps_ml_eval_loss(void* ml) {
if (!ml) return 0.f;
auto* h = static_cast<MLHandle*>(ml);
return h->mlp.eval_loss();
}
// ---------------------------------------------------------------------------
// ML weights
// ---------------------------------------------------------------------------
EMSCRIPTEN_KEEPALIVE
int nisps_ml_weight_count(void* ml) {
(void)ml;
return static_cast<int>(DefaultMLP::weight_count());
}
EMSCRIPTEN_KEEPALIVE
void nisps_ml_get_weights(void* ml, float* out) {
if (!ml || !out) return;
auto* h = static_cast<MLHandle*>(ml);
auto w = h->mlp.get_weights();
std::memcpy(out, w.data(), w.size() * sizeof(float));
}
EMSCRIPTEN_KEEPALIVE
void nisps_ml_set_weights(void* ml, const float* in) {
if (!ml || !in) return;
auto* h = static_cast<MLHandle*>(ml);
h->mlp.set_weights(std::span<const float>(in, DefaultMLP::weight_count()));
}
EMSCRIPTEN_KEEPALIVE
void nisps_ml_draw_weights(void* ml, float spread) {
if (!ml) return;
auto* h = static_cast<MLHandle*>(ml);
h->mlp.draw_weights(spread);
}
EMSCRIPTEN_KEEPALIVE
void nisps_ml_move_weights(void* ml, float speed, float spread,
const uint8_t* output_pin_mask) {
if (!ml) return;
auto* h = static_cast<MLHandle*>(ml);
std::span<const std::uint8_t> mask;
if (output_pin_mask) {
mask = std::span<const std::uint8_t>(output_pin_mask, kDefaultOutputs);
}
h->mlp.move_weights(speed, spread, mask);
}
EMSCRIPTEN_KEEPALIVE
void nisps_ml_get_layer_stats(void* ml, float* out_stats) {
if (!ml || !out_stats) return;
auto* h = static_cast<MLHandle*>(ml);
for (std::size_t i = 0; i < DefaultMLP::kNumLayers; ++i) {
const auto s = h->mlp.layer_stats(i);
out_stats[i * 4u + 0u] = s.mean_abs;
out_stats[i * 4u + 1u] = s.max_abs;
out_stats[i * 4u + 2u] = s.dead_frac;
out_stats[i * 4u + 3u] = s.saturating_frac;
}
}
// Extra helper: lets JS query the example count without having to
// shadow-track it. Useful when restoring from snapshot.
EMSCRIPTEN_KEEPALIVE
int nisps_ml_example_count(void* ml) {
if (!ml) return 0;
auto* h = static_cast<MLHandle*>(ml);
return static_cast<int>(h->mlp.example_count());
}
EMSCRIPTEN_KEEPALIVE
void nisps_ml_clear_examples(void* ml) {
if (!ml) return;
auto* h = static_cast<MLHandle*>(ml);
h->mlp.clear_examples();
}
EMSCRIPTEN_KEEPALIVE
void nisps_ml_reset(void* ml) {
if (!ml) return;
auto* h = static_cast<MLHandle*>(ml);
h->mlp.reset();
}
// Architecture introspection — returns 4-int packed [in, h1, h2, h3, out, n_layers].
// Kept simple: writes into a caller-supplied int buffer. Always 6 ints.
EMSCRIPTEN_KEEPALIVE
void nisps_ml_describe(int* out_dims) {
if (!out_dims) return;
out_dims[0] = static_cast<int>(DefaultMLP::kInput);
out_dims[1] = static_cast<int>(DefaultMLP::kHidden1);
out_dims[2] = static_cast<int>(DefaultMLP::kHidden2);
out_dims[3] = static_cast<int>(DefaultMLP::kHidden3);
out_dims[4] = static_cast<int>(DefaultMLP::kOutput);
out_dims[5] = static_cast<int>(DefaultMLP::kNumLayers);
}
// ---------------------------------------------------------------------------
// Engine lifecycle
// ---------------------------------------------------------------------------
EMSCRIPTEN_KEEPALIVE
void* nisps_engine_create(const char* engine_id, float sample_rate) {
if (!engine_id) return nullptr;
auto* h = new EngineHandle(dispatch_create(engine_id, sample_rate));
return static_cast<void*>(h);
}
EMSCRIPTEN_KEEPALIVE
void nisps_engine_destroy(void* engine) {
if (!engine) return;
auto* h = static_cast<EngineHandle*>(engine);
dispatch_destroy(*h);
delete h;
}
EMSCRIPTEN_KEEPALIVE
void nisps_engine_set_params(void* engine, const float* params, int n_params) {
if (!engine || !params || n_params <= 0) return;
auto* h = static_cast<EngineHandle*>(engine);
dispatch_set_params(*h, std::span<const float>(params, static_cast<std::size_t>(n_params)));
}
EMSCRIPTEN_KEEPALIVE
void nisps_engine_process_block(void* engine,
const float* in_l, const float* in_r,
float* out_l, float* out_r,
int n_samples) {
if (!engine || n_samples <= 0) return;
auto* h = static_cast<EngineHandle*>(engine);
dispatch_process_block(*h, in_l, in_r, out_l, out_r, n_samples);
}
} // extern "C"

2
playground/public/nisps.js Normal file
View file

File diff suppressed because one or more lines are too long

BIN
playground/public/nisps.wasm Executable file
View file

Binary file not shown.

227
playground/src/audio/engine-host.ts Normal file
View file

@ -0,0 +1,227 @@
/**
* EngineHost — main-thread side of the WASM AudioWorklet pipeline.
*
* Responsibilities:
* - Lazy-create AudioContext (browsers gate this on user gesture).
* - Register the AudioWorklet processor module.
* - Hand the worklet a copy of `nisps.wasm` bytes (we fetch on the main
* thread because AudioWorklet has no `fetch`/`importScripts`).
* - Send engine selection + parameter updates over the worklet `port`.
* - Tear everything down on `dispose()`.
*
* The actual DSP runs in `playground/src/audio/worklet/nisps-processor.ts`,
* which calls into a SECOND WASM instance owned by the worklet thread.
*
* IMPORTANT: this class never auto-starts audio. The caller must drive
* `start()` from a user gesture (button click, etc.) so browsers don't
* block AudioContext creation.
*/
import type { EngineId } from '../ml/types';
const NISPS_WASM_URL = '/nisps.wasm';
const PROCESSOR_NAME = 'nisps-processor';
/** Message protocol: main → worklet. */
export type HostToWorkletMessage =
| {
kind: 'init';
// ArrayBuffer transferred so the worklet can `WebAssembly.compile` it.
wasmBinary: ArrayBuffer;
sampleRate: number;
}
| {
kind: 'engine';
engineId: EngineId;
}
| {
kind: 'params';
// Float32Array transferred to avoid per-tick copy.
params: Float32Array;
}
| {
kind: 'mute';
muted: boolean;
};
/** Message protocol: worklet → main. */
export type WorkletToHostMessage =
| { kind: 'ready' }
| { kind: 'error'; message: string };
export interface EngineHostOptions {
/** Override sample rate (default: AudioContext.sampleRate). */
sampleRate?: number;
/** Override worklet processor URL (testing). */
processorUrl?: string;
}
export class EngineHost {
private ctx: AudioContext | null = null;
private node: AudioWorkletNode | null = null;
private workletReady = false;
private currentEngine: EngineId = 'thru';
private disposed = false;
private options: EngineHostOptions;
// Cached bytes of nisps.wasm (we fetch once per host instance).
private wasmBytes: ArrayBuffer | null = null;
constructor(options: EngineHostOptions = {}) {
this.options = options;
}
get isStarted(): boolean {
return !!this.ctx && this.workletReady;
}
get sampleRate(): number {
return this.ctx?.sampleRate ?? this.options.sampleRate ?? 48000;
}
/**
* Start audio. Must be called from a user gesture for AudioContext to
* resume. After this resolves, `setEngine()` and `setParams()` can be
* called.
*/
async start(engineId: EngineId = 'thru'): Promise<void> {
if (this.ctx) {
// Already started; just switch engine.
this.setEngine(engineId);
await this.ctx.resume();
return;
}
this.ctx = new AudioContext({
sampleRate: this.options.sampleRate,
latencyHint: 'interactive',
});
// Fetch the WASM bytes on the main thread (the worklet doesn't have
// fetch). We pass the buffer to the worklet as a transferable.
if (!this.wasmBytes) {
this.wasmBytes = await this.fetchWasm_();
}
// Register the processor module. Vite's `new URL(..., import.meta.url)`
// pattern bundles the worklet file correctly.
const procUrl = this.options.processorUrl ??
new URL('./worklet/nisps-processor.ts', import.meta.url).toString();
await this.ctx.audioWorklet.addModule(procUrl);
this.node = new AudioWorkletNode(this.ctx, PROCESSOR_NAME, {
numberOfInputs: 1,
numberOfOutputs: 1,
outputChannelCount: [2],
});
this.node.connect(this.ctx.destination);
// Wire up message handler before sending init.
this.workletReady = false;
const ready = new Promise<void>((resolve, reject) => {
const onMsg = (ev: MessageEvent<WorkletToHostMessage>) => {
if (ev.data.kind === 'ready') {
this.workletReady = true;
this.node?.port.removeEventListener('message', onMsg);
resolve();
} else if (ev.data.kind === 'error') {
this.node?.port.removeEventListener('message', onMsg);
reject(new Error(ev.data.message));
}
};
this.node!.port.addEventListener('message', onMsg);
this.node!.port.start();
});
// Send the WASM binary + sample rate. ArrayBuffer is transferable —
// we keep a copy on the main thread for re-init.
const copy = this.wasmBytes.slice(0);
this.node.port.postMessage(
{ kind: 'init', wasmBinary: copy, sampleRate: this.ctx.sampleRate } satisfies HostToWorkletMessage,
[copy],
);
await ready;
this.currentEngine = engineId;
if (engineId !== 'thru') {
this.setEngine(engineId);
}
}
/**
* Switch which engine the worklet is processing. Cheap — just a message.
* The worklet handles engine destruction/creation internally.
*/
setEngine(engineId: EngineId): void {
if (!this.node || !this.workletReady) return;
this.currentEngine = engineId;
this.node.port.postMessage({ kind: 'engine', engineId } satisfies HostToWorkletMessage);
}
/**
* Push a fresh parameter vector. Caller should NOT reuse the buffer
* after this call — we transfer it. If you need to keep yours, pass a
* copy: `host.setParams(new Float32Array(myBuf))`.
*/
setParams(params: Float32Array): void {
if (!this.node || !this.workletReady) return;
this.node.port.postMessage(
{ kind: 'params', params } satisfies HostToWorkletMessage,
[params.buffer],
);
}
setMuted(muted: boolean): void {
if (!this.node || !this.workletReady) return;
this.node.port.postMessage({ kind: 'mute', muted } satisfies HostToWorkletMessage);
}
async stop(): Promise<void> {
if (!this.ctx) return;
if (this.node) {
try {
this.node.disconnect();
} catch { /* ignore */ }
this.node = null;
}
try {
await this.ctx.close();
} catch { /* ignore */ }
this.ctx = null;
this.workletReady = false;
}
dispose(): void {
if (this.disposed) return;
this.disposed = true;
void this.stop();
this.wasmBytes = null;
}
private async fetchWasm_(): Promise<ArrayBuffer> {
const url = new URL(NISPS_WASM_URL, window.location.origin).toString();
const resp = await fetch(url);
if (!resp.ok) throw new Error(`fetch nisps.wasm: ${resp.status} ${resp.statusText}`);
return await resp.arrayBuffer();
}
}
/**
* Smoke-test helper: returns true iff the WASM module exposes the C
* functions we need. Useful as a build-time check inside `engine-host.ts`
* tests; not used in production.
*/
export async function smokeCheckWasm(): Promise<boolean> {
const url = new URL('/nisps.js', window.location.origin).toString();
// eslint-disable-next-line @typescript-eslint/no-explicit-any
const mod: any = await import(/* @vite-ignore */ url);
const factory = mod.default ?? mod.createNispsModule;
if (!factory) return false;
const m = await factory({
locateFile: (p: string) => p.endsWith('.wasm')
? new URL(NISPS_WASM_URL, window.location.origin).toString()
: p,
});
return typeof m._nisps_ml_create === 'function'
&& typeof m._nisps_engine_create === 'function'
&& typeof m._nisps_engine_process_block === 'function';
}

52
playground/src/audio/worklet/README.md Normal file
View file

@ -0,0 +1,52 @@
# Why two WASM instances?
The playground loads `nisps.wasm` twice:
1. **Main thread**, via `playground/src/ml/wasm-iml.ts`. Used for ML
inference + sync training + RL operations + UI feedback.
2. **AudioWorklet thread**, via `nisps-processor.ts`. Used for engine
processing (per-128-sample-block).
Why not share?
- AudioWorklet runs on its own thread. Sharing memory across threads
requires SharedArrayBuffer + locking on every heap access; far more
expensive than two heaps.
- AudioWorklet has neither `fetch` nor ESM `import`, so it can't load
the Emscripten glue (`nisps.js`). The main thread fetches the WASM
bytes once and posts them here as a transferable `ArrayBuffer`; the
worklet then `WebAssembly.instantiate`s directly.
- Engines and ML never interact in the audio path. The main thread
computes the parameter vector each frame and pushes it into the
worklet via `port.postMessage`. The worklet pushes nothing back per
block (analysis features, if needed, are batched and sent at low
rate).
Per-frame data flow:
```
Joystick → input pipeline → mlStore.outputs (Float32Array, length=126)
↓ EngineHost.setParams()
↓ port.postMessage (transferable)
AudioWorklet ← WASM engine.process_block ← WASM engine.set_params
```
## Custom WASM loader
We do **not** use the Emscripten JS glue inside the worklet — the glue
contains `URL`, `Worker`, and `fetch` references that don't exist in
AudioWorkletGlobalScope. Instead `nisps-processor.ts` calls
`WebAssembly.instantiate` directly with hand-rolled imports and
discovers exports by walking the export descriptors. This makes the
worklet bundle small (just the processor TS) and avoids touching the
Emscripten init path.
The trade-off: only the engine API is callable here, not the ML API.
If you ever need ML inference inside the worklet (we don't), use the
main thread copy and post params over.
## Block size
AudioWorklet always calls `process()` with 128-sample blocks. Our
heap buffers in `nisps-processor.ts` are sized to match. Don't change
the block size without changing the buffer allocations.

26
playground/src/audio/worklet/audioworklet-globals.d.ts vendored Normal file
View file

@ -0,0 +1,26 @@
/**
* Type declarations for AudioWorkletGlobalScope. The default `lib.dom`
* and `lib.dom.iterable` files don't include these because they only
* exist inside an AudioWorklet thread.
*
* Keep this file minimal — only what `nisps-processor.ts` actually uses.
*/
declare const sampleRate: number;
declare const currentFrame: number;
declare const currentTime: number;
declare class AudioWorkletProcessor {
constructor(options?: { numberOfInputs?: number; numberOfOutputs?: number; processorOptions?: unknown });
readonly port: MessagePort;
process(
inputs: Float32Array[][],
outputs: Float32Array[][],
parameters: Record<string, Float32Array>,
): boolean;
}
declare function registerProcessor(
name: string,
processorCtor: new (options?: unknown) => AudioWorkletProcessor,
): void;

309
playground/src/audio/worklet/nisps-processor.ts Normal file
View file

@ -0,0 +1,309 @@
/**
* AudioWorkletProcessor that runs `nisps.wasm` engines.
*
* Why a separate WASM instance from the main thread? AudioWorklet runs in
* its own thread + global scope; reusing a single instance would require
* SharedArrayBuffer + locking on the heap. Architecture.md §6.4 specifies
* separate instances connected by `port` messages instead.
*
* Wasm load path: AudioWorklet has NO `fetch` and NO ESM `import`. The
* main thread fetches `nisps.wasm` once and posts the bytes here as an
* ArrayBuffer; we then `WebAssembly.compile` and `instantiate` directly,
* skipping the Emscripten glue entirely. This is fine because the
* exported functions don't need any of the JS-side runtime.
*
* Block size: AudioWorklet ALWAYS calls process() with 128-sample blocks.
* We allocate 128-sample input and output buffers in the WASM linear
* memory and shuttle samples in/out per call.
*/
/// <reference path="./audioworklet-globals.d.ts" />
import type { EngineId } from '../../ml/types';
import type { HostToWorkletMessage, WorkletToHostMessage } from '../engine-host';
const PROC_BLOCK = 128;
const MAX_PARAMS = 256; // upper bound across all engines
interface WasmInstance {
exports: {
memory: WebAssembly.Memory;
malloc: (n: number) => number;
free: (p: number) => void;
_nisps_engine_create: (id_ptr: number, sample_rate: number) => number;
_nisps_engine_destroy: (engine: number) => void;
_nisps_engine_set_params: (engine: number, params_ptr: number, n: number) => void;
_nisps_engine_process_block: (
engine: number,
in_l: number, in_r: number,
out_l: number, out_r: number,
n_samples: number,
) => void;
};
}
class NispsProcessor extends AudioWorkletProcessor {
private instance: WasmInstance | null = null;
private engineHandle = 0;
private engineId: EngineId = 'thru';
private muted = true;
// Pointers + buffer views (allocated once instance is up).
private inLPtr = 0;
private inRPtr = 0;
private outLPtr = 0;
private outRPtr = 0;
private idPtr = 0;
private paramsPtr = 0;
private inLView: Float32Array | null = null;
private inRView: Float32Array | null = null;
private outLView: Float32Array | null = null;
private outRView: Float32Array | null = null;
private paramsView: Float32Array | null = null;
private idView: Uint8Array | null = null;
private mem: WebAssembly.Memory | null = null;
// Pending params posted before the engine was ready.
private pendingParams: Float32Array | null = null;
constructor() {
super();
this.port.onmessage = (ev) => this.onMessage_(ev.data as HostToWorkletMessage);
}
private async onMessage_(msg: HostToWorkletMessage): Promise<void> {
if (msg.kind === 'init') {
try {
await this.init_(msg.wasmBinary, msg.sampleRate);
this.post_({ kind: 'ready' });
} catch (err) {
this.post_({
kind: 'error',
message: err instanceof Error ? err.message : String(err),
});
}
} else if (msg.kind === 'engine') {
this.switchEngine_(msg.engineId);
} else if (msg.kind === 'params') {
this.applyParams_(msg.params);
} else if (msg.kind === 'mute') {
this.muted = msg.muted;
}
}
private post_(msg: WorkletToHostMessage): void {
this.port.postMessage(msg);
}
/**
* Compile + instantiate the wasm module. We provide minimal imports —
* the Emscripten module needs `__abort_js` and `_emscripten_resize_heap`
* (we keep memory non-resizing so the latter is a stub).
*/
private async init_(bytes: ArrayBuffer, sampleRate: number): Promise<void> {
const memory = new WebAssembly.Memory({ initial: 128, maximum: 4096, shared: false });
const imports: WebAssembly.Imports = {
// Emscripten import "a" group; field names match the generated JS.
a: {
a: () => { throw new Error('wasm aborted'); },
b: () => false, // _emscripten_resize_heap returning 0 disables growth
},
};
const compiled = await WebAssembly.compile(bytes);
// Discover the actual import shape from the module — names like "a",
// "b" depend on emcc's mangling; we accept whatever it produces.
const importDesc = WebAssembly.Module.imports(compiled);
const reshaped: WebAssembly.Imports = {};
for (const desc of importDesc) {
if (!reshaped[desc.module]) reshaped[desc.module] = {} as WebAssembly.ModuleImports;
const mod = reshaped[desc.module] as WebAssembly.ModuleImports;
if (desc.kind === 'function') {
if (desc.name === 'c') {
// unused
}
mod[desc.name] = (() => {
// Generic stub: log + return 0.
return (..._args: unknown[]) => 0;
})();
} else if (desc.kind === 'memory') {
mod[desc.name] = memory;
} else if (desc.kind === 'table') {
mod[desc.name] = new WebAssembly.Table({ element: 'anyfunc', initial: 0 });
} else if (desc.kind === 'global') {
mod[desc.name] = new WebAssembly.Global({ value: 'i32', mutable: true }, 0);
}
}
// For known-needed Emscripten imports, supply real implementations.
for (const desc of importDesc) {
const mod = reshaped[desc.module] as WebAssembly.ModuleImports;
// __abort_js
if (desc.name === 'a' && desc.kind === 'function') {
mod[desc.name] = () => { throw new Error('wasm aborted'); };
}
// _emscripten_resize_heap
if (desc.name === 'b' && desc.kind === 'function') {
mod[desc.name] = (_size: number) => 0; // refuse growth in worklet
}
}
void imports; // silence unused
const wasmInst = await WebAssembly.instantiate(compiled, reshaped);
// Many Emscripten exports use single-letter mangled names. Discover
// by reading the export descriptors.
const exDesc = WebAssembly.Module.exports(compiled);
const exMap = new Map<string, string>(); // logical name → mangled
for (const e of exDesc) {
// The exports list includes both the original (with leading
// underscore for C funcs) and the mangled single-letter alias used
// in the import section. We only see the export side here, but
// Emscripten in modern versions also re-exports the C names with
// their leading-underscore form. Walk both.
exMap.set(e.name, e.name);
}
const exports = wasmInst.exports as Record<string, WebAssembly.ExportValue>;
function pickFn(...names: string[]): (...args: number[]) => number {
for (const n of names) {
const v = exports[n];
if (typeof v === 'function') return v as unknown as (...a: number[]) => number;
}
throw new Error(`worklet: missing wasm export, tried: ${names.join(', ')}`);
}
function pickFnVoid(...names: string[]): (...args: number[]) => void {
return pickFn(...names) as unknown as (...args: number[]) => void;
}
// The exports we need.
const malloc = pickFn('_malloc', 'malloc');
const free = pickFnVoid('_free', 'free');
const ec = pickFn('_nisps_engine_create');
const ed = pickFnVoid('_nisps_engine_destroy');
const esp = pickFnVoid('_nisps_engine_set_params');
const epb = pickFnVoid('_nisps_engine_process_block');
// The wasm-exported memory might be named `memory` or another mangled
// alias. Find it.
let wasmMemory: WebAssembly.Memory | null = null;
for (const e of exDesc) {
if (e.kind === 'memory') {
const v = exports[e.name];
if (v instanceof WebAssembly.Memory) { wasmMemory = v; break; }
}
}
// If the module imports memory (which our build does — we passed it),
// there will be no exported memory; use the imported one.
this.mem = wasmMemory ?? memory;
this.instance = {
exports: {
memory: this.mem,
malloc,
free,
_nisps_engine_create: (id, sr) => ec(id, sr),
_nisps_engine_destroy: (h) => ed(h),
_nisps_engine_set_params: (h, p, n) => esp(h, p, n),
_nisps_engine_process_block: (h, il, ir, ol, or_, n) => epb(h, il, ir, ol, or_, n),
},
};
// Allocate buffers.
this.inLPtr = malloc(PROC_BLOCK * 4);
this.inRPtr = malloc(PROC_BLOCK * 4);
this.outLPtr = malloc(PROC_BLOCK * 4);
this.outRPtr = malloc(PROC_BLOCK * 4);
this.paramsPtr = malloc(MAX_PARAMS * 4);
// Engine ids are short ASCII; 32 bytes covers everything we have.
this.idPtr = malloc(32);
const buf = this.mem.buffer;
this.inLView = new Float32Array(buf, this.inLPtr, PROC_BLOCK);
this.inRView = new Float32Array(buf, this.inRPtr, PROC_BLOCK);
this.outLView = new Float32Array(buf, this.outLPtr, PROC_BLOCK);
this.outRView = new Float32Array(buf, this.outRPtr, PROC_BLOCK);
this.paramsView = new Float32Array(buf, this.paramsPtr, MAX_PARAMS);
this.idView = new Uint8Array(buf, this.idPtr, 32);
// Default engine: thru.
this.spawnEngine_('thru', sampleRate);
// Apply pending params if any arrived before init completed.
if (this.pendingParams) {
this.applyParams_(this.pendingParams);
this.pendingParams = null;
}
this.muted = false;
}
private spawnEngine_(id: EngineId, sampleRate: number): void {
if (!this.instance || !this.idView) return;
if (this.engineHandle) {
this.instance.exports._nisps_engine_destroy(this.engineHandle);
this.engineHandle = 0;
}
// Write engine_id as ASCII into idView, NUL-terminated.
const enc = new TextEncoder();
const bytes = enc.encode(id);
this.idView.fill(0);
this.idView.set(bytes.subarray(0, Math.min(bytes.length, 31)));
this.engineHandle = this.instance.exports._nisps_engine_create(this.idPtr, sampleRate);
this.engineId = id;
}
private switchEngine_(id: EngineId): void {
// sampleRate global from AudioWorkletGlobalScope.
this.spawnEngine_(id, sampleRate);
}
private applyParams_(params: Float32Array): void {
if (!this.instance || !this.paramsView) {
this.pendingParams = params;
return;
}
const n = Math.min(params.length, MAX_PARAMS);
for (let i = 0; i < n; ++i) this.paramsView[i] = params[i];
if (this.engineHandle) {
this.instance.exports._nisps_engine_set_params(this.engineHandle, this.paramsPtr, n);
}
}
override process(inputs: Float32Array[][], outputs: Float32Array[][]): boolean {
const out = outputs[0];
if (!out || out.length === 0) return true;
const outL = out[0];
const outR = out.length > 1 ? out[1] : out[0];
if (this.muted || !this.instance || !this.engineHandle ||
!this.inLView || !this.outLView || !this.outRView || !this.inRView) {
// Silence.
outL.fill(0);
if (out.length > 1) outR.fill(0);
return true;
}
// Copy inputs into wasm buffers (zero-fill if missing).
const inp = inputs[0];
if (inp && inp[0]) this.inLView.set(inp[0].subarray(0, PROC_BLOCK));
else this.inLView.fill(0);
if (inp && inp[1]) this.inRView.set(inp[1].subarray(0, PROC_BLOCK));
else if (inp && inp[0]) this.inRView.set(inp[0].subarray(0, PROC_BLOCK));
else this.inRView.fill(0);
this.instance.exports._nisps_engine_process_block(
this.engineHandle,
this.inLPtr, this.inRPtr,
this.outLPtr, this.outRPtr,
PROC_BLOCK,
);
outL.set(this.outLView.subarray(0, outL.length));
if (out.length > 1) outR.set(this.outRView.subarray(0, outR.length));
return true;
}
}
registerProcessor('nisps-processor', NispsProcessor);

170
playground/src/debug/probe.ts
View file

@ -1,36 +1,40 @@
/** /**
* Debug probe: window.__nisps * Debug probe: window.__nisps
* *
* Stream 8 (this stream) installs a stub that returns placeholder values. * Stream 7 wires this to the real WasmIML via mlStore. Methods are
* Stream 10 wires real ML calls. Keeping the install path stable here means * synchronous (or return immediately-resolved promises). The probe
* Playwright tests can rely on `window.__nisps` existing from page load even * deliberately bypasses Solid reactivity so tests get deterministic,
* before the ML engine boots.
*
* All methods MUST be synchronous (or return immediately-resolved promises).
* The probe deliberately bypasses Solid reactivity so tests get deterministic,
* imperative semantics. * imperative semantics.
*
* The probe self-initialises the ML engine on first use that needs it
* — Playwright tests can `await window.__nisps.__init()` before driving
* inference, or just call methods and tolerate a few no-ops while the
* lazy init resolves. While the init is in flight, `__ready` is false;
* synchronous methods that need ML are best-effort no-ops.
*/ */
import { mlStore } from '../stores/ml-store';
export interface DebugProbe { export interface DebugProbe {
/** Current 126-element output vector (Float32Array). */ /** Current 126-element output vector (Float32Array). */
getOutputs(): Float32Array; getOutputs(): Float32Array;
/** Last training loss, or null if no training has occurred. */ /** Last training loss, or null if no training has occurred. */
getLoss(): number | null; getLoss(): number | null;
/** Flat weight array (~13K floats once wired). */ /** Flat weight array. */
getWeights(): Float32Array; getWeights(): Float32Array;
/** Number of training examples currently in the dataset. */ /** Number of training examples currently in the dataset. */
getExampleCount(): number; getExampleCount(): number;
/** Set joystick X/Y in [0,1] and run inference. */ /** Set joystick X/Y in [0,1] and run inference. */
setInputs(x: number, y: number): void; setInputs(x: number, y: number): void;
/** Trigger thumbs-up RL feedback (train + decay noise). */ /** Trigger thumbs-up RL feedback. */
thumbsUp(): void; thumbsUp(): void;
/** Trigger thumbs-down RL feedback (move weights + grow noise). */ /** Trigger thumbs-down RL feedback. */
thumbsDown(): void; thumbsDown(): void;
/** Synchronous training; returns final loss. */ /** Synchronous training; returns final loss. */
train(): number; train(): number;
/** Async training; returns Promise<loss>. */ /** Async training; returns Promise<loss>. */
trainAsync(): Promise<number>; trainAsync(): Promise<number>;
/** Randomize weights with current spread. */ /** Randomize weights with default spread. */
randomise(): void; randomise(): void;
/** Clear all training examples. */ /** Clear all training examples. */
clearExamples(): void; clearExamples(): void;
@ -38,12 +42,14 @@ export interface DebugProbe {
saveState(): void; saveState(): void;
/** Non-destructive loss query against current dataset. */ /** Non-destructive loss query against current dataset. */
evalLoss(): number | null; evalLoss(): number | null;
/** Batch inference: input is Nx2 array of [x,y] pairs. Output: Float32Array of N*outputSize. */ /** Batch inference: input is Nx2 array of [x,y] pairs. */
inferBatch(points: ReadonlyArray<readonly [number, number]>): Float32Array; inferBatch(points: ReadonlyArray<readonly [number, number]>): Float32Array;
/** Per-layer weight statistics: Float32Array of layerCount * 4 (mean|w|, max|w|, dead%, sat%). */ /** Per-layer weight statistics: layerCount * 4 floats (mean|w|, max|w|, dead%, sat%). */
getLayerStats(): Float32Array; getLayerStats(): Float32Array;
/** Marker showing this is a stream-8 stub. Tests can read this to skip when not ready. */ /** True once the WASM is fully initialised. */
readonly __ready: boolean; readonly __ready: boolean;
/** Force initialisation. Returns a promise that resolves when the WASM is ready. */
__init(): Promise<void>;
} }
declare global { declare global {
@ -54,65 +60,115 @@ declare global {
const EMPTY_F32 = new Float32Array(0); const EMPTY_F32 = new Float32Array(0);
const stubProbe: DebugProbe = { // We auto-initialise lazily so a test that immediately calls `.train()`
getOutputs() { // after page load doesn't silently no-op. The promise is shared across
return EMPTY_F32; // calls so we don't kick off two simultaneous loads.
let lazyInitPromise: Promise<void> | null = null;
function lazyInit(): Promise<void> {
if (mlStore.iml) return Promise.resolve();
if (!lazyInitPromise) {
lazyInitPromise = mlStore.initialize().then(() => undefined);
}
return lazyInitPromise;
}
const probe: DebugProbe = {
get __ready(): boolean {
return !!mlStore.iml && mlStore.state.ready;
}, },
getLoss() {
return null; __init(): Promise<void> {
return lazyInit();
}, },
getWeights() {
return EMPTY_F32; getOutputs(): Float32Array {
return mlStore.outputs();
}, },
getExampleCount() {
getLoss(): number | null {
return mlStore.state.lastLoss;
},
getWeights(): Float32Array {
return mlStore.getWeights();
},
getExampleCount(): number {
return mlStore.state.exampleCount;
},
setInputs(x: number, y: number): void {
if (!mlStore.iml) {
void lazyInit();
return;
}
mlStore.iml.inferXY(x, y);
},
thumbsUp(): void {
if (!mlStore.iml) return;
// Stream 10 will replace this with the full RL controller; the
// legacy probe behaviour is "train, then settle". For now we run
// a sync training step.
mlStore.iml.train();
},
thumbsDown(): void {
if (!mlStore.iml) return;
// Default RL noise burst at the playground's typical spread. Stream
// 10 will hook the noise cap from the control surface state.
mlStore.iml.moveWeights(0.1, 0.6);
},
train(): number {
if (!mlStore.iml) {
void lazyInit();
return 0; return 0;
}
return mlStore.iml.train();
}, },
setInputs(_x: number, _y: number) {
/* no-op until ML wired */ async trainAsync(): Promise<number> {
await lazyInit();
if (!mlStore.iml) return 0;
return mlStore.iml.trainAsync();
}, },
thumbsUp() {
/* no-op */ randomise(): void {
if (!mlStore.iml) return;
mlStore.iml.randomiseWeights(0.6);
}, },
thumbsDown() {
/* no-op */ clearExamples(): void {
mlStore.clearExamples();
}, },
train() {
return 0; saveState(): void {
mlStore.saveNow();
}, },
trainAsync() {
return Promise.resolve(0); evalLoss(): number | null {
if (!mlStore.iml) return null;
return mlStore.iml.evalLoss();
}, },
randomise() {
/* no-op */ inferBatch(points: ReadonlyArray<readonly [number, number]>): Float32Array {
if (!mlStore.iml) return new Float32Array(points.length * mlStore.state.outputSize);
return mlStore.iml.inferBatch(points);
}, },
clearExamples() {
/* no-op */ getLayerStats(): Float32Array {
if (!mlStore.iml) return EMPTY_F32;
return mlStore.iml.getLayerStatsFlat();
}, },
saveState() {
/* no-op */
},
evalLoss() {
return null;
},
inferBatch(points) {
// Return a zero array of the right size for at least the inputs.
return new Float32Array(points.length);
},
getLayerStats() {
return EMPTY_F32;
},
__ready: false,
}; };
/** /**
* Install the probe on window. Idempotent. * Install the probe on window. Idempotent — the probe object is a
* * singleton, so capturing `window.__nisps` once is safe across hot
* Stream 10 will replace this with a fully-wired version. Until then the stub * reloads and re-installs.
* advertises `__ready === false`, letting tests skip ML-dependent assertions.
*/ */
export function installDebugProbe(): void { export function installDebugProbe(): void {
if (typeof window === 'undefined') return; if (typeof window === 'undefined') return;
// Always overwrite — later streams may replace it; the marker prevents stale window.__nisps = probe;
// probes from passing tests.
window.__nisps = stubProbe;
} }

193
playground/src/ml/dataset.ts Normal file
View file

@ -0,0 +1,193 @@
/**
* Dataset — JS-side training-example store.
*
* Why duplicate the C++ ring buffer? Two reasons:
* 1. Sample-weight computation (recency / spatial / combined) lives in JS so
* that adjusting weighting modes doesn't burn a WASM round-trip.
* 2. The dataset is part of session state we serialize to localStorage —
* the WASM heap is wiped on reload.
*
* On train() we ship features + labels into WASM via `addExample` calls. The
* order of insertion is preserved; FIFO eviction matches the C++ MLP's
* `dataset_head_` pointer so weighting stays consistent.
*
* The implementation is a faithful TypeScript port of the legacy
* `playground/_archive/js/nisps/dataset.js` with:
* - Float32Array backing instead of `Array<Array<number>>`
* - Stricter types
* - No `withBias` flag (the WASM bindings don't take a bias term)
*/
export type WeightMode = 'global' | 'local' | 'combined' | 'uniform';
export interface ComputeWeightsParams {
/** [0,1] — how strongly to bias toward newest examples (global/combined). */
recencyBias?: number;
/** Current input position, used for local/combined spatial weighting. */
queryInput?: ReadonlyArray<number>;
/** Spatial radius in input space (local/combined). */
radius?: number;
}
export class Dataset {
/** Maximum number of examples retained. FIFO eviction beyond this. */
readonly maxSize: number;
/** Length of feature vectors. Set on first add(); locked thereafter. */
private inputSize_ = 0;
/** Length of label vectors. Set on first add(); locked thereafter. */
private outputSize_ = 0;
/** Flat arrays — entries `[i*inputSize, (i+1)*inputSize)` belong to example i. */
private features_: Float32Array = new Float32Array(0);
private labels_: Float32Array = new Float32Array(0);
private size_ = 0;
constructor(maxSize = 100) {
if (maxSize <= 0) throw new Error('Dataset.maxSize must be > 0');
this.maxSize = maxSize;
}
/** Number of examples currently stored. */
get size(): number {
return this.size_;
}
isEmpty(): boolean {
return this.size_ === 0;
}
/**
* Add a feature/label pair. Returns true on success, false if the
* dimensions don't match a previously-added example.
*
* Eviction: when at capacity, the oldest example is removed (shift),
* then the new one is appended. This matches the legacy JS behaviour
* (and is conceptually equivalent to the C++ side's ring buffer with
* `head_` advancement).
*/
add(features: ReadonlyArray<number>, labels: ReadonlyArray<number>): boolean {
if (this.size_ === 0) {
this.inputSize_ = features.length;
this.outputSize_ = labels.length;
// Allocate full-capacity buffers up front to avoid growth thrash.
this.features_ = new Float32Array(this.maxSize * this.inputSize_);
this.labels_ = new Float32Array(this.maxSize * this.outputSize_);
}
if (features.length !== this.inputSize_ || labels.length !== this.outputSize_) {
return false;
}
if (this.size_ >= this.maxSize) {
// FIFO: shift left in place. This is O(n*dim) and could be replaced
// with a head pointer; for maxSize ≤ a few hundred it's fine.
this.features_.copyWithin(0, this.inputSize_);
this.labels_.copyWithin(0, this.outputSize_);
this.size_ = this.maxSize - 1;
}
const fOff = this.size_ * this.inputSize_;
const lOff = this.size_ * this.outputSize_;
for (let i = 0; i < this.inputSize_; ++i) this.features_[fOff + i] = features[i];
for (let i = 0; i < this.outputSize_; ++i) this.labels_[lOff + i] = labels[i];
this.size_++;
return true;
}
clear(): void {
this.size_ = 0;
}
/** Read-only view of the i-th feature vector. */
feature(i: number): Float32Array {
if (i < 0 || i >= this.size_) throw new RangeError(`feature index ${i} out of bounds`);
return this.features_.subarray(i * this.inputSize_, (i + 1) * this.inputSize_);
}
/** Read-only view of the i-th label vector. */
label(i: number): Float32Array {
if (i < 0 || i >= this.size_) throw new RangeError(`label index ${i} out of bounds`);
return this.labels_.subarray(i * this.outputSize_, (i + 1) * this.outputSize_);
}
/** Flat view of all features (size * inputSize). */
featuresFlat(): Float32Array {
return this.features_.subarray(0, this.size_ * this.inputSize_);
}
/** Flat view of all labels (size * outputSize). */
labelsFlat(): Float32Array {
return this.labels_.subarray(0, this.size_ * this.outputSize_);
}
get inputSize(): number {
return this.inputSize_;
}
get outputSize(): number {
return this.outputSize_;
}
/**
* Compute per-sample training weights. Returns Float32Array (size=this.size)
* normalized to sum to 1. For an empty dataset returns a 0-length array;
* for a singleton, [1.0].
*
* Modes:
* - `uniform` — every weight = 1/n.
* - `global` — exponential recency decay over insertion order.
* - `local` — within `radius` of `queryInput`, suppress older neighbours.
* - `combined` — global × local.
*/
computeWeights(mode: WeightMode = 'uniform', params: ComputeWeightsParams = {}): Float32Array {
const n = this.size_;
if (n === 0) return new Float32Array(0);
if (n === 1) return new Float32Array([1.0]);
const weights = new Float32Array(n).fill(1.0);
if (mode === 'global' || mode === 'combined') {
const bias = params.recencyBias ?? 0.6;
if (bias > 0) {
const decay = 1 - 0.3 * bias;
for (let i = n - 2; i >= 0; --i) weights[i] = weights[i + 1] * decay;
}
}
if ((mode === 'local' || mode === 'combined') && params.queryInput) {
const query = params.queryInput;
const radius = params.radius ?? 0.15;
const radiusSq = radius * radius;
const dim = this.inputSize_;
for (let i = 0; i < n; ++i) {
const fOffI = i * dim;
let distSq = 0;
for (let d = 0; d < dim; ++d) {
const diff = this.features_[fOffI + d] - (query[d] ?? 0);
distSq += diff * diff;
}
if (distSq < radiusSq) {
const proximity = 1 - Math.sqrt(distSq) / radius;
let newerNearby = 0;
for (let j = i + 1; j < n; ++j) {
const fOffJ = j * dim;
let djSq = 0;
for (let d = 0; d < dim; ++d) {
const diff = this.features_[fOffI + d] - this.features_[fOffJ + d];
djSq += diff * diff;
}
if (djSq < radiusSq) ++newerNearby;
}
if (newerNearby > 0) {
weights[i] *= Math.pow(1 - proximity, newerNearby);
}
}
}
}
let sum = 0;
for (let i = 0; i < n; ++i) sum += weights[i];
if (sum > 0) for (let i = 0; i < n; ++i) weights[i] /= sum;
return weights;
}
}

156
playground/src/ml/types.ts Normal file
View file

@ -0,0 +1,156 @@
/**
* TypeScript types matching the C API surface in `nisps/wasm/bindings.cpp`.
*
* These types are intentionally minimal: they describe the JS-visible shape
* of the Emscripten module, the heap views we read/write, and the per-layer
* stats record. They DO NOT mirror any internal C++ struct.
*
* The Emscripten glue produced by `scripts/build-wasm.sh` exposes a factory
* function, `createNispsModule(opts?) => Promise<NispsModule>`, which we
* call from `wasm-iml.ts` and `wasm-worker.ts`.
*/
/**
* Shape of the loaded WASM module — the subset we use.
* Emscripten generates more on it; we type only what we need.
*
* NOTE: `_*` prefixed methods are the raw exported C functions (Emscripten
* naming convention). They take/return numbers (pointers + primitives).
*/
export interface NispsModule {
// Memory views (re-bound after grow).
HEAP8: Int8Array;
HEAP16: Int16Array;
HEAP32: Int32Array;
HEAPU8: Uint8Array;
HEAPU16: Uint16Array;
HEAPU32: Uint32Array;
HEAPF32: Float32Array;
HEAPF64: Float64Array;
_malloc(bytes: number): number;
_free(ptr: number): void;
// ML lifecycle.
// Seed is a uint32_t (not 64-bit) — see bindings.cpp file comment.
_nisps_ml_create(input_size: number, output_size: number, hidden_ptr: number, n_hidden: number, seed: number): number;
_nisps_ml_destroy(ml: number): void;
_nisps_ml_reset(ml: number): void;
// ML inference.
_nisps_ml_set_input(ml: number, idx: number, v: number): void;
_nisps_ml_process(ml: number): void;
_nisps_ml_outputs(ml: number): number; // returns float* into HEAPF32
_nisps_ml_infer_batch(ml: number, points_ptr: number, n_points: number, out_ptr: number): void;
// ML training.
_nisps_ml_add_example(ml: number, features_ptr: number, labels_ptr: number): void;
_nisps_ml_train(ml: number, lr: number, max_iter: number, min_err: number, sample_weights_ptr: number): number;
_nisps_ml_eval_loss(ml: number): number;
// ML examples.
_nisps_ml_clear_examples(ml: number): void;
_nisps_ml_example_count(ml: number): number;
// ML weights.
_nisps_ml_weight_count(ml: number): number;
_nisps_ml_get_weights(ml: number, out_ptr: number): void;
_nisps_ml_set_weights(ml: number, in_ptr: number): void;
_nisps_ml_draw_weights(ml: number, spread: number): void;
_nisps_ml_move_weights(ml: number, speed: number, spread: number, mask_ptr: number): void;
_nisps_ml_get_layer_stats(ml: number, out_ptr: number): void;
_nisps_ml_describe(out_ptr: number): void;
// Engines.
_nisps_engine_create(id_ptr: number, sample_rate: number): number;
_nisps_engine_destroy(engine: number): void;
_nisps_engine_set_params(engine: number, params_ptr: number, n_params: number): void;
_nisps_engine_process_block(
engine: number,
in_l_ptr: number, in_r_ptr: number,
out_l_ptr: number, out_r_ptr: number,
n_samples: number,
): void;
}
/** Factory function exposed by the Emscripten glue. */
export type NispsModuleFactory = (opts?: {
locateFile?: (path: string, prefix: string) => string;
wasmBinary?: ArrayBuffer | Uint8Array;
print?: (msg: string) => void;
printErr?: (msg: string) => void;
}) => Promise<NispsModule>;
/** Architecture descriptor returned from `nisps_ml_describe`. */
export interface MLArchitecture {
inputSize: number;
hidden: [number, number, number];
outputSize: number;
numLayers: number;
}
/** Per-layer weight health record (one per layer). */
export interface LayerStats {
meanAbs: number;
maxAbs: number;
deadFrac: number;
saturatingFrac: number;
}
/** The `engine_id` strings the C++ side recognises. Anything else falls back to "thru". */
export type EngineId =
| 'thru'
| 'paf_synth'
| 'channel_strip'
| 'xiasri'
| 'verb_fx'
| 'memlcelium'
| 'breakor'
| 'elysiamorf'
| 'analysis';
/** Message protocol between main thread and `wasm-worker.ts`. */
export type WorkerRequest =
| {
kind: 'init';
seed: number;
}
| {
kind: 'train';
requestId: number;
// Flat features: nExamples * inputSize floats.
features: Float32Array;
// Flat labels: nExamples * outputSize floats.
labels: Float32Array;
// Optional per-example weights, sums to 1. Empty = uniform.
sampleWeights: Float32Array;
// Current weights to seed worker MLP.
weights: Float32Array;
lr: number;
maxIter: number;
minErr: number;
inputSize: number;
outputSize: number;
}
| {
kind: 'dispose';
};
export type WorkerResponse =
| {
kind: 'ready';
}
| {
kind: 'result';
requestId: number;
loss: number;
weights: Float32Array;
// Loss curve (per-iteration). Currently always empty — the C++ MLP
// exposes loss_history but the WASM bridge does not yet plumb it.
lossHistory: Float32Array;
}
| {
kind: 'error';
requestId: number;
message: string;
};

650
playground/src/ml/wasm-iml.ts Normal file
View file

@ -0,0 +1,650 @@
/**
* WasmIML — main-thread ML interface backed by `nisps.wasm`.
*
* Owns:
* - One `nisps.wasm` instance.
* - One MLP handle.
* - A JS-side `Dataset` (mirrors the C++ ring buffer; see dataset.ts).
* - Pre-allocated heap buffers for inputs/outputs/weights/etc.
* - A lazy `WasmTrainer` worker for off-thread async training.
*
* Threading model (architecture.md §6.4):
* - Inference + sync training run on this main-thread instance.
* - `trainAsync()` spawns/uses a worker which holds a SECOND wasm
* instance; weights round-trip through `getWeights()`/`setWeights()`.
*
* Side effects: every mutation that should be visible to the UI calls into
* `mlStore`. The store is the single source of truth for Solid components.
*
* Concept compatibility: this class re-implements the legacy WasmIML
* surface area documented in `recon/04-playground.md §6` so the existing
* Playwright debug-probe tests can keep passing once the new probe is
* wired up.
*/
import { produce } from 'solid-js/store';
import { mlStore } from '../stores/ml-store';
import { coreBus } from '../stores/bus';
import { Dataset } from './dataset';
import type {
LayerStats,
MLArchitecture,
NispsModule,
NispsModuleFactory,
} from './types';
import { createTrainer, type WasmTrainer } from './wasm-worker';
const NISPS_JS_URL = '/nisps.js';
const NISPS_WASM_URL = '/nisps.wasm';
/** Default architecture matches `nisps/wasm/bindings.cpp` instantiation. */
const DEFAULT_INPUT_SIZE = 2;
const DEFAULT_OUTPUT_SIZE = 126;
let cachedFactory: NispsModuleFactory | null = null;
/**
* Load the Emscripten glue once, cache the factory.
*
* The glue is served from `playground/public/nisps.js` (committed). We
* dynamically import it so the WASM only loads when the ML system is first
* used — `/dev/primitives` doesn't pay for it.
*/
async function getFactory(): Promise<NispsModuleFactory> {
if (cachedFactory) return cachedFactory;
// Vite serves `/nisps.js` as a normal asset; we use a dynamic-eval import
// to avoid Vite trying to resolve it at build time.
const url = new URL(NISPS_JS_URL, window.location.origin).toString();
const mod = await import(/* @vite-ignore */ url);
// Emscripten MODULARIZE=1 default export key is `default`.
// tslint:disable-next-line:no-any
const factory = (mod as { default?: NispsModuleFactory; createNispsModule?: NispsModuleFactory })
.default ?? (mod as { createNispsModule?: NispsModuleFactory }).createNispsModule;
if (!factory) throw new Error('[wasm-iml] nisps.js does not export a module factory');
cachedFactory = factory;
return factory;
}
/** Aligned float-array allocation helper. Returns ptr + a view. */
class HeapBuffer {
readonly ptr: number;
readonly view: Float32Array;
constructor(private mod: NispsModule, public readonly count: number) {
this.ptr = mod._malloc(count * 4);
if (!this.ptr) throw new Error(`malloc(${count * 4}) failed`);
this.view = new Float32Array(mod.HEAPF32.buffer, this.ptr, count);
}
/** After memory growth, refresh the view onto the new ArrayBuffer. */
rebind(): void {
// Re-create the view with the (possibly new) underlying buffer.
Object.defineProperty(this, 'view', {
value: new Float32Array(this.mod.HEAPF32.buffer, this.ptr, this.count),
writable: false,
});
}
free(): void {
this.mod._free(this.ptr);
}
}
class HeapU8 {
readonly ptr: number;
readonly view: Uint8Array;
constructor(private mod: NispsModule, public readonly count: number) {
this.ptr = mod._malloc(count);
if (!this.ptr) throw new Error(`malloc(${count}) failed`);
this.view = new Uint8Array(mod.HEAPU8.buffer, this.ptr, count);
}
rebind(): void {
Object.defineProperty(this, 'view', {
value: new Uint8Array(this.mod.HEAPU8.buffer, this.ptr, this.count),
writable: false,
});
}
free(): void {
this.mod._free(this.ptr);
}
}
export interface WasmIMLOptions {
inputSize?: number;
outputSize?: number;
hiddenLayers?: ReadonlyArray<number>;
seed?: number;
/** localStorage key the loaded weights/dataset will be persisted under. */
storageKey?: string;
maxExamples?: number;
}
export class WasmIML {
// WASM binding state.
private module!: NispsModule;
private mlHandle = 0;
private weightCount_ = 0;
// Architecture descriptor (resolved post-init from the WASM build).
private arch_: MLArchitecture = {
inputSize: DEFAULT_INPUT_SIZE,
hidden: [10, 14, 18],
outputSize: DEFAULT_OUTPUT_SIZE,
numLayers: 4,
};
// Pre-allocated heap buffers.
private featuresBuf!: HeapBuffer;
private labelsBuf!: HeapBuffer;
private weightsBuf!: HeapBuffer;
private statsBuf!: HeapBuffer;
private batchInBuf!: HeapBuffer;
private batchOutBuf!: HeapBuffer;
private pinMaskBuf!: HeapU8;
private describeBuf!: HeapBuffer; // 6 ints, reused as 6 floats on the heap is wrong;
// We use HEAP32 directly via a tiny scratch malloc:
private describePtr = 0;
// JS-side state.
readonly dataset: Dataset;
private lastLoss_: number | null = null;
private trainer: WasmTrainer | null = null;
private storageKey: string;
private saveTimer: number | null = null;
private destroyed = false;
static MAX_BATCH = 4096;
private constructor(opts: WasmIMLOptions) {
this.dataset = new Dataset(opts.maxExamples ?? 100);
this.storageKey = opts.storageKey ?? 'nisps:wasm-iml';
}
/**
* Async factory. Loads the wasm, creates the MLP handle, allocates heap
* buffers, and rehydrates persisted state.
*/
static async create(opts: WasmIMLOptions = {}): Promise<WasmIML> {
const inst = new WasmIML(opts);
await inst.init_(opts);
return inst;
}
private async init_(opts: WasmIMLOptions): Promise<void> {
const factory = await getFactory();
this.module = await factory({
// Vite serves /nisps.wasm at the root; the default locateFile would
// resolve relative to nisps.js (also at root) so this is the same
// result, but explicit is better.
locateFile: (path: string) => {
if (path.endsWith('.wasm')) return new URL(NISPS_WASM_URL, window.location.origin).toString();
return path;
},
});
// Resolve architecture via the WASM module (compile-time fixed; we
// stash the values for callers that need them).
this.describePtr = this.module._malloc(6 * 4);
this.module._nisps_ml_describe(this.describePtr);
const dims = new Int32Array(this.module.HEAP32.buffer, this.describePtr, 6);
this.arch_ = {
inputSize: dims[0],
hidden: [dims[1], dims[2], dims[3]],
outputSize: dims[4],
numLayers: dims[5],
};
// Caller-supplied dimensions are accepted but ignored; warn if mismatch.
const wantedIn = opts.inputSize ?? this.arch_.inputSize;
const wantedOut = opts.outputSize ?? this.arch_.outputSize;
if (wantedIn !== this.arch_.inputSize || wantedOut !== this.arch_.outputSize) {
console.warn(
`[wasm-iml] requested ${wantedIn}->${wantedOut} but WASM build is fixed at ` +
`${this.arch_.inputSize}->${this.arch_.outputSize}; extras are ignored.`,
);
}
// Create the MLP. We pass dummy hidden ptr/count — the binding ignores them.
const seed = (opts.seed ?? (Date.now() >>> 0)) >>> 0;
this.mlHandle = this.module._nisps_ml_create(
this.arch_.inputSize,
this.arch_.outputSize,
0, // hidden ptr (unused)
0, // n_hidden (unused)
seed,
);
if (!this.mlHandle) throw new Error('[wasm-iml] nisps_ml_create returned null');
this.weightCount_ = this.module._nisps_ml_weight_count(this.mlHandle);
// Heap buffers (created after we know the architecture).
this.featuresBuf = new HeapBuffer(this.module, this.arch_.inputSize);
this.labelsBuf = new HeapBuffer(this.module, this.arch_.outputSize);
this.weightsBuf = new HeapBuffer(this.module, this.weightCount_);
this.statsBuf = new HeapBuffer(this.module, this.arch_.numLayers * 4);
this.batchInBuf = new HeapBuffer(this.module, WasmIML.MAX_BATCH * this.arch_.inputSize);
this.batchOutBuf = new HeapBuffer(this.module, WasmIML.MAX_BATCH * this.arch_.outputSize);
this.pinMaskBuf = new HeapU8(this.module, this.arch_.outputSize);
// Push initial state to the store.
mlStore.__setState(produce((s) => {
s.inputSize = this.arch_.inputSize;
s.outputSize = this.arch_.outputSize;
s.exampleCount = 0;
s.lastLoss = null;
s.lossHistory = [];
s.training = false;
s.ready = true;
}));
mlStore.__setOutputs(new Float32Array(this.arch_.outputSize));
this.publishWeights_();
// Load persisted state if present (best-effort).
this.tryLoadFromStorage_();
}
// -------------------------------------------------------------------
// Lifecycle
// -------------------------------------------------------------------
dispose(): void {
if (this.destroyed) return;
this.destroyed = true;
if (this.saveTimer !== null) {
clearTimeout(this.saveTimer);
this.saveTimer = null;
}
if (this.trainer) {
this.trainer.dispose();
this.trainer = null;
}
if (this.module && this.mlHandle) {
this.module._nisps_ml_destroy(this.mlHandle);
this.mlHandle = 0;
}
if (this.featuresBuf) this.featuresBuf.free();
if (this.labelsBuf) this.labelsBuf.free();
if (this.weightsBuf) this.weightsBuf.free();
if (this.statsBuf) this.statsBuf.free();
if (this.batchInBuf) this.batchInBuf.free();
if (this.batchOutBuf) this.batchOutBuf.free();
if (this.pinMaskBuf) this.pinMaskBuf.free();
if (this.describePtr) this.module._free(this.describePtr);
mlStore.__setState(produce((s) => {
s.ready = false;
}));
}
get architecture(): MLArchitecture {
return this.arch_;
}
get weightCount(): number {
return this.weightCount_;
}
get exampleCount(): number {
return this.dataset.size;
}
get lastLoss(): number | null {
return this.lastLoss_;
}
// -------------------------------------------------------------------
// Inference
// -------------------------------------------------------------------
setInput(idx: number, value: number): void {
this.module._nisps_ml_set_input(this.mlHandle, idx, value);
}
process(): Float32Array {
this.module._nisps_ml_process(this.mlHandle);
const ptr = this.module._nisps_ml_outputs(this.mlHandle);
// Copy out so the caller can hold onto it across memory growth.
const view = new Float32Array(this.module.HEAPF32.buffer, ptr, this.arch_.outputSize);
const out = new Float32Array(view); // copy
mlStore.__setOutputs(out);
return out;
}
/** Convenience: setInput(0,x); setInput(1,y); process(). */
inferXY(x: number, y: number): Float32Array {
this.setInput(0, x);
this.setInput(1, y);
return this.process();
}
/**
* Batch inference. `points` is an array of [x,y] tuples (or any vector
* length <= inputSize; trailing entries zero-padded). Returns a flat
* Float32Array of length n * outputSize.
*
* Larger requests than `MAX_BATCH` are chunked transparently.
*/
inferBatch(points: ReadonlyArray<ReadonlyArray<number>>): Float32Array {
const n = points.length;
const inSz = this.arch_.inputSize;
const outSz = this.arch_.outputSize;
const result = new Float32Array(n * outSz);
let written = 0;
for (let offset = 0; offset < n; offset += WasmIML.MAX_BATCH) {
const chunk = Math.min(WasmIML.MAX_BATCH, n - offset);
// Pack into batchInBuf.
for (let i = 0; i < chunk; ++i) {
const src = points[offset + i];
const base = i * inSz;
for (let j = 0; j < inSz; ++j) this.batchInBuf.view[base + j] = src[j] ?? 0;
}
this.module._nisps_ml_infer_batch(
this.mlHandle,
this.batchInBuf.ptr,
chunk,
this.batchOutBuf.ptr,
);
// Copy out into result.
const slice = this.batchOutBuf.view.subarray(0, chunk * outSz);
result.set(slice, written);
written += chunk * outSz;
}
return result;
}
// -------------------------------------------------------------------
// Training
// -------------------------------------------------------------------
/**
* Add a feature/label pair to BOTH the JS dataset and the WASM ring
* buffer. The two stay in sync because every train() call pushes the
* full JS dataset back into WASM (in case of weight recompute, undo
* restore, etc.). For the sake of correctness, we re-sync on each add
* too — cheap relative to training.
*/
addExample(features: ReadonlyArray<number>, labels: ReadonlyArray<number>): boolean {
const ok = this.dataset.add(features, labels);
if (!ok) return false;
this.copyExampleToWasm_(features, labels);
mlStore.__setState(produce((s) => {
s.exampleCount = this.dataset.size;
}));
coreBus.emit('ml.example_added', { count: this.dataset.size });
this.scheduleSave_();
return true;
}
private copyExampleToWasm_(features: ReadonlyArray<number>, labels: ReadonlyArray<number>): void {
const fv = this.featuresBuf.view;
const lv = this.labelsBuf.view;
const inSz = this.arch_.inputSize;
const outSz = this.arch_.outputSize;
for (let i = 0; i < inSz; ++i) fv[i] = features[i] ?? 0;
for (let i = 0; i < outSz; ++i) lv[i] = labels[i] ?? 0;
this.module._nisps_ml_add_example(this.mlHandle, this.featuresBuf.ptr, this.labelsBuf.ptr);
}
/**
* Synchronous training. Returns the final loss (also stored in
* `lastLoss`). Updates `mlStore.lastLoss` and emits `ml.trained`.
*
* Caller can pass per-sample weights; if omitted the WASM side uses
* uniform 1/n weighting.
*/
train(lr = 1.0, maxIter = 1000, minErr = 0.001, sampleWeights?: Float32Array): number {
if (this.dataset.isEmpty()) {
this.lastLoss_ = 0;
mlStore.__setState(produce((s) => { s.lastLoss = 0; }));
return 0;
}
let weightsPtr = 0;
let weightsHandle: HeapBuffer | null = null;
if (sampleWeights && sampleWeights.length === this.dataset.size) {
weightsHandle = new HeapBuffer(this.module, sampleWeights.length);
weightsHandle.view.set(sampleWeights);
weightsPtr = weightsHandle.ptr;
}
mlStore.__setState(produce((s) => { s.training = true; }));
let loss = 0;
try {
loss = this.module._nisps_ml_train(this.mlHandle, lr, maxIter, minErr, weightsPtr);
} finally {
if (weightsHandle) weightsHandle.free();
mlStore.__setState(produce((s) => { s.training = false; }));
}
this.lastLoss_ = loss;
mlStore.__setState(produce((s) => {
s.lastLoss = loss;
// The C++ MLP stores per-iter history but we don't currently expose
// it via the WASM bindings. Stream 9 may add nisps_ml_loss_history.
s.lossHistory = [loss];
}));
this.publishWeights_();
coreBus.emit('ml.trained', { loss });
this.scheduleSave_();
return loss;
}
/**
* Async training via worker. The worker holds a SECOND wasm instance,
* receives current weights + dataset, runs SGD, and returns updated
* weights. Main-thread weights are then `setWeights()`-restored.
*/
async trainAsync(lr = 1.0, maxIter = 1000, minErr = 0.001, sampleWeights?: Float32Array): Promise<number> {
if (this.dataset.isEmpty()) {
this.lastLoss_ = 0;
return 0;
}
if (!this.trainer) this.trainer = await createTrainer();
const weights = this.getWeights();
const features = new Float32Array(this.dataset.featuresFlat());
const labels = new Float32Array(this.dataset.labelsFlat());
const sw = sampleWeights ? new Float32Array(sampleWeights) : new Float32Array(0);
mlStore.__setState(produce((s) => { s.training = true; }));
try {
const result = await this.trainer.train({
weights,
features,
labels,
sampleWeights: sw,
lr,
maxIter,
minErr,
inputSize: this.arch_.inputSize,
outputSize: this.arch_.outputSize,
});
this.setWeights(result.weights);
this.lastLoss_ = result.loss;
mlStore.__setState(produce((s) => {
s.lastLoss = result.loss;
s.lossHistory = Array.from(result.lossHistory);
}));
coreBus.emit('ml.trained', { loss: result.loss });
this.scheduleSave_();
return result.loss;
} finally {
mlStore.__setState(produce((s) => { s.training = false; }));
}
}
/** Non-destructive loss query. */
evalLoss(): number {
return this.module._nisps_ml_eval_loss(this.mlHandle);
}
clearExamples(): void {
this.dataset.clear();
this.module._nisps_ml_clear_examples(this.mlHandle);
mlStore.__setState(produce((s) => { s.exampleCount = 0; }));
coreBus.emit('ml.examples_cleared', undefined);
this.scheduleSave_();
}
// -------------------------------------------------------------------
// RL ops
// -------------------------------------------------------------------
randomiseWeights(spread = 0.6): void {
this.module._nisps_ml_draw_weights(this.mlHandle, spread);
this.publishWeights_();
coreBus.emit('ml.delta_update', { reason: 'randomize' });
this.scheduleSave_();
}
moveWeights(speed: number, spread: number, pinMask?: Uint8Array): void {
let maskPtr = 0;
if (pinMask) {
const sz = Math.min(pinMask.length, this.arch_.outputSize);
for (let i = 0; i < sz; ++i) this.pinMaskBuf.view[i] = pinMask[i];
for (let i = sz; i < this.arch_.outputSize; ++i) this.pinMaskBuf.view[i] = 0;
maskPtr = this.pinMaskBuf.ptr;
}
this.module._nisps_ml_move_weights(this.mlHandle, speed, spread, maskPtr);
this.publishWeights_();
// The caller (RL handler) decides whether this is a thumbs-up/down;
// we emit a generic delta_update.
coreBus.emit('ml.delta_update', { reason: 'thumbs_down' });
}
// -------------------------------------------------------------------
// Weights I/O
// -------------------------------------------------------------------
getWeights(): Float32Array {
this.module._nisps_ml_get_weights(this.mlHandle, this.weightsBuf.ptr);
// Copy out so caller can mutate freely.
return new Float32Array(this.weightsBuf.view);
}
setWeights(w: Float32Array | Uint8Array): void {
if (w.length < this.weightCount_) {
throw new Error(`setWeights: expected ${this.weightCount_} floats, got ${w.length}`);
}
this.weightsBuf.view.set(w as Float32Array, 0);
this.module._nisps_ml_set_weights(this.mlHandle, this.weightsBuf.ptr);
this.publishWeights_();
}
/** Per-layer weight stats. Returns one record per layer. */
getLayerStats(): LayerStats[] {
this.module._nisps_ml_get_layer_stats(this.mlHandle, this.statsBuf.ptr);
const out: LayerStats[] = [];
for (let i = 0; i < this.arch_.numLayers; ++i) {
const base = i * 4;
out.push({
meanAbs: this.statsBuf.view[base],
maxAbs: this.statsBuf.view[base + 1],
deadFrac: this.statsBuf.view[base + 2],
saturatingFrac: this.statsBuf.view[base + 3],
});
}
return out;
}
/** Flat layer-stats Float32Array (numLayers * 4). For probe API. */
getLayerStatsFlat(): Float32Array {
this.module._nisps_ml_get_layer_stats(this.mlHandle, this.statsBuf.ptr);
return new Float32Array(this.statsBuf.view);
}
// -------------------------------------------------------------------
// Misc
// -------------------------------------------------------------------
reset(): void {
this.module._nisps_ml_reset(this.mlHandle);
this.dataset.clear();
this.lastLoss_ = null;
mlStore.__setState(produce((s) => {
s.exampleCount = 0;
s.lastLoss = null;
s.lossHistory = [];
}));
this.publishWeights_();
coreBus.emit('ml.examples_cleared', undefined);
this.scheduleSave_();
}
// -------------------------------------------------------------------
// Persistence
// -------------------------------------------------------------------
private scheduleSave_(): void {
if (this.saveTimer !== null) clearTimeout(this.saveTimer);
this.saveTimer = window.setTimeout(() => this.saveNow(), 500);
}
saveNow(): void {
if (this.destroyed) return;
if (this.saveTimer !== null) {
clearTimeout(this.saveTimer);
this.saveTimer = null;
}
try {
const weights = this.getWeights();
const payload = {
v: 1,
arch: this.arch_,
weights: Array.from(weights),
features: Array.from(this.dataset.featuresFlat()),
labels: Array.from(this.dataset.labelsFlat()),
size: this.dataset.size,
lastLoss: this.lastLoss_,
};
localStorage.setItem(this.storageKey, JSON.stringify(payload));
} catch (err) {
// localStorage might be full or unavailable; not fatal.
console.warn('[wasm-iml] saveNow failed:', err);
}
}
private tryLoadFromStorage_(): void {
try {
const raw = localStorage.getItem(this.storageKey);
if (!raw) return;
const payload = JSON.parse(raw) as {
v: number;
weights: number[];
features: number[];
labels: number[];
size: number;
lastLoss: number | null;
};
if (payload.v !== 1) return;
// Restore dataset (rebuild via add()).
const inSz = this.arch_.inputSize;
const outSz = this.arch_.outputSize;
if (payload.size > 0 && payload.features.length === payload.size * inSz &&
payload.labels.length === payload.size * outSz) {
for (let i = 0; i < payload.size; ++i) {
const f = payload.features.slice(i * inSz, (i + 1) * inSz);
const l = payload.labels.slice(i * outSz, (i + 1) * outSz);
this.dataset.add(f, l);
// Also push to the WASM ring buffer.
this.copyExampleToWasm_(f, l);
}
}
// Restore weights.
if (payload.weights.length === this.weightCount_) {
this.setWeights(new Float32Array(payload.weights));
}
this.lastLoss_ = payload.lastLoss;
mlStore.__setState(produce((s) => {
s.exampleCount = this.dataset.size;
s.lastLoss = this.lastLoss_;
}));
} catch (err) {
console.warn('[wasm-iml] tryLoadFromStorage failed:', err);
}
}
// -------------------------------------------------------------------
// Helpers
// -------------------------------------------------------------------
private publishWeights_(): void {
const w = this.getWeights();
mlStore.__setWeights(w);
}
}

334
playground/src/ml/wasm-worker.ts Normal file
View file

@ -0,0 +1,334 @@
/**
* Disposable Web Worker that runs SGD off the main thread.
*
* The worker holds its own `nisps.wasm` instance (architecture.md §6.4).
* The main thread sends:
* - current weights (so the worker is in the same state as the UI),
* - dataset (features + labels),
* - SGD hyperparameters,
* and receives updated weights + final loss.
*
* This module exposes:
* - `createTrainer()` - factory that spawns the worker, loads its WASM,
* and returns a `WasmTrainer` handle.
* - The worker entry-point itself (when this file runs in a Worker).
*
* The worker is implemented inline so a single TS file becomes both the
* main-thread API and the worker bundle. Vite's `new Worker(new URL(...,
* import.meta.url))` pattern packs it correctly.
*
* Lifecycle: each `WasmTrainer` is disposable via `.dispose()` which
* terminates the worker. Tests should always dispose to avoid leaks.
*/
import type { NispsModule, NispsModuleFactory, WorkerRequest, WorkerResponse } from './types';
// ---------------------------------------------------------------------------
// Main-thread side
// ---------------------------------------------------------------------------
export interface TrainArgs {
weights: Float32Array;
features: Float32Array;
labels: Float32Array;
/** Optional; pass empty for uniform weighting. */
sampleWeights: Float32Array;
lr: number;
maxIter: number;
minErr: number;
inputSize: number;
outputSize: number;
}
export interface TrainResult {
loss: number;
weights: Float32Array;
lossHistory: Float32Array;
}
export class WasmTrainer {
private worker: Worker;
private nextId = 1;
private pending = new Map<number, { resolve: (r: TrainResult) => void; reject: (e: unknown) => void }>();
private disposed = false;
static async create(): Promise<WasmTrainer> {
const trainer = new WasmTrainer();
await trainer.init_();
return trainer;
}
private constructor() {
this.worker = new Worker(new URL('./wasm-worker.ts', import.meta.url), { type: 'module' });
this.worker.onmessage = (ev) => this.onMessage_(ev.data as WorkerResponse);
this.worker.onerror = (ev) => {
// Fail any pending requests.
for (const { reject } of this.pending.values()) reject(ev.message ?? 'worker error');
this.pending.clear();
};
}
private init_(): Promise<void> {
return new Promise((resolve, reject) => {
const handler = (ev: MessageEvent) => {
const msg = ev.data as WorkerResponse;
if (msg.kind === 'ready') {
this.worker.removeEventListener('message', handler);
resolve();
} else if (msg.kind === 'error') {
this.worker.removeEventListener('message', handler);
reject(new Error(msg.message));
}
};
this.worker.addEventListener('message', handler);
const seed = (Date.now() ^ Math.floor(Math.random() * 0xffffffff)) >>> 0;
this.worker.postMessage({ kind: 'init', seed } satisfies WorkerRequest);
});
}
train(args: TrainArgs): Promise<TrainResult> {
if (this.disposed) return Promise.reject(new Error('WasmTrainer disposed'));
const requestId = this.nextId++;
return new Promise((resolve, reject) => {
this.pending.set(requestId, { resolve, reject });
const msg: WorkerRequest = {
kind: 'train',
requestId,
weights: args.weights,
features: args.features,
labels: args.labels,
sampleWeights: args.sampleWeights,
lr: args.lr,
maxIter: args.maxIter,
minErr: args.minErr,
inputSize: args.inputSize,
outputSize: args.outputSize,
};
// Transfer all the typed-array buffers we no longer need on the
// main thread; faster than copying. Caller has already cloned.
this.worker.postMessage(msg, [
args.weights.buffer,
args.features.buffer,
args.labels.buffer,
args.sampleWeights.buffer,
]);
});
}
dispose(): void {
if (this.disposed) return;
this.disposed = true;
try {
this.worker.postMessage({ kind: 'dispose' } satisfies WorkerRequest);
} catch {
/* ignore */
}
this.worker.terminate();
for (const { reject } of this.pending.values()) reject(new Error('disposed'));
this.pending.clear();
}
private onMessage_(msg: WorkerResponse): void {
if (msg.kind === 'result') {
const p = this.pending.get(msg.requestId);
if (p) {
this.pending.delete(msg.requestId);
p.resolve({ loss: msg.loss, weights: msg.weights, lossHistory: msg.lossHistory });
}
} else if (msg.kind === 'error') {
const p = this.pending.get(msg.requestId);
if (p) {
this.pending.delete(msg.requestId);
p.reject(new Error(msg.message));
}
}
// 'ready' handled in init_().
}
}
export function createTrainer(): Promise<WasmTrainer> {
return WasmTrainer.create();
}
// ---------------------------------------------------------------------------
// Worker-thread side
// ---------------------------------------------------------------------------
//
// When this module is loaded in a Worker, `self` is `WorkerGlobalScope` and
// `window` is undefined. We use that as the dispatch.
//
// We import the wasm via a same-origin fetch (no DOM available so we can't
// use a regular import URL — but Vite's bundler treats `new Worker(...)`
// specially and `?url` imports work for assets).
// Inside a Worker, `self` is a `DedicatedWorkerGlobalScope`. To keep TS
// happy in both build contexts we use a structural cast.
declare const self: {
postMessage: (msg: unknown, transfer?: Transferable[]) => void;
addEventListener: (event: string, handler: (ev: MessageEvent) => void) => void;
location: { origin: string };
importScripts?: unknown;
};
const isWorker =
typeof window === 'undefined' &&
typeof self !== 'undefined' &&
typeof (self as { importScripts?: unknown }).importScripts !== 'undefined';
if (isWorker) {
// Module-level state in worker scope.
let mod: NispsModule | null = null;
let mlHandle = 0;
let weightCount = 0;
// Heap buffers (allocated on first train).
let weightsPtr = 0;
let weightsViewLen = 0;
let featuresPtr = 0;
let featuresLen = 0;
let labelsPtr = 0;
let labelsLen = 0;
let sampleWeightsPtr = 0;
let sampleWeightsLen = 0;
async function loadModule(seed: number): Promise<void> {
// Same-origin fetch to /nisps.js. The worker is served by the dev
// server with COOP/COEP set, so this works.
const factoryMod = await import(/* @vite-ignore */ new URL('/nisps.js', self.location.origin).toString());
const factory: NispsModuleFactory =
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(factoryMod as any).default ?? (factoryMod as any).createNispsModule;
mod = await factory({
locateFile: (path: string) => {
if (path.endsWith('.wasm')) return new URL('/nisps.wasm', self.location.origin).toString();
return path;
},
});
mlHandle = mod._nisps_ml_create(0, 0, 0, 0, seed >>> 0);
weightCount = mod._nisps_ml_weight_count(mlHandle);
}
function ensureBuffers(features: Float32Array, labels: Float32Array, sampleWeights: Float32Array, weights: Float32Array): void {
if (!mod) throw new Error('worker module not loaded');
if (weightsViewLen !== weightCount) {
if (weightsPtr) mod._free(weightsPtr);
weightsPtr = mod._malloc(weightCount * 4);
weightsViewLen = weightCount;
}
if (features.length !== featuresLen) {
if (featuresPtr) mod._free(featuresPtr);
featuresPtr = mod._malloc(features.length * 4);
featuresLen = features.length;
}
if (labels.length !== labelsLen) {
if (labelsPtr) mod._free(labelsPtr);
labelsPtr = mod._malloc(labels.length * 4);
labelsLen = labels.length;
}
if (sampleWeights.length !== sampleWeightsLen) {
if (sampleWeightsPtr) mod._free(sampleWeightsPtr);
sampleWeightsPtr = sampleWeights.length > 0 ? mod._malloc(sampleWeights.length * 4) : 0;
sampleWeightsLen = sampleWeights.length;
}
new Float32Array(mod.HEAPF32.buffer, weightsPtr, weightCount).set(weights);
new Float32Array(mod.HEAPF32.buffer, featuresPtr, features.length).set(features);
new Float32Array(mod.HEAPF32.buffer, labelsPtr, labels.length).set(labels);
if (sampleWeightsPtr) {
new Float32Array(mod.HEAPF32.buffer, sampleWeightsPtr, sampleWeights.length).set(sampleWeights);
}
}
function trainOnce(req: Extract<WorkerRequest, { kind: 'train' }>): WorkerResponse {
if (!mod) {
return { kind: 'error', requestId: req.requestId, message: 'worker not initialised' };
}
try {
ensureBuffers(req.features, req.labels, req.sampleWeights, req.weights);
// Push current weights into our MLP.
mod._nisps_ml_set_weights(mlHandle, weightsPtr);
// Seed the example ring buffer. We must clear first because past
// train calls may have left examples there.
mod._nisps_ml_clear_examples(mlHandle);
const inSz = req.inputSize;
const outSz = req.outputSize;
const n = req.features.length / inSz;
// Allocate small per-example scratch (re-used across iterations of
// this loop).
// We use stack-equivalents by allocating once, then shifting pointers.
for (let i = 0; i < n; ++i) {
const fPtr = featuresPtr + i * inSz * 4;
const lPtr = labelsPtr + i * outSz * 4;
mod._nisps_ml_add_example(mlHandle, fPtr, lPtr);
}
// Run training.
const swPtr = req.sampleWeights.length > 0 ? sampleWeightsPtr : 0;
const loss = mod._nisps_ml_train(mlHandle, req.lr, req.maxIter, req.minErr, swPtr);
// Read out final weights.
mod._nisps_ml_get_weights(mlHandle, weightsPtr);
const view = new Float32Array(mod.HEAPF32.buffer, weightsPtr, weightCount);
const outWeights = new Float32Array(view); // copy
// Loss history not yet plumbed via WASM; emit just final loss.
const lossHistory = new Float32Array([loss]);
return {
kind: 'result',
requestId: req.requestId,
loss,
weights: outWeights,
lossHistory,
};
} catch (err) {
return {
kind: 'error',
requestId: req.requestId,
message: err instanceof Error ? err.message : String(err),
};
}
}
function disposeModule(): void {
if (!mod) return;
if (mlHandle) {
mod._nisps_ml_destroy(mlHandle);
mlHandle = 0;
}
if (weightsPtr) { mod._free(weightsPtr); weightsPtr = 0; }
if (featuresPtr) { mod._free(featuresPtr); featuresPtr = 0; }
if (labelsPtr) { mod._free(labelsPtr); labelsPtr = 0; }
if (sampleWeightsPtr) { mod._free(sampleWeightsPtr); sampleWeightsPtr = 0; }
mod = null;
}
self.addEventListener('message', async (ev: MessageEvent<WorkerRequest>) => {
const req = ev.data;
if (req.kind === 'init') {
try {
await loadModule(req.seed);
self.postMessage({ kind: 'ready' } satisfies WorkerResponse);
} catch (err) {
self.postMessage({
kind: 'error',
requestId: 0,
message: err instanceof Error ? err.message : String(err),
} satisfies WorkerResponse);
}
} else if (req.kind === 'train') {
const res = trainOnce(req);
// Transfer weights back to main thread to avoid copy.
if (res.kind === 'result') {
self.postMessage(res, [res.weights.buffer, res.lossHistory.buffer]);
} else {
self.postMessage(res);
}
} else if (req.kind === 'dispose') {
disposeModule();
// Worker terminates from main thread side via .terminate().
}
});
}

177
playground/src/stores/ml-store.ts
View file

@ -1,19 +1,27 @@
/** /**
* ML store — placeholder shape for the ML engine state. * ML store — Solid-side state for the WASM-backed ML engine.
* *
* Stream 7 wires WASM under this. For now the methods that mutate the engine * Stream 7 wires this to a `WasmIML` instance. The store still owns the
* throw `not implemented`. The shape of the store and the signal types are * Solid-reactive state (sizes, loss, training flag, ready flag) and a
* final — modes and primitives can read them. * Float32Array signal for outputs. The WasmIML class drives the values
* via `__set*` setters — kept exported so the WasmIML implementation can
* write through without going through reactive accessors.
* *
* Why a Solid store + a separate Float32Array signal: * Why two layers (store + class)?
* - `createStore` is great for object-like state with fine reactivity. * - The class encapsulates the WASM heap, buffers, and worker.
* - Float32Array outputs are large and frequently updated; `createSignal` * - The store is the consumer-facing surface for Solid components and
* with explicit reference replacement is cheaper. * the debug probe. Components shouldn't reach into the WASM directly.
*
* Singleton model: there is exactly one `WasmIML` per browser tab, owned
* by the store. `initialize()` creates it; subsequent calls return the
* existing one. This matches the legacy playground.
*/ */
import { createSignal, type Accessor } from 'solid-js'; import { createSignal, type Accessor } from 'solid-js';
import { createStore, produce } from 'solid-js/store'; import { createStore, produce } from 'solid-js/store';
import { coreBus } from './bus'; import { coreBus } from './bus';
import type { WasmIML } from '../ml/wasm-iml';
import type { LayerStats } from '../ml/types';
export interface MLStoreState { export interface MLStoreState {
exampleCount: number; exampleCount: number;
@ -32,11 +40,6 @@ export interface MLStoreState {
} }
const EMPTY_OUTPUTS = new Float32Array(0); const EMPTY_OUTPUTS = new Float32Array(0);
const NOT_IMPLEMENTED = (op: string): never => {
throw new Error(
`[ml-store] ${op} not implemented in stream-8 scaffold; awaits stream 7 (WASM bindings)`
);
};
const [state, setState] = createStore<MLStoreState>({ const [state, setState] = createStore<MLStoreState>({
exampleCount: 0, exampleCount: 0,
@ -56,61 +59,163 @@ const [weights, setWeights] = createSignal<Float32Array>(EMPTY_OUTPUTS, {
equals: false, equals: false,
}); });
// Singleton WasmIML, lazily created by initialize().
let imlInstance: WasmIML | null = null;
let initPromise: Promise<WasmIML> | null = null;
function requireIML(op: string): WasmIML {
if (!imlInstance) {
throw new Error(
`[ml-store] ${op} called before initialize() — call mlStore.initialize() first`,
);
}
return imlInstance;
}
export const mlStore = { export const mlStore = {
// ---- read ---- // ---- read ----
state, state,
outputs: outputs as Accessor<Float32Array>, outputs: outputs as Accessor<Float32Array>,
weights: weights as Accessor<Float32Array>, weights: weights as Accessor<Float32Array>,
// ---- internal setters (used by future WASM wiring; exposed for stub /** Direct access to the WasmIML instance (null until initialize() resolves). */
// wiring during this stream so primitive demos can drive values) ---- get iml(): WasmIML | null {
return imlInstance;
},
// ---- internal setters (used by WasmIML to push state into the store) ----
__setOutputs: setOutputs, __setOutputs: setOutputs,
__setState: setState, __setState: setState,
__setWeights: setWeights, __setWeights: setWeights,
// ---- ML lifecycle (stubbed) ---- // ---- ML lifecycle ----
initialize(_inputSize: number, _outputSize: number): Promise<void> {
return NOT_IMPLEMENTED('initialize'); /**
* Load the WASM and create the singleton WasmIML. Idempotent: returns
* the cached instance on subsequent calls.
*
* `inputSize` / `outputSize` are accepted for forward compatibility but
* ignored if they don't match the WASM build's compile-time architecture.
*/
async initialize(inputSize?: number, outputSize?: number): Promise<WasmIML> {
if (imlInstance) return imlInstance;
if (initPromise) return initPromise;
// Lazy-import to keep the WASM glue out of the bundle until needed.
initPromise = (async () => {
const { WasmIML: WasmIMLCtor } = await import('../ml/wasm-iml');
const inst = await WasmIMLCtor.create({
inputSize,
outputSize,
});
imlInstance = inst;
return inst;
})();
return initPromise;
}, },
setInput(_idx: number, _value: number): void {
NOT_IMPLEMENTED('setInput'); setInput(idx: number, value: number): void {
requireIML('setInput').setInput(idx, value);
}, },
process(): void { process(): void {
NOT_IMPLEMENTED('process'); requireIML('process').process();
}, },
addExample(_features: ReadonlyArray<number>, _labels: ReadonlyArray<number>): void {
NOT_IMPLEMENTED('addExample'); inferXY(x: number, y: number): Float32Array {
return requireIML('inferXY').inferXY(x, y);
}, },
train(_lr?: number, _maxIter?: number): number {
return NOT_IMPLEMENTED('train'); addExample(features: ReadonlyArray<number>, labels: ReadonlyArray<number>): boolean {
return requireIML('addExample').addExample(features, labels);
}, },
trainAsync(_lr?: number, _maxIter?: number): Promise<number> {
return NOT_IMPLEMENTED('trainAsync'); train(lr?: number, maxIter?: number, minErr?: number, sampleWeights?: Float32Array): number {
return requireIML('train').train(lr, maxIter, minErr, sampleWeights);
}, },
drawWeights(_spread: number): void {
NOT_IMPLEMENTED('drawWeights'); trainAsync(lr?: number, maxIter?: number, minErr?: number, sampleWeights?: Float32Array): Promise<number> {
return requireIML('trainAsync').trainAsync(lr, maxIter, minErr, sampleWeights);
}, },
moveWeights(_speed: number, _spread: number, _pinMask?: Uint8Array): void {
NOT_IMPLEMENTED('moveWeights'); drawWeights(spread: number): void {
requireIML('drawWeights').randomiseWeights(spread);
}, },
moveWeights(speed: number, spread: number, pinMask?: Uint8Array): void {
requireIML('moveWeights').moveWeights(speed, spread, pinMask);
},
evalLoss(): number | null { evalLoss(): number | null {
return null; if (!imlInstance) return null;
return imlInstance.evalLoss();
}, },
inferBatch(_points: ReadonlyArray<readonly [number, number]>): Float32Array {
return NOT_IMPLEMENTED('inferBatch'); inferBatch(points: ReadonlyArray<readonly [number, number]>): Float32Array {
if (!imlInstance) {
// Until the WASM is up the probe gets a zero array of the expected
// total size. Matches the stub behaviour expected by tests.
return new Float32Array(points.length * state.outputSize);
}
return imlInstance.inferBatch(points);
}, },
getLayerStats(): Float32Array { getLayerStats(): Float32Array {
return EMPTY_OUTPUTS; if (!imlInstance) return EMPTY_OUTPUTS;
return imlInstance.getLayerStatsFlat();
}, },
getLayerStatsRecords(): LayerStats[] {
if (!imlInstance) return [];
return imlInstance.getLayerStats();
},
getWeights(): Float32Array {
if (!imlInstance) return EMPTY_OUTPUTS;
return imlInstance.getWeights();
},
setWeights(w: Float32Array): void {
requireIML('setWeights').setWeights(w);
},
reset(): void { reset(): void {
NOT_IMPLEMENTED('reset'); requireIML('reset').reset();
}, },
clearExamples(): void { clearExamples(): void {
if (imlInstance) {
imlInstance.clearExamples();
return;
}
setState(produce((s) => { setState(produce((s) => {
s.exampleCount = 0; s.exampleCount = 0;
})); }));
coreBus.emit('ml.examples_cleared', undefined); coreBus.emit('ml.examples_cleared', undefined);
}, },
saveNow(): void {
imlInstance?.saveNow();
},
/** Disposes the singleton. Used by tests and on hot-reload. */
__dispose(): void {
if (imlInstance) {
imlInstance.dispose();
imlInstance = null;
}
initPromise = null;
setState({
exampleCount: 0,
lastLoss: null,
lossHistory: [],
inputSize: 2,
outputSize: 126,
training: false,
ready: false,
});
setOutputs(EMPTY_OUTPUTS);
setWeights(EMPTY_OUTPUTS);
},
}; };
export type MLStore = typeof mlStore; export type MLStore = typeof mlStore;

75
scripts/build-wasm.sh Executable file
View file

@ -0,0 +1,75 @@
#!/usr/bin/env bash
# scripts/build-wasm.sh — compile nisps/wasm/bindings.cpp via Emscripten and
# write the result to playground/public/.
#
# Requires emcc. Defaults to /usr/lib/emscripten/emcc; override via the EMCC
# env var. Sample invocation:
#
# EMCC=$(which emcc) scripts/build-wasm.sh
#
# Output:
# playground/public/nisps.js — Emscripten glue, MODULARIZE factory
# playground/public/nisps.wasm — the compiled module
set -euo pipefail
EMCC="${EMCC:-/usr/lib/emscripten/emcc}"
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
OUT="$ROOT/playground/public"
SRC="$ROOT/nisps/wasm/bindings.cpp"
if [[ ! -x "$EMCC" && ! -f "$EMCC" ]]; then
echo "[build-wasm] emcc not found at $EMCC" >&2
echo "[build-wasm] set EMCC=/path/to/emcc and retry." >&2
exit 2
fi
mkdir -p "$OUT"
# Exported C functions. Keep this list synchronised with the
# EMSCRIPTEN_KEEPALIVE annotations in bindings.cpp; the build will not
# fail if extras are listed but it WILL fail (or silently strip) if any
# function is missing.
EXPORTED_FUNCS='[
"_malloc","_free",
"_nisps_ml_create","_nisps_ml_destroy","_nisps_ml_reset",
"_nisps_ml_set_input","_nisps_ml_process","_nisps_ml_outputs","_nisps_ml_infer_batch",
"_nisps_ml_add_example","_nisps_ml_train","_nisps_ml_eval_loss",
"_nisps_ml_clear_examples","_nisps_ml_example_count",
"_nisps_ml_weight_count","_nisps_ml_get_weights","_nisps_ml_set_weights",
"_nisps_ml_draw_weights","_nisps_ml_move_weights",
"_nisps_ml_get_layer_stats","_nisps_ml_describe",
"_nisps_engine_create","_nisps_engine_destroy",
"_nisps_engine_set_params","_nisps_engine_process_block"
]'
# Runtime helpers we want exposed on the JS module. HEAP* views are needed by
# wasm-iml.ts/wasm-worker.ts/nisps-processor.ts to copy buffers in and out.
EXPORTED_RUNTIME='[
"HEAP8","HEAP16","HEAP32","HEAPU8","HEAPU16","HEAPU32","HEAPF32","HEAPF64",
"ccall","cwrap"
]'
set -x
"$EMCC" "$SRC" \
-std=c++20 -O3 \
-I "$ROOT/nisps" \
-fno-exceptions \
-fno-rtti \
-s WASM=1 \
-s MODULARIZE=1 \
-s EXPORT_NAME=createNispsModule \
-s ENVIRONMENT=web,worker \
-s ALLOW_MEMORY_GROWTH=1 \
-s INITIAL_MEMORY=8388608 \
-s STACK_SIZE=1048576 \
-s FILESYSTEM=0 \
-s SINGLE_FILE=0 \
-s ASSERTIONS=0 \
-s EXPORTED_FUNCTIONS="$EXPORTED_FUNCS" \
-s EXPORTED_RUNTIME_METHODS="$EXPORTED_RUNTIME" \
-o "$OUT/nisps.js"
{ set +x; } 2>/dev/null
echo "[build-wasm] wrote $OUT/nisps.js + $OUT/nisps.wasm"
ls -lh "$OUT/nisps.js" "$OUT/nisps.wasm"