docs(playground): revise ShapeSeq spec after fresh-eyes review
Major revisions addressing architectural gaps: - Fix data flow: clock drives everything (top of diagram) - Add fixed MLP + param mapping layer to handle dynamic chain sizing - Introduce symbolic chain evaluation (pattern descriptions, not concrete values) - Categorize primitives: generator, processor, timing modifier, converter - Generator combination modes (additive/multiplicative, user-configurable) - Swing and Ratchet are now timing modifiers on the pattern description - Remove Scale Quantizer from projection layer (Interval Lock is the sole quantizer) - Clarify mode integration: ShapeSeq replaces particle viz in synth mode - Build on existing imlJoy/imlHand dual-instance pattern - Add Open Design Questions section for deferred decisions - Port-ready applies to primitive layer only, not orchestration
This commit is contained in:
w1n5t0n
parent
1a0f2c4d1c
commit
4fd0b39d26
1 changed files with 202 additions and 103 deletions
|
|
@ -4,62 +4,77 @@
|
|||
|
||||
ShapeSeq is a generative sequencing system for the NISPS playground where interactive ML (via the NISPS MLP engine) controls **parameters of algorithmic sequencing primitives** rather than raw note data. The user shapes sequences by navigating a learned parameter space with a joystick or hand tracking, can freeze sequences they like, then selectively re-expose specific parameters for further ML-driven exploration.
|
||||
|
||||
ShapeSeq replaces the existing placeholder arpeggiator.
|
||||
ShapeSeq replaces the existing placeholder arpeggiator. In synth output mode, the ShapeSeq UI (circular step visualizer + chain builder + param sliders) replaces the flow-field particle system.
|
||||
|
||||
## Core Architecture
|
||||
|
||||
### Design Principles
|
||||
|
||||
1. **MLP outputs are abstract [0,1] values** — musical meaning is applied downstream by a configurable projection layer
|
||||
2. **Separate NISPS instances** for timbre control and sequence control, with architecture supporting future unification into a single instance
|
||||
3. **Port-ready JS** — no closures in hot paths, explicit state, data structures that map cleanly to C++ for future RP2040 firmware porting
|
||||
1. **MLP outputs are abstract [0,1] values** — musical meaning is applied downstream by the primitive chain and its symbolic processing
|
||||
2. **Separate NISPS instances** for timbre control and sequence control, following the existing `imlJoy`/`imlHand` dual-instance pattern in `a-app.js`. Architecture supports future unification into a single instance
|
||||
3. **Port-ready JS** — no closures in hot paths, explicit state, data structures that map cleanly to C++ for future RP2040 firmware porting. Note: the event bus and clock orchestration are JS-only concerns and not expected to port directly; the primitives themselves are the portable layer
|
||||
4. **Modular primitives** — small, combinable algorithmic building blocks that generate musical patterns from continuous parameters
|
||||
5. **Symbolic chain** — primitives compose as transforms over a pattern *description*, not concrete values. Each primitive takes the previous "pattern-generating machine" specification and produces a new one. The chain is evaluated once per loop (or on param change) to produce a complete pattern, which the clock then steps through
|
||||
|
||||
### System Diagram
|
||||
|
||||
```
|
||||
┌─────────────────┐
|
||||
│ Input Router │
|
||||
│ (configurable) │
|
||||
└──┬──────────┬───┘
|
||||
│ │
|
||||
┌────────▼──┐ ┌───▼────────┐
|
||||
│ NISPS MLP │ │ NISPS MLP │
|
||||
│ (timbre) │ │ (sequence) │
|
||||
└────────┬───┘ └───┬────────┘
|
||||
│ │
|
||||
┌────────▼──┐ ┌───▼────────────────────┐
|
||||
│ Synth │ │ Delta Controller │
|
||||
│ Param Map │ │ (frozen vals + deltas) │
|
||||
└────────┬───┘ └───┬────────────────────┘
|
||||
│ │
|
||||
│ ┌────▼──────────────┐
|
||||
│ │ Primitive Chain │
|
||||
│ │ Euclid→ProbGate→… │
|
||||
│ └────┬──────────────┘
|
||||
│ │
|
||||
│ ┌────▼──────────────┐
|
||||
│ │ Projection Layer │
|
||||
│ │ (scale quant, etc) │
|
||||
│ └────┬──────────────┘
|
||||
│ │
|
||||
│ ┌────▼──────────┐
|
||||
│ │ Clock Engine │
|
||||
│ │ (AudioContext) │
|
||||
│ └────┬──────────┘
|
||||
│ │
|
||||
┌────▼──────────▼────┐
|
||||
│ Namespaced Event Bus│
|
||||
│ seq.* ml.* ui.* │
|
||||
└────────┬───────────┘
|
||||
│
|
||||
┌────────▼───────┐
|
||||
│ C15 Synth │
|
||||
│ (noteOn/Off + │
|
||||
│ param changes)│
|
||||
└────────────────┘
|
||||
┌──────────────────┐
|
||||
│ Clock Engine │ ← drives everything
|
||||
│ (AudioContext) │
|
||||
└──────┬───────────┘
|
||||
│ tick
|
||||
▼
|
||||
┌──────────────────┐ ┌─────────────────┐
|
||||
│ Sequencer Core │◄────│ Input Router │
|
||||
│ (orchestrator) │ │ (configurable) │
|
||||
└──┬───────────┬───┘ └──┬──────────┬───┘
|
||||
│ │ │ │
|
||||
│ query │ query ┌────▼───┐ ┌───▼────────┐
|
||||
│ pattern │ MLP │NISPS │ │ NISPS MLP │
|
||||
│ │ │(timbre)│ │ (sequence) │
|
||||
│ │ └───┬────┘ └───┬────────┘
|
||||
│ │ │ │
|
||||
│ ┌──────▼────────┐ │ ┌─────▼──────────┐
|
||||
│ │ Param Mapping │ │ │ Param Mapping │
|
||||
│ │ (16 MLP outs │ │ │ (16 MLP outs → │
|
||||
│ │ → N prim │ │ │ 126 synth │
|
||||
│ │ params) │ │ │ params) │
|
||||
│ └──────┬────────┘ │ └─────┬──────────┘
|
||||
│ │ │ │
|
||||
│ ┌──────▼────────┐ │ ┌─────▼──────────┐
|
||||
│ │Delta Controller│ │ │ Synth Param Map │
|
||||
│ │(frozen+deltas)│ │ └─────┬──────────┘
|
||||
│ └──────┬────────┘ │ │
|
||||
│ │ │ │
|
||||
│ ┌──────▼────────┐ │ │
|
||||
│ │Primitive Chain │ │ │
|
||||
│ │(symbolic eval) │ │ │
|
||||
│ └──────┬────────┘ │ │
|
||||
│ │ │ │
|
||||
┌──────▼───────────▼───┐ │ │
|
||||
│ Namespaced Event Bus │ │ │
|
||||
│ seq.* ml.* ui.* │ │ │
|
||||
└──┬───────────────┬───┘ │ │
|
||||
│ │ │ │
|
||||
┌──────▼───────┐ ┌────▼───────────▼────────▼──┐
|
||||
│ Circular Viz │ │ C15 Synth │
|
||||
│ + Chain UI │ │ (noteOn/Off + params) │
|
||||
└──────────────┘ └────────────────────────────┘
|
||||
```
|
||||
|
||||
### Fixed MLP + Param Mapping Layer
|
||||
|
||||
The `WasmIML` creates an MLP with a **fixed output count** at construction time — it cannot be resized. Since the primitive chain is dynamic (users add/remove primitives, changing total param count), the MLP cannot output directly to primitive params.
|
||||
|
||||
**Solution:** The sequence MLP always outputs a fixed number of values (e.g., 16). A **param mapping layer** fans these 16 outputs to however many primitive params the current chain requires. This is the same pattern used by the timbre MLP (which maps to 126 synth params via `param-map.js`).
|
||||
|
||||
The mapping can be:
|
||||
- **Automatic** (default): outputs are distributed across primitive params in chain order. If there are 30 primitive params and 16 MLP outputs, each output influences ~2 params via interpolation.
|
||||
- Future: configurable user-defined mapping.
|
||||
|
||||
> **Design note:** The fixed-16-output approach is the simplest starting point. If experimentation reveals that 16 is too few (or too many), the MLP can be reconstructed with a different size — this is a one-time setup cost, not a per-frame cost. The mapping layer insulates the rest of the system from this choice. Revisit if the mapping layer becomes a bottleneck for expressiveness.
|
||||
|
||||
### Namespaced Event Bus
|
||||
|
||||
A pub/sub event system with namespaced channels:
|
||||
|
|
@ -72,9 +87,11 @@ A pub/sub event system with namespaced channels:
|
|||
|
||||
All events carry a timestamp (AudioContext.currentTime for `seq.*`, performance.now() for others).
|
||||
|
||||
Note: the event bus is a JS-only orchestration concern (string-namespaced pub/sub). It does not need to be port-ready — the portable layer is the primitives themselves.
|
||||
|
||||
### Input Routing Matrix
|
||||
|
||||
A configurable routing layer that maps any input source to either NISPS instance's inputs:
|
||||
A configurable routing layer that maps any input source to either NISPS instance's inputs. Builds on the existing `imlJoy`/`imlHand` switching pattern in `a-app.js`.
|
||||
|
||||
**Input sources:**
|
||||
- Joystick X, Y (2 values)
|
||||
|
|
@ -89,75 +106,129 @@ Default: joystick → timbre NISPS, hand tracking features 0+1 → sequence NISP
|
|||
|
||||
## Sequencing Primitives
|
||||
|
||||
### Primitive Categories
|
||||
|
||||
Primitives are categorized by their role in the chain:
|
||||
|
||||
| Category | Role | Examples |
|
||||
|----------|------|----------|
|
||||
| **Generator** | Produces data from params alone (no input required) | Euclidean, Density Morph, Pitch Walker |
|
||||
| **Processor** | Transforms incoming data | Probability Gate, Velocity Shaper |
|
||||
| **Timing Modifier** | Modulates the timing of events in the pattern description | Swing/Groove, Ratchet |
|
||||
| **Converter** | Changes data type (e.g., continuous → discrete) | Interval Lock |
|
||||
|
||||
**Generator combination rule:** When multiple generators appear in the same chain, their outputs combine according to the chain's **combination mode** (user-configurable in real time):
|
||||
- **Additive** (OR) — triggers from any generator fire. Pitch/velocity values are averaged where multiple generators contribute.
|
||||
- **Multiplicative** (AND) — only steps where ALL generators agree will fire. Creates sparser, more selective patterns.
|
||||
|
||||
### Symbolic Chain Evaluation
|
||||
|
||||
Primitives do NOT process concrete note data step-by-step. Instead, each primitive takes the previous **pattern description** (a symbolic representation of the entire sequence) and produces a new one. The complete chain is evaluated to produce a full pattern, which the clock then steps through.
|
||||
|
||||
This means:
|
||||
- **Timing modifiers** (Swing, Ratchet) work by annotating the pattern description with timing offsets and subdivisions *before* any concrete scheduling happens
|
||||
- The clock reads the finalized pattern description and schedules all events (including ratchet subdivisions and swing offsets) using AudioContext.currentTime
|
||||
- Re-evaluation happens when params change (MLP output updates, user edits), not on every tick
|
||||
|
||||
**Pattern description structure:**
|
||||
```javascript
|
||||
// The symbolic output of the chain — a complete loop description
|
||||
{
|
||||
steps: [
|
||||
{
|
||||
trigger: true, // whether this step fires
|
||||
pitch: 0.72, // [0,1] abstract pitch (pre-quantization)
|
||||
velocity: 0.85, // [0,1]
|
||||
accent: false, // accent flag
|
||||
timeOffset: 0.0, // swing offset in fractions of a step (-0.5 to +0.5)
|
||||
subdivisions: 1, // ratchet: 1 = normal, 2-4 = subdivided
|
||||
},
|
||||
// ... one per step
|
||||
],
|
||||
stepCount: 8,
|
||||
metadata: { ... } // chain-specific info for visualization
|
||||
}
|
||||
```
|
||||
|
||||
### Primitive Definitions
|
||||
|
||||
Each primitive is a pure function (or stateful generator with explicit state) that accepts a parameter object and produces typed output. All parameters are normalized [0,1].
|
||||
|
||||
### 1. Euclidean Rhythm Generator
|
||||
#### 1. Euclidean Rhythm Generator
|
||||
|
||||
**Category:** Generator
|
||||
**Params:** `steps` (int, from continuous), `pulses` (int), `rotation` (int)
|
||||
**Output type:** trigger pattern (boolean array)
|
||||
**Output:** trigger pattern (boolean array)
|
||||
**Stateless:** yes
|
||||
|
||||
Generates Bjorklund-distributed trigger patterns. The continuous [0,1] params are projected to integer ranges based on current step count.
|
||||
|
||||
### 2. Probability Gate
|
||||
#### 2. Probability Gate
|
||||
|
||||
**Category:** Processor
|
||||
**Params:** `density` [0,1], `accentProbability` [0,1]
|
||||
**Input type:** trigger pattern
|
||||
**Output type:** filtered trigger pattern with accent flags
|
||||
**Input:** trigger pattern
|
||||
**Output:** filtered trigger pattern with accent flags
|
||||
**Stateless:** yes (per-step coin flip using seeded PRNG)
|
||||
|
||||
Each incoming trigger survives with probability `density`. Surviving triggers receive accent flag with probability `accentProbability`.
|
||||
|
||||
### 3. Pitch Walker
|
||||
#### 3. Pitch Walker
|
||||
|
||||
**Category:** Generator
|
||||
**Params:** `stepSize` [0,1], `directionBias` [0,1] (0.5=unbiased), `gravity` [0,1] (pull toward center), `range` [0,1]
|
||||
**Input type:** trigger pattern
|
||||
**Output type:** pitch values [0,1] per triggered step
|
||||
**Output:** pitch values [0,1] per triggered step
|
||||
**Stateful:** yes — maintains current position in pitch space
|
||||
|
||||
Constrained random walk that generates melodic contour. `gravity` pulls the walk toward center (0.5), preventing it from getting stuck at extremes. State includes current position and PRNG state.
|
||||
|
||||
### 4. Ratchet
|
||||
#### 4. Ratchet
|
||||
|
||||
**Category:** Timing Modifier
|
||||
**Params:** `maxDivision` [0,1] (maps to 1-4 subdivisions), `probability` [0,1]
|
||||
**Input type:** trigger pattern
|
||||
**Output type:** trigger pattern with subdivision timing offsets
|
||||
**Input:** pattern description with triggers
|
||||
**Output:** pattern description with `subdivisions` field set per step
|
||||
**Stateless:** yes (per-step coin flip)
|
||||
|
||||
Subdivides triggered steps into rapid repeats. Division count determined by `maxDivision`, applied probabilistically.
|
||||
Annotates triggered steps with subdivision counts. The clock engine reads `subdivisions` and schedules rapid repeats within the step's time window. Division count determined by `maxDivision`, applied probabilistically.
|
||||
|
||||
### 5. Swing / Groove
|
||||
#### 5. Swing / Groove
|
||||
|
||||
**Category:** Timing Modifier
|
||||
**Params:** `swingAmount` [0,1] (0=straight, 1=full swing), `swingGrid` [0,1] (which subdivisions swing)
|
||||
**Input type:** timing information
|
||||
**Output type:** modified timing offsets
|
||||
**Input:** pattern description
|
||||
**Output:** pattern description with `timeOffset` field set per step
|
||||
**Stateless:** yes
|
||||
|
||||
Shifts timing of alternating steps. At `swingAmount=0.67` this produces classic 2:1 shuffle. `swingGrid` controls whether swing applies to 8th notes, 16th notes, or triplets.
|
||||
Annotates alternating steps with timing offsets. At `swingAmount=0.67` this produces classic 2:1 shuffle. `swingGrid` controls whether swing applies to 8th notes, 16th notes, or triplets. The clock engine reads `timeOffset` and adjusts scheduling accordingly.
|
||||
|
||||
### 6. Density Morph
|
||||
#### 6. Density Morph
|
||||
|
||||
**Category:** Generator
|
||||
**Params:** `density` [0,1], `clustering` [0,1] (0=spread evenly, 1=clustered together)
|
||||
**Input type:** step count
|
||||
**Output type:** trigger pattern
|
||||
**Output:** trigger pattern
|
||||
**Stateless:** yes
|
||||
|
||||
Alternative to Euclidean — generates trigger patterns with controllable density and spatial distribution. At high clustering, triggers group together creating bursts; at low clustering, triggers spread evenly.
|
||||
|
||||
### 7. Interval Lock (Scale Quantizer)
|
||||
#### 7. Interval Lock (Scale Quantizer)
|
||||
|
||||
**Category:** Converter
|
||||
**Params:** `root` [0,1] (maps to 0-11 semitones), `mode` [0,1] (maps to scale index), `octaveRange` [0,1] (1-4 octaves)
|
||||
**Input type:** pitch values [0,1]
|
||||
**Output type:** MIDI note numbers
|
||||
**Input:** pitch values [0,1]
|
||||
**Output:** MIDI note numbers
|
||||
**Stateless:** yes
|
||||
|
||||
The sole pitch quantization mechanism — the projection layer does NOT duplicate this. All pitch quantization goes through Interval Lock.
|
||||
|
||||
Available scales: chromatic, major, natural minor, harmonic minor, pentatonic major, pentatonic minor, blues, dorian, mixolydian, whole tone, diminished.
|
||||
|
||||
### 8. Velocity Shaper
|
||||
#### 8. Velocity Shaper
|
||||
|
||||
**Category:** Processor
|
||||
**Params:** `curveType` [0,1] (maps to: flat, accent-every-N, crescendo, decrescendo, random), `depth` [0,1], `phase` [0,1]
|
||||
**Input type:** trigger pattern with step indices
|
||||
**Output type:** velocity values [0,1] per step
|
||||
**Input:** trigger pattern with step indices
|
||||
**Output:** velocity values [0,1] per step
|
||||
**Stateless:** yes
|
||||
|
||||
Applies cyclic velocity patterns. `phase` rotates the pattern, `depth` controls contrast between quiet and loud.
|
||||
|
|
@ -167,14 +238,16 @@ Applies cyclic velocity patterns. `phase` rotates the pattern, `depth` controls
|
|||
```javascript
|
||||
// Port-ready: explicit state, no closures
|
||||
class Primitive {
|
||||
constructor(name, paramSchema) { ... }
|
||||
constructor(name, paramSchema, category) { ... }
|
||||
|
||||
// category: 'generator' | 'processor' | 'timing' | 'converter'
|
||||
|
||||
// paramSchema: array of { name, min, max, default, boundary }
|
||||
// boundary: 'clamp' | 'wrap' | 'scaled'
|
||||
// For 'scaled': operates within ±scaledRange of frozen value
|
||||
|
||||
// Pure processing function
|
||||
process(params, input, state, stepCount, rng) → { output, nextState }
|
||||
// Symbolic processing: transforms a pattern description
|
||||
process(params, patternDesc, state, rng) → { patternDesc, nextState }
|
||||
|
||||
// State management for freeze
|
||||
getState() → serializable object
|
||||
|
|
@ -188,14 +261,16 @@ class Primitive {
|
|||
|
||||
Three configurable modes for how primitives connect in a chain:
|
||||
|
||||
**1. Sequential Pipeline** — each primitive transforms the previous output. Order matters. Signal flows left to right.
|
||||
**1. Sequential Pipeline** — each primitive transforms the pattern description in order. Generators create initial data, processors/timing modifiers transform it. If multiple generators appear, they combine according to the generator combination mode (additive/multiplicative, configurable in real time).
|
||||
|
||||
**2. Parallel + Merge** — each primitive runs independently, outputs merged (OR for triggers, average for continuous values). Order doesn't matter.
|
||||
**2. Parallel + Merge** — each primitive runs independently and produces a pattern description. Descriptions merge (OR for triggers in additive mode, AND in multiplicative mode; average for continuous values). Order doesn't matter.
|
||||
|
||||
**3. Typed Routing** — primitives connect via typed ports. A primitive's output connects to the next primitive that accepts that type. Multiple primitives can feed the same type (merged). Most flexible, most complex.
|
||||
|
||||
The chain connection mode is a global setting (per-chain), configurable via UI. Default: sequential pipeline.
|
||||
|
||||
**Generator combination mode** (additive/multiplicative) is an independent setting, also configurable in real time via UI.
|
||||
|
||||
## Delta Control System
|
||||
|
||||
### Freeze Workflow
|
||||
|
|
@ -239,7 +314,6 @@ class ClockEngine {
|
|||
// Properties
|
||||
bpm // beats per minute
|
||||
stepCount // total steps in sequence
|
||||
subdivision // ticks per step (for ratchet/swing resolution)
|
||||
|
||||
// Lookahead scheduling: schedule events slightly ahead of time
|
||||
// using AudioContext.currentTime for sample-accurate timing
|
||||
|
|
@ -247,8 +321,15 @@ class ClockEngine {
|
|||
stop() → void
|
||||
setTempo(bpm) → void
|
||||
|
||||
// Callback: called with { stepIndex, time, isAccent }
|
||||
onStep(callback) → void
|
||||
// The clock reads the finalized pattern description and schedules
|
||||
// all events, including:
|
||||
// - timeOffset per step (swing)
|
||||
// - subdivisions per step (ratchet)
|
||||
// - accent flags (velocity scaling)
|
||||
schedulePattern(patternDesc) → void
|
||||
|
||||
// Callback: called with { stepIndex, time, velocity, pitch, isSubdivision }
|
||||
onEvent(callback) → void
|
||||
}
|
||||
```
|
||||
|
||||
|
|
@ -256,16 +337,18 @@ The clock uses the standard Web Audio lookahead pattern:
|
|||
- A setInterval (~25ms) checks if any events need scheduling in the next ~100ms
|
||||
- Events are scheduled using AudioContext.currentTime for sample-accurate timing
|
||||
- This decouples visual updates (requestAnimationFrame) from audio timing
|
||||
- The clock handles ratchet subdivisions and swing offsets natively by reading the pattern description's per-step `subdivisions` and `timeOffset` fields
|
||||
|
||||
## Projection Layer
|
||||
|
||||
A composable chain of transform functions that convert raw [0,1] MLP outputs into musical values. Each transform is a small, independent module.
|
||||
A composable chain of transform functions that convert raw [0,1] primitive outputs into final musical values. Each transform is a small, independent module.
|
||||
|
||||
Note: pitch quantization is handled by the **Interval Lock** primitive, not the projection layer. The projection layer handles non-pitch transforms only.
|
||||
|
||||
### Available Transforms
|
||||
|
||||
| Transform | Input | Output | Params |
|
||||
|-----------|-------|--------|--------|
|
||||
| Scale Quantizer | [0,1] | MIDI note | root, scale, octave range |
|
||||
| Velocity Curve | [0,1] | [0,1] | curve shape (linear, exponential, S-curve) |
|
||||
| Gate Threshold | [0,1] | boolean | threshold value |
|
||||
| Range Map | [0,1] | [min,max] | min, max |
|
||||
|
|
@ -277,15 +360,23 @@ Transforms snap together: output type of one must match input type of next. The
|
|||
### Projection Presets
|
||||
|
||||
Pre-built chain configurations for common use cases:
|
||||
- **Melodic Minor** — scale quant (A minor) → octave fold (2 oct) → velocity curve (exponential)
|
||||
- **Pentatonic Drift** — scale quant (pentatonic) → octave fold (3 oct) → velocity curve (S)
|
||||
- **Chromatic Chaos** — range map (full MIDI) → velocity curve (linear)
|
||||
- **Rhythmic Only** — gate threshold (0.5) → velocity curve (accent)
|
||||
- **Expressive** — velocity curve (exponential) → range map (48-84)
|
||||
- **Percussive** — gate threshold (0.5) → velocity curve (accent)
|
||||
- **Full Range** — range map (24-96) → velocity curve (linear)
|
||||
|
||||
Users can edit any preset or build custom chains.
|
||||
|
||||
## UI Design
|
||||
|
||||
### Mode Integration
|
||||
|
||||
ShapeSeq activates in **synth output mode**. When synth mode is active:
|
||||
- The flow-field particle visualizer is replaced by the ShapeSeq UI (circular step viz + chain builder + param sliders)
|
||||
- The timbre NISPS instance continues to control C15 synth parameters as before
|
||||
- The sequence NISPS instance drives the ShapeSeq primitive chain
|
||||
|
||||
In visual output mode, the particle system remains unchanged.
|
||||
|
||||
### Circular Step Visualizer
|
||||
|
||||
Steps arranged in a circle with even angular spacing (7 steps = heptagon, 13 steps = 13-gon, etc.). No grid overlay — the ear provides rhythmic context.
|
||||
|
|
@ -347,7 +438,7 @@ Live params show their current NISPS delta as a secondary indicator on the slide
|
|||
│ └──────────────────────────────┘ │
|
||||
├──────────────────────────────────┤
|
||||
│ [▶ Play] [❄ Freeze] [Chain:Seq] │
|
||||
│ BPM: 120 Steps: 8 Scale: Cm │
|
||||
│ [+×] BPM:120 Steps:8 Gen:Add │
|
||||
└──────────────────────────────────┘
|
||||
```
|
||||
|
||||
|
|
@ -358,16 +449,17 @@ Live params show their current NISPS delta as a secondary indicator on the slide
|
|||
**Goal:** All 8 primitives working, chain builder, basic UI, NISPS control. Full architecture with minimal polish.
|
||||
|
||||
1. **Event bus** — namespaced pub/sub system
|
||||
2. **Clock engine** — AudioContext-based precise timing
|
||||
3. **Primitive framework** — base class, param schema, state management
|
||||
4. **All 8 primitives** — implement each with their param schemas
|
||||
5. **Sequential chain** — primitives connected in sequence (pipeline mode only)
|
||||
6. **Projection layer** — scale quantizer + velocity curve (2 transforms minimum)
|
||||
7. **Sequence NISPS instance** — second MLP controlling primitive chain params
|
||||
8. **Basic circular viz** — step circle with playback indicator
|
||||
9. **Basic chain UI** — vertical stack with sliders, add/remove primitives
|
||||
10. **Bridge integration** — sequence events → C15 noteOn/noteOff via event bus
|
||||
11. **Remove arpeggiator** — replace with ShapeSeq
|
||||
2. **Clock engine** — AudioContext-based precise timing with pattern description scheduling (handles swing offsets + ratchet subdivisions)
|
||||
3. **Primitive framework** — base class, param schema, category system, state management, symbolic pattern description structure
|
||||
4. **All 8 primitives** — implement each with their param schemas and categories
|
||||
5. **Sequential chain** — primitives connected in sequence (pipeline mode only), with additive/multiplicative generator combination mode
|
||||
6. **Param mapping layer** — fixed 16-output MLP → N primitive params, automatic distribution
|
||||
7. **Projection layer** — velocity curve + gate threshold (2 transforms minimum, no scale quantizer — that's Interval Lock)
|
||||
8. **Sequence NISPS instance** — second WasmIML (16 outputs), following existing `imlJoy`/`imlHand` pattern
|
||||
9. **Basic circular viz** — step circle with playback indicator
|
||||
10. **Basic chain UI** — vertical stack with sliders, add/remove primitives, generator combo mode toggle
|
||||
11. **Bridge integration** — sequence events → C15 noteOn/noteOff via event bus
|
||||
12. **Replace arpeggiator** — remove old arpeggiator, ShapeSeq takes over in synth mode
|
||||
|
||||
### Phase 2 — Freeze & Delta Control
|
||||
|
||||
|
|
@ -394,9 +486,18 @@ Live params show their current NISPS delta as a secondary indicator on the slide
|
|||
5. **Freeform lasso param selection** (see Future Work)
|
||||
6. **Per-track variable step counts** (polyrhythm)
|
||||
|
||||
## Open Design Questions
|
||||
|
||||
These are deliberately deferred decisions to be revisited after experimentation:
|
||||
|
||||
1. **MLP output count:** Is 16 the right number for the sequence MLP? Too few may limit expressiveness; too many may make learning harder. The param mapping layer insulates the system, so this can be changed without architectural impact.
|
||||
2. **Param mapping strategy:** Automatic distribution is the starting point. Should users be able to manually wire MLP outputs to specific primitive params? This could enable more intentional control but adds UI complexity.
|
||||
3. **Generator combination modes:** Additive and multiplicative are the starting pair. Other modes worth exploring: weighted average, priority (first generator wins), XOR (one or the other but not both).
|
||||
4. **Chain evaluation frequency:** Currently re-evaluates when params change. Should there be an option for per-loop re-evaluation (stateful primitives produce different patterns each loop)?
|
||||
|
||||
## Future Work
|
||||
|
||||
- **Freeform lasso selection** — draw/lasso over the step visualization to select params spatially. Intuitive but complex to implement. (Create bd backlog issue.)
|
||||
- **Freeform lasso selection** — draw/lasso over the step visualization to select params spatially. Intuitive but complex to implement. (Backlog issue: meml-hud)
|
||||
- **MIDI clock sync** — accept external MIDI clock for hardware sync
|
||||
- **OSC output** — route sequencer events via OSC for external software/hardware
|
||||
- **C++ port** — port primitive framework and chain system to nisps-core for RP2040 firmware
|
||||
|
|
@ -409,13 +510,15 @@ Live params show their current NISPS delta as a secondary indicator on the slide
|
|||
|
||||
### Port-Ready JS Conventions
|
||||
|
||||
To facilitate future C++ porting:
|
||||
To facilitate future C++ porting of the **primitive layer**:
|
||||
- No closures in primitive process functions — all state is explicit
|
||||
- Use typed arrays (Float32Array) for parameter vectors where possible
|
||||
- Primitives are pure functions with explicit state in/out
|
||||
- Seeded PRNG (not Math.random()) for deterministic replay
|
||||
- All time values in seconds (AudioContext convention), not milliseconds
|
||||
|
||||
The orchestration layer (event bus, clock, UI) is JS-only and not expected to port.
|
||||
|
||||
### PRNG
|
||||
|
||||
Use a seedable PRNG (e.g., mulberry32 or xoshiro128) so that:
|
||||
|
|
@ -425,10 +528,6 @@ Use a seedable PRNG (e.g., mulberry32 or xoshiro128) so that:
|
|||
|
||||
### Performance Budget
|
||||
|
||||
The sequencer tick runs at most every ~15ms (at 250 BPM with 16th note subdivision). Each tick must:
|
||||
1. Query NISPS MLP (already fast — the MLP inference is <1ms)
|
||||
2. Run primitive chain (8 primitives × simple math = negligible)
|
||||
3. Apply projection transforms (simple lookups/math)
|
||||
4. Emit events
|
||||
The chain evaluates on param change, not per tick. The clock merely steps through the pre-computed pattern description. Per-tick cost is minimal: read the next step from the pattern, schedule the event. MLP inference (~<1ms) only runs when input changes.
|
||||
|
||||
Total budget per tick: ~5ms. This is comfortable even on mobile.
|
||||
Chain re-evaluation (all 8 primitives) happens when the MLP output changes. At ~60fps input update rate, this means ~16ms budget per evaluation. Each primitive is simple math, so 8 primitives is well within budget even on mobile.
|
||||
|
|
|
|||
Loading…
Reference in a new issue