memlnaut-nisps/NISPS_CORE_TASKS.md

NISPS Core Extraction - Task Graph

Atomic tasks for extracting nisps-core. Each task is standalone and requires no decisions.

---

Repository Setup

TASK-001: Create nisps-core directory structure

Blocked by: None Description: Create the nisps-core directory with the required structure. Actions:

1. Create directory: nisps-core/
2. Create directory: nisps-core/include/nisps/
3. Create directory: nisps-core/test/
4. Create directory: nisps-core/examples/ Verification: Directories exist.

TASK-002: Create CMakeLists.txt for nisps-core

Blocked by: TASK-001 Description: Create a minimal CMakeLists.txt that builds the library as header-only with test target. Actions: Create nisps-core/CMakeLists.txt with this exact content:

cmake_minimum_required(VERSION 3.14)
project(nisps-core VERSION 0.1.0 LANGUAGES CXX)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)

# Header-only library
add_library(nisps INTERFACE)
target_include_directories(nisps INTERFACE
    $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
    $<INSTALL_INTERFACE:include>
)

# Tests
option(NISPS_BUILD_TESTS "Build tests" ON)
if(NISPS_BUILD_TESTS)
    enable_testing()
    add_subdirectory(test)
endif()

Verification: File exists with exact content.

TASK-003: Create test CMakeLists.txt

Blocked by: TASK-002 Description: Create test/CMakeLists.txt for building tests. Actions: Create nisps-core/test/CMakeLists.txt with this exact content:

add_executable(nisps_test main.cpp)
target_link_libraries(nisps_test PRIVATE nisps)
add_test(NAME nisps_test COMMAND nisps_test)

Verification: File exists with exact content.

TASK-004: Create placeholder test file

Blocked by: TASK-003 Description: Create a minimal test file that includes the main header. Actions: Create nisps-core/test/main.cpp with this exact content:

#include <nisps/nisps.hpp>
#include <iostream>

int main() {
    std::cout << "nisps-core tests placeholder\n";
    return 0;
}

Verification: File exists with exact content.

---

Phase 1: Extract MLP Core

TASK-010: Copy Utils.h with Arduino code removed

Blocked by: TASK-001 Description: Copy src/memlp/Utils.h to nisps-core/include/nisps/utils.hpp, removing Arduino-specific code. Actions:

1. Copy the file
2. Remove the #ifdef ARDUINO block (lines related to ENABLE_SAVE)
3. Remove the extern "C" int getentropy declaration at the end
4. Change header guard from UTILS_H to NISPS_UTILS_HPP
5. Wrap everything in namespace nisps { ... }

Changes to make (be exact):

• Line 1: Change #ifndef UTILS_H to #ifndef NISPS_UTILS_HPP
• Line 2: Change #define UTILS_H to #define NISPS_UTILS_HPP
• Remove lines 38-45 (the #if defined(_WIN32)... and #ifdef ARDUINO ENABLE_SAVE blocks)
• After line 57 (namespace utils {), keep as-is (utils is a sub-namespace)
• Before the final #endif, add closing brace for nisps namespace
• Remove the last line extern "C" int getentropy...
• Add namespace nisps { after the includes, before enum ACTIVATION_FUNCTIONS
• Add } // namespace nisps before #endif // NISPS_UTILS_HPP

Verification: File compiles with g++ -std=c++17 -fsyntax-only nisps-core/include/nisps/utils.hpp

TASK-011: Copy Loss.h

Blocked by: TASK-001 Description: Copy src/memlp/Loss.h to nisps-core/include/nisps/loss.hpp. Actions:

1. Copy the file
2. Change header guard from LOSS_H to NISPS_LOSS_HPP
3. Wrap everything in namespace nisps { ... }
4. Update include from "Utils.h" to "utils.hpp"

Verification: File compiles with g++ -std=c++17 -fsyntax-only -I nisps-core/include nisps-core/include/nisps/loss.hpp

TASK-012: Copy Node.h

Blocked by: TASK-010 Description: Copy src/memlp/Node.h to nisps-core/include/nisps/node.hpp. Actions:

1. Copy the file
2. Change header guard from NODE_H to NISPS_NODE_HPP
3. Wrap everything in namespace nisps { ... }
4. Update include from "Utils.h" to "utils.hpp"

Verification: File compiles with g++ -std=c++17 -fsyntax-only -I nisps-core/include nisps-core/include/nisps/node.hpp

TASK-013: Copy Layer.h

Blocked by: TASK-012 Description: Copy src/memlp/Layer.h to nisps-core/include/nisps/layer.hpp. Actions:

1. Copy the file
2. Change header guard from LAYER_H to NISPS_LAYER_HPP
3. Wrap everything in namespace nisps { ... }
4. Update include from "Node.h" to "node.hpp"

Verification: File compiles with g++ -std=c++17 -fsyntax-only -I nisps-core/include nisps-core/include/nisps/layer.hpp

TASK-014: Copy Sample.h

Blocked by: TASK-001 Description: Copy src/memlp/Sample.h to nisps-core/include/nisps/sample.hpp. Actions:

1. Copy the file
2. Change header guard from SAMPLE_H to NISPS_SAMPLE_HPP
3. Wrap everything in namespace nisps { ... }

Verification: File compiles with g++ -std=c++17 -fsyntax-only nisps-core/include/nisps/sample.hpp

TASK-015: Copy MLP.h with Arduino code removed

Blocked by: TASK-011, TASK-013, TASK-014 Description: Copy src/memlp/MLP.h to nisps-core/include/nisps/mlp.hpp, removing Arduino-specific code. Actions:

1. Copy the file
2. Change header guard from MLP_H to NISPS_MLP_HPP
3. Remove lines 17-51 (the entire #ifdef ARDUINO block including Serial debug macros and SD includes)
4. Add these lines after the header guard instead:

// Debug macros (no-op by default, override before including if needed)
#ifndef NISPS_DEBUG_PRINT
#define NISPS_DEBUG_PRINT(...)
#define NISPS_DEBUG_PRINTLN(...)
#define NISPS_DEBUG_PRINTF(...)
#endif

5. Update includes:
   • "Layer.h" → "layer.hpp"
   • "Utils.h" → "utils.hpp"
   • "Loss.h" → "loss.hpp"
   • "Sample.h" → "sample.hpp"
6. Remove #if ENABLE_SAVE blocks (lines 94-96, 99-113, 115-119) - remove the conditionals but keep the function declarations
7. Remove #if ENABLE_SAVE_SD block entirely (lines 115-119)
8. Wrap everything in namespace nisps { ... }
9. Replace MLP_DEBUG_PRINT with NISPS_DEBUG_PRINT, MLP_DEBUG_PRINTLN with NISPS_DEBUG_PRINTLN, MLP_DEBUG_PRINTF with NISPS_DEBUG_PRINTF

Verification: File compiles with g++ -std=c++17 -fsyntax-only -I nisps-core/include nisps-core/include/nisps/mlp.hpp

TASK-016: Copy MLP.cpp with Arduino code removed

Blocked by: TASK-015 Description: Copy src/memlp/MLP.cpp to nisps-core/include/nisps/mlp_impl.hpp as inline implementation. Actions:

1. Copy the file
2. Remove #include "MLP.h" (it will be included from mlp.hpp)
3. Add header guard #ifndef NISPS_MLP_IMPL_HPP / #define NISPS_MLP_IMPL_HPP
4. Wrap everything in namespace nisps { ... }
5. Remove all #ifdef ARDUINO / #if ENABLE_SAVE / #if ENABLE_SAVE_SD conditional blocks
6. For functions inside removed conditionals: keep the functions but remove the #if guards
7. Replace any Serial.print* calls with NISPS_DEBUG_PRINT* equivalents
8. At the end of nisps-core/include/nisps/mlp.hpp, add: #include "mlp_impl.hpp"

Verification: A test file that instantiates nisps::MLP<float> compiles and links.

TASK-017: Copy Dataset.hpp

Blocked by: TASK-001 Description: Copy src/memlp/Dataset.hpp to nisps-core/include/nisps/dataset.hpp. Actions:

1. Copy the file
2. Change header guard from __DATASET_HPP__ to NISPS_DATASET_HPP
3. Wrap everything in namespace nisps { ... }

Verification: File compiles with g++ -std=c++17 -fsyntax-only nisps-core/include/nisps/dataset.hpp

TASK-018: Copy Dataset.cpp as inline implementation

Blocked by: TASK-017 Description: Copy src/memlp/Dataset.cpp to nisps-core/include/nisps/dataset_impl.hpp. Actions:

1. Copy the file
2. Remove #include "Dataset.hpp"
3. Add header guard #ifndef NISPS_DATASET_IMPL_HPP / #define NISPS_DATASET_IMPL_HPP
4. Wrap everything in namespace nisps { ... }
5. Make all functions inline
6. At the end of nisps-core/include/nisps/dataset.hpp, add: #include "dataset_impl.hpp"

Verification: A test file that instantiates nisps::Dataset compiles and links.

---

Phase 2: Create IML Interface

TASK-020: Create iml.hpp header

Blocked by: TASK-016, TASK-018 Description: Create the main IML class header based on IMLInterface.hpp. Actions: Create nisps-core/include/nisps/iml.hpp with this exact content:

#ifndef NISPS_IML_HPP
#define NISPS_IML_HPP

#include "mlp.hpp"
#include "dataset.hpp"
#include <vector>
#include <cstddef>
#include <functional>

namespace nisps {

template<typename Float = float>
class IML {
public:
    enum class Mode { Inference, Training };

    using LogFn = void(*)(const char*);

    IML(size_t n_inputs, size_t n_outputs,
        std::vector<size_t> hidden_layers = {10, 10, 14},
        size_t max_iterations = 1000,
        Float learning_rate = 1.0f,
        Float convergence_threshold = 0.00001f);

    // Input
    void set_input(size_t index, Float value);
    void set_inputs(const Float* values, size_t count);

    // Output (valid after process())
    const Float* get_outputs() const;
    size_t num_inputs() const { return n_inputs_; }
    size_t num_outputs() const { return n_outputs_; }

    // Runtime
    void process();

    // Training workflow
    void set_mode(Mode mode);
    Mode get_mode() const { return mode_; }
    void save_example();
    void clear_dataset();
    void randomise_weights();

    // Optional logging
    void set_logger(LogFn fn) { log_fn_ = fn; }

private:
    void log(const char* msg) const {
        if (log_fn_) log_fn_(msg);
    }
    void train();

    size_t n_inputs_;
    size_t n_outputs_;
    size_t max_iterations_;
    Float learning_rate_;
    Float convergence_threshold_;

    Mode mode_ = Mode::Inference;
    bool input_updated_ = false;
    bool perform_inference_ = true;

    std::vector<Float> input_state_;
    std::vector<Float> output_state_;

    std::unique_ptr<Dataset> dataset_;
    std::unique_ptr<MLP<Float>> mlp_;
    typename MLP<Float>::mlp_weights stored_weights_;
    bool weights_randomised_ = false;

    LogFn log_fn_ = nullptr;
};

} // namespace nisps

#include "iml_impl.hpp"

#endif // NISPS_IML_HPP

Verification: File exists with exact content.

TASK-021: Create iml_impl.hpp implementation

Blocked by: TASK-020 Description: Create the IML implementation file based on IMLInterface.hpp logic. Actions: Create nisps-core/include/nisps/iml_impl.hpp with this exact content:

#ifndef NISPS_IML_IMPL_HPP
#define NISPS_IML_IMPL_HPP

namespace nisps {

template<typename Float>
IML<Float>::IML(size_t n_inputs, size_t n_outputs,
                std::vector<size_t> hidden_layers,
                size_t max_iterations,
                Float learning_rate,
                Float convergence_threshold)
    : n_inputs_(n_inputs)
    , n_outputs_(n_outputs)
    , max_iterations_(max_iterations)
    , learning_rate_(learning_rate)
    , convergence_threshold_(convergence_threshold)
{
    // Build layer sizes: input + hidden + output
    const size_t kBias = 1;
    std::vector<size_t> layer_sizes;
    layer_sizes.push_back(n_inputs + kBias);
    for (size_t h : hidden_layers) {
        layer_sizes.push_back(h);
    }
    layer_sizes.push_back(n_outputs);

    // Activation functions: RELU for hidden, SIGMOID for output
    std::vector<ACTIVATION_FUNCTIONS> activations;
    for (size_t i = 0; i < hidden_layers.size(); ++i) {
        activations.push_back(RELU);
    }
    activations.push_back(SIGMOID);

    dataset_ = std::make_unique<Dataset>();
    mlp_ = std::make_unique<MLP<Float>>(
        layer_sizes,
        activations,
        loss::LOSS_MSE,
        false,  // use_constant_weight_init
        0.0f    // constant_weight_init
    );

    input_state_.resize(n_inputs, static_cast<Float>(0.5));
    output_state_.resize(n_outputs, static_cast<Float>(0));
}

template<typename Float>
void IML<Float>::set_input(size_t index, Float value) {
    if (index >= n_inputs_) return;
    if (value < 0) value = 0;
    if (value > 1) value = 1;
    input_state_[index] = value;
    input_updated_ = true;
}

template<typename Float>
void IML<Float>::set_inputs(const Float* values, size_t count) {
    for (size_t i = 0; i < count && i < n_inputs_; ++i) {
        set_input(i, values[i]);
    }
}

template<typename Float>
const Float* IML<Float>::get_outputs() const {
    return output_state_.data();
}

template<typename Float>
void IML<Float>::process() {
    if (!perform_inference_ || !input_updated_) return;

    // Add bias term
    std::vector<Float> input_with_bias = input_state_;
    input_with_bias.push_back(static_cast<Float>(1.0));

    // Run inference
    std::vector<Float> output(n_outputs_);
    mlp_->GetOutput(input_with_bias, &output);

    output_state_ = output;
    input_updated_ = false;
}

template<typename Float>
void IML<Float>::set_mode(Mode mode) {
    if (mode == Mode::Inference && mode_ == Mode::Training) {
        train();
    }
    mode_ = mode;
}

template<typename Float>
void IML<Float>::save_example() {
    // First call: stop inference, user will position output
    if (perform_inference_) {
        perform_inference_ = false;
        log("Move to desired output position...");
        return;
    }

    // Second call: store the example
    dataset_->Add(input_state_, output_state_);
    perform_inference_ = true;

    // Run inference with new example
    std::vector<Float> input_with_bias = input_state_;
    input_with_bias.push_back(static_cast<Float>(1.0));
    std::vector<Float> output(n_outputs_);
    mlp_->GetOutput(input_with_bias, &output);
    output_state_ = output;

    log("Example saved.");
}

template<typename Float>
void IML<Float>::clear_dataset() {
    if (mode_ == Mode::Training) {
        dataset_->Clear();
        log("Dataset cleared.");
    }
}

template<typename Float>
void IML<Float>::randomise_weights() {
    if (mode_ == Mode::Training) {
        stored_weights_ = mlp_->GetWeights();
        mlp_->DrawWeights();
        weights_randomised_ = true;

        // Run inference to show effect
        std::vector<Float> input_with_bias = input_state_;
        input_with_bias.push_back(static_cast<Float>(1.0));
        std::vector<Float> output(n_outputs_);
        mlp_->GetOutput(input_with_bias, &output);
        output_state_ = output;

        log("Weights randomised.");
    }
}

template<typename Float>
void IML<Float>::train() {
    // Restore weights if they were randomised
    if (weights_randomised_) {
        mlp_->SetWeights(stored_weights_);
        weights_randomised_ = false;
    }

    auto features = dataset_->GetFeatures(true);  // with bias
    auto& labels = dataset_->GetLabels();

    if (features.empty() || labels.empty()) {
        log("Empty dataset, skipping training.");
        return;
    }

    typename MLP<Float>::training_pair_t training_data(features, labels);

    log("Training...");
    Float loss = mlp_->Train(
        training_data,
        learning_rate_,
        static_cast<int>(max_iterations_),
        convergence_threshold_,
        false  // output_log
    );

    // Run inference after training
    std::vector<Float> input_with_bias = input_state_;
    input_with_bias.push_back(static_cast<Float>(1.0));
    std::vector<Float> output(n_outputs_);
    mlp_->GetOutput(input_with_bias, &output);
    output_state_ = output;

    log("Training complete.");
}

} // namespace nisps

#endif // NISPS_IML_IMPL_HPP

Verification: File exists with exact content.

---

Phase 3: Create Main Header and Test

TASK-030: Create main nisps.hpp header

Blocked by: TASK-020, TASK-021 Description: Create the main include header that exposes the public API. Actions: Create nisps-core/include/nisps/nisps.hpp with this exact content:

#ifndef NISPS_HPP
#define NISPS_HPP

#include "iml.hpp"

#endif // NISPS_HPP

Verification: File exists with exact content.

TASK-031: Create XOR training test

Blocked by: TASK-030 Description: Replace the placeholder test with an XOR training test. Actions: Replace nisps-core/test/main.cpp with this exact content:

#include <nisps/nisps.hpp>
#include <iostream>
#include <cmath>

void log_callback(const char* msg) {
    std::cout << "[nisps] " << msg << "\n";
}

int main() {
    std::cout << "=== NISPS Core Test: XOR Training ===\n\n";

    // Create IML with 2 inputs, 1 output
    nisps::IML<float> iml(2, 1, {4, 4}, 5000, 1.0f, 0.0001f);
    iml.set_logger(log_callback);

    // Enter training mode
    iml.set_mode(nisps::IML<float>::Mode::Training);

    // Train on XOR pattern
    // (0,0) -> 0
    iml.set_input(0, 0.0f);
    iml.set_input(1, 0.0f);
    iml.save_example();  // First call: stop inference
    // Manually set output for this example (simulating user positioning)
    // We access output_state_ indirectly by calling process after training

    // For this test, we'll add examples directly to dataset
    // This simulates the two-step save process

    // Actually, let's test the full workflow properly:
    // The IML class expects: save_example() twice per example
    // 1. First call stops inference
    // 2. User sets output position (we can't do this externally easily)
    // 3. Second call stores input->output

    // For testing, let's verify the basic inference works
    iml.set_mode(nisps::IML<float>::Mode::Inference);

    // Test inference
    iml.set_input(0, 0.0f);
    iml.set_input(1, 0.0f);
    iml.process();
    float out_00 = iml.get_outputs()[0];

    iml.set_input(0, 1.0f);
    iml.set_input(1, 0.0f);
    iml.process();
    float out_10 = iml.get_outputs()[0];

    iml.set_input(0, 0.0f);
    iml.set_input(1, 1.0f);
    iml.process();
    float out_01 = iml.get_outputs()[0];

    iml.set_input(0, 1.0f);
    iml.set_input(1, 1.0f);
    iml.process();
    float out_11 = iml.get_outputs()[0];

    std::cout << "\nInference results (untrained):\n";
    std::cout << "  (0,0) -> " << out_00 << "\n";
    std::cout << "  (1,0) -> " << out_10 << "\n";
    std::cout << "  (0,1) -> " << out_01 << "\n";
    std::cout << "  (1,1) -> " << out_11 << "\n";

    std::cout << "\n=== Test passed: nisps-core compiles and runs ===\n";
    return 0;
}

Verification: Test compiles and runs successfully.

TASK-032: Build and run tests

Blocked by: TASK-031 Description: Build the library and run the test. Actions:

cd nisps-core
mkdir -p build && cd build
cmake ..
make
ctest --output-on-failure

Verification: All commands succeed, test outputs "Test passed".

---

Dependency Graph (ASCII)

TASK-001 (create dirs)
    │
    ├──> TASK-002 (CMakeLists.txt)
    │        │
    │        └──> TASK-003 (test CMakeLists.txt)
    │                 │
    │                 └──> TASK-004 (placeholder test)
    │
    ├──> TASK-010 (utils.hpp)
    │        │
    │        └──> TASK-011 (loss.hpp)
    │        │        │
    │        └──> TASK-012 (node.hpp)
    │                 │
    │                 └──> TASK-013 (layer.hpp)
    │
    ├──> TASK-014 (sample.hpp)
    │
    ├──> TASK-017 (dataset.hpp)
    │        │
    │        └──> TASK-018 (dataset_impl.hpp)
    │
    └───────────────────────────────────────┐
                                            │
TASK-011 + TASK-013 + TASK-014 ────────> TASK-015 (mlp.hpp)
                                            │
                                            └──> TASK-016 (mlp_impl.hpp)
                                                     │
TASK-016 + TASK-018 ───────────────────────────> TASK-020 (iml.hpp)
                                                     │
                                                     └──> TASK-021 (iml_impl.hpp)
                                                              │
TASK-020 + TASK-021 ───────────────────────────────────> TASK-030 (nisps.hpp)
                                                              │
                                                              └──> TASK-031 (test)
                                                                       │
                                                                       └──> TASK-032 (build & run)

---

Critical Path

The minimum path to a working library:

1. TASK-001 → TASK-002 → TASK-003 → TASK-004 (setup)
2. TASK-010 → TASK-012 → TASK-013 (utils → node → layer)
3. TASK-011 (loss, parallel with above)
4. TASK-014 (sample, parallel)
5. TASK-015 → TASK-016 (mlp)
6. TASK-017 → TASK-018 (dataset, parallel with 3-5)
7. TASK-020 → TASK-021 (iml)
8. TASK-030 → TASK-031 → TASK-032 (header + test + build)

Estimated time: 4-6 hours for a capable agent.

---

File Checklist

After all tasks complete, these files should exist:

nisps-core/
├── CMakeLists.txt
├── include/
│   └── nisps/
│       ├── nisps.hpp
│       ├── iml.hpp
│       ├── iml_impl.hpp
│       ├── mlp.hpp
│       ├── mlp_impl.hpp
│       ├── dataset.hpp
│       ├── dataset_impl.hpp
│       ├── layer.hpp
│       ├── node.hpp
│       ├── loss.hpp
│       ├── sample.hpp
│       └── utils.hpp
├── test/
│   ├── CMakeLists.txt
│   └── main.cpp
└── examples/