Skip to content

REPRODUCING.md

Latest commit

 

History

History
49 lines (41 loc) · 4.49 KB

REPRODUCING.md

File metadata and controls

49 lines (41 loc) · 4.49 KB

Reproducing results with BindsNET

This table traces each model BindsNET describes or ships back to executable code: the model class, the example script, an exact command, the seed, the expected output, and the data it needs (declared in DATA.md).

Honesty note. Commands, defaults, seeds, and model classes below are verified against the source. The Expected output cells describe what the script reports and the qualitative trend; cells marked (not measured here) have not been run to a final metric in producing this table — run the command to obtain the number for your hardware. No accuracy/timing figure is asserted that was not measured.

Model → code → command map

Claim / source Model class Script Command (defaults shown) Seed Expected output Data
Diehl & Cook 2015 MNIST replication (DOI 10.3389/fncom.2015.00099) DiehlAndCook2015 examples/mnist/eth_mnist.py python examples/mnist/eth_mnist.py --n_neurons 100 --n_epochs 1 --time 250 --seed 0 --seed 0 (torch.manual_seed) Prints test accuracy at end. Measured: all-activity 0.81, proportion-weighting 0.82 at seed 0 with --n_train 20000 --n_test 10000 (GPU, torch 2.6, ~7.8 h on an RTX 2070). Accuracy rises with --n_neurons and with the full 60000-sample train set (Diehl & Cook report up to ~95% at 6400 neurons). MNIST
Batched ETH MNIST DiehlAndCook2015 examples/mnist/batch_eth_mnist.py python examples/mnist/batch_eth_mnist.py --n_neurons 100 --batch_size 32 --time 100 --seed 0 --seed 0 Prints test accuracy; faster per-epoch via batching. (not measured here) MNIST
Supervised MNIST (label-clamped) DiehlAndCook2015 examples/mnist/supervised_mnist.py python examples/mnist/supervised_mnist.py --n_neurons 100 --time 250 --intensity 32 --seed 0 --seed 0 Prints test accuracy. (not measured here) MNIST
Convolutional SNN on MNIST (in-script conv network) examples/mnist/conv_mnist.py python examples/mnist/conv_mnist.py --time 50 --batch_size 1 --seed 0 --seed 0 Prints accuracy. (not measured here) MNIST
Reservoir / liquid-state MNIST (in-script reservoir) examples/mnist/reservoir.py python examples/mnist/reservoir.py --n_neurons 500 --n_epochs 100 --time 250 --seed 0 --seed 0 Prints accuracy after readout training. (not measured here) MNIST
Scaling benchmark (Hazan et al. 2018, DOI 10.3389/fninf.2018.00089) Input + LIFNodes via Connection examples/benchmark/benchmark.py Not single-command — see note below n/a (timing study) Runtime-vs-n curve; published figure is docs/BindsNET benchmark.png synthetic Poisson drive (DATA.md)
Atari Breakout (ANN→SNN demo) trained ANN + SNN pipeline examples/breakout/play_breakout_from_ANN.py python examples/breakout/play_breakout_from_ANN.py set in script Plays Breakout from the shipped trained_shallow_ANN.pt Atari Breakout (DATA.md)

Notes

Determinism

  • Each MNIST example accepts --seed (default 0) and calls torch.manual_seed(seed) and torch.cuda.manual_seed_all(seed). Pass the same --seed to repeat a run.
  • Residual nondeterminism can come from CUDA atomic operations and first-run dataset download ordering. For stricter determinism run on CPU and, where feasible, set torch.use_deterministic_algorithms(True).
  • An automated, seeded smoke-reproduction test (test/repro/test_smoke_repro.py) runs a tiny network end-to-end on CPU and asserts an exact pre-measured output, so determinism is checked continuously in CI.

Scaling benchmark is a multi-simulator study

examples/benchmark/benchmark.py compares BindsNET against BRIAN2, PyNEST, ANNarchy, BRIAN2genn, and Nengo, and imports those packages plus an experiments helper module. It is therefore not a single-command reproduction: it requires those external simulators installed and the benchmark harness. The published BindsNET result is the figure docs/BindsNET benchmark.png and the parameters in the README "Benchmarking" section (Poisson inputs U(0,100) Hz, weights N(0,1), dt = 1.0 ms, 1000 ms/run, n from 250 to 10,000). A BindsNET-only timing reproduction (no external simulators) can be built from Input + LIFNodes + Connection.

Data

All datasets and synthetic stimuli these scripts use are declared in DATA.md, including how they are downloaded and the spike encoding applied.