Automatically save Probe Interface files when recording starts

[bparks13]

Summary

This PR introduces automatic saving of Probe Interface files when recording starts. The changes are aimed at improving compatibility with SpikeInterface and simplifying downstream file processing by providing JSON metadata directly.

Key Changes

• Streamlined stream name creation to make naming more consistent and easier to standardize across files.
• Renamed methods to follow general C++ conventions (camelCase).
• Introduced a new Neuropixels metadata class, which now reads all available probe information instead of only the serial number.
• Automatic JSON saving: when recording starts, all streams that have a ProbeInterface object will now save a corresponding JSON file in the data directory defined by the GUI Control Panel.

Motivation

These updates are primarily intended to support SpikeInterface compatibility, reducing friction during data analysis by allowing metadata to be loaded directly from a JSON file.

Folder Structure

When a recording is started, the data directory can contain two top-level folders:

• ONIX Source 100
  • Contains experiment subfolders (experiment1, experiment2, experiment3, etc.) whenever there are streams that need to save a Probe Interface file
  • Each experiment stores <stream_name>.json file, where the filename matches the associated stream. More than one JSON file can exist in each folder.
• Record Node 102
  • Contains the same experiment subfolders (experiment1, experiment2, experiment3, etc.).
  • Also includes recording configuration files: settings.xml, settings_2.xml, settings_3.xml, etc.

This layout ensures that probe metadata is saved alongside recording data, with stream-specific JSON files simplifying downstream processing and analysis. For a visual representation of the possible folder structure, see below:

2025-08-19_16-03-56/
├── ONIX Source 100/
│   ├── experiment1/
│   │   └── <stream_name>.json
│   ├── experiment2/
│   │   └── <stream_name>.json
│   └── experiment3/
│       └── <stream_name>.json
└── Record Node 102/
    ├── experiment1/
    ├── experiment2/
    ├── experiment3/
    ├── settings.xml
    ├── settings_2.xml
    └── settings_3.xml

It is important to note two things:

1. If no streams are being recorded that need to save a ProbeInterface file, then the ONIX Source folder will not be created for that experiment.
2. The ONIX Source plugin sets its data directory based on the Control Panel of the GUI, which can become desynchronized from the Record Node if the user chooses to save ephys data in a location other than the data directory from the Control Panel. This is expected behavior according to the dev team, as there are instances where it is beneficial to save ephys data across multiple folders using multiple Record Nodes. In that case, it is up to the user to consolidate the Record Node and ONIX Source folders into the folder structure above, or post processing may not be able to automatically discover the associated JSON files.

Fixes #135

[bparks13]

Attached here is a zip file with the folder structure shown above. The ephys data has been removed to keep the file size small.

2025-08-19_16-03-56.zip

[bparks13]

Attached here are three more electrode configurations from a 1.0 probe to test the loading scripts. The specific configurations used are listed below, where the electrode indices are zero-indexed and are the global index of the electrode:

1. Bank C (electrodes 576-959)
2. Tetrodes (Clusters of four electrodes, starting with 0-3, then 8-11, then 16-19, etc.)
3. Bank B (electrodes 384-767)

2025-09-05_14-09-32.zip

[bparks13]

Attached here are additional recordings using two Neuropixels 2.0 probes in one headstage, with the given electrode configurations for each experiment

| Experiment# | 23176523171          | 12345601013        |
| :---------- | :----------          | :----------------- |
| 1           | All Shanks 1-96      | Shank 4 Bank A     |
| 2           | All Shanks 577-762   | Shank 2 Bank C     |
| 2           | All Shanks 1153-1248 | Shank 3 Bank B     |
| 4           | Shank 1 Bank C       | All Shanks 385-480 |

ONIX_dual_2.0-multishank.zip

[chrishalcrow]

Attached here are additional recordings using two Neuropixels 2.0 probes in one headstage, with the given electrode configurations for each experiment

| Experiment# | 23176523171          | 12345601013        |
| :---------- | :----------          | :----------------- |
| 1           | All Shanks 1-96      | Shank 4 Bank A     |
| 2           | All Shanks 577-762   | Shank 2 Bank C     |
| 2           | All Shanks 1153-1248 | Shank 3 Bank B     |
| 4           | Shank 1 Bank C       | All Shanks 385-480 |

ONIX_dual_2.0-multishank.zip

Question about this dataset, looking at settings.xml. PROBE0 is a NP2004 probe, which is a single shank NP2.0 probe (https://www.neuropixels.org/probe-2-0-single-shank). This only has 1280 channels, but the selected channels in the settings file contains indices going up to about 4000. This makes me think I'm misunderstanding the selection indices. Can someone enlighten me?
(maybe @alejoe91 can)

[bparks13]

@anjaldoshi Can you provide any insight into this? The probe part number is supposed to be read directly from hardware, so if this is incorrect I will need to put a fix in.

[bparks13]

@chrishalcrow We figured it out, the PROBE0 is indeed a single-shank probe. The plugin does not currently check or know which type of probe is connected, it simply assumes that it is a quad-shank probe and configures it as such. That is why the electrode selection indices appear to be way outside the range.

I have created a new issue on the plugin repo to add support for single-shank probes so that we do not report incorrect electrode indices in the future.

Sorry about the confusion with that!

[chrishalcrow]

That's great - no worries about the confusion, glad we caught the bug.

[alejoe91]

Hi @bparks13

I added the first set of settings as tests and cleaned up the probeinterface reader on our end.
Now the preferred way to instantiate a probe is with the SELECTED_CHANNELS field (maybe we should rename int SELECTED_ELECTRODES though!). This opens up for a refactoring of the Neuropix-PXI and OneBox plugins to also use this strategy.

Can you check if everything looks good?

[alejoe91]

PROBE0

Waiting for this to be fixed to add some NP2 and multiprobe test files :)