Within this repository you'll find example MRI data from GE, Siemens, and Philips environments. The file formats and directory structures have been set up to mimic (as best as possible) how these data would appear during a real-time scan. The only exception is that the data have been de-identified by removing/altering some of the raw header fields.
[toc]
The goal of this section is to test the complete Pyneal Pipeline using conditions that are similar to what you'd find across one of the 3 major scanner manufacturers.
To do so, we'll use the Scanner Simulator command line tool that comes with Pyneal. This tool will mock the behavior of an actual scanner by writing image data to an output directory in sequence and at a steady rate. Users can specify the rate at which new images appear by setting the --TR
parameter in the tool.
The source data, also included, is actual scan images from 3 different scanner platforms: GE, Philips, and Siemens. These data are meant to simulate the format and directory structure you might encounter while scanning on each of these platforms. This tutorial will allow users to test the complete Pyneal pipeline on any of these platforms prior to any real data collection.
There are instructions below for how to complete this tutorial using each of the platforms. But each platform follows the same general steps:
- Set up Scan Simulator
- Set up Pyneal Scanner
- Set up Pyneal
Platform-specific instructions:
Note: These instructions assume that you have downloaded Pyneal and the Pyneal-tutorial repositories to your local home directory. If so, you should have the following two directories in your local home directory:
~/pyneal
~/pyneal-tutorial
If you've downloaded these respositories to different locations, update the commands listed below accordingly.
Inside the GE_demo
folder, there's a directory named scanner
. This directory serves as our mock scanner for this tutorial, and follows a structure similar to what you might find on GE scanners. There's a single exam directory (p1/e123
) that contains two series directories: s1923
(anatomical image) and s1925
(functional series) (for more source data detail, see Appendix: GE source data)
We will use the Scanner Simulator tool to simulate a new functional series, using s1925
as our source data. The new series will appear in the exam directory alongside the existing series directories, and will be named s1926
.
-
open a new terminal and navigate to the Scanner Simulator tool:
cd ~/pyneal/pyneal_scanner/simulation/scannerSimulators
-
launch
GE_sim.py
, specifying paths to the source data and output directory, and setting a TR of 1000ms:python GE_sim.py ~/pyneal-tutorial/GE_demo/scanner/p1/e123/s1925 -t 1000 -o ~/pyneal-tutorial/GE_demo/scanner/p1/e123/s1926
-
You should see details about the current scan, and an option to press
ENTER
to begin the scan:------------------------- Source slices: ~/pyneal-tutorial/GE_demo/scanner/p1/e123/s1925 Output dir: ~/pyneal-tutorial/GE_demo/scanner/p1/e123/s1926 Total Slices Found: 1080 TR: 1000 Vols: 60 slices per vol: 18 delay between slices: 0.05555555555555556 Press ENTER to begin the "scan"
-
Before starting the simulator, complete the Set up Pyneal Scanner and Set up Pyneal steps.
Configure Pyneal Scanner to watch for new scan data in the exam directory for the GE mock scanner.
-
open a 2nd terminal window, and navigate to Pyneal Scanner:
cd ~/pyneal/pyneal_scanner
-
create (or edit the existing)
scannerConfig.yaml
file in this directory to set thescannerMake
toGE
and thescannerBaseDir
to the mock scanner folder. The contents of thescannerConfig.yaml
file should look like:pynealSocketHost: 127.0.0.1 pynealSocketPort: '5555' scannerBaseDir: ~/pyneal-tutorial/GE_demo/scanner scannerMake: GE
-
launch Pyneal Scanner:
python pynealScanner.py
-
You should see details about the current session, and an indication that Pyneal Scanner is attempting to connect to Pyneal:
=============== SCANNER SETTINGS: pynealSocketHost: 127.0.0.1 pynealSocketPort: 5555 scannerBaseDir: ~/pyneal-tutorial/GE_demo/scanner scannerMake: GE =============== Session Dir: ~/pyneal-tutorial/GE_demo/scanner/p1/e123 Series Dirs: s1925 26.0 MB 748310 min, 8 s ago s1923 23.5 MB 748308 min, 36 s ago MainThread - Connecting to pynealSocket...
-
There is nothing more to do in this terminal window. Once Pyneal is set up and the Scan Simulator tool starts, Pyneal Scanner will begin processing new images as they appear and sending the data to Pyneal. You can monitor the progress via the log messages that appear in this terminal.
-
Open a third terminal window, and navigate to Pyneal. Launch Pyneal
cd ~/pyneal python pyneal.py
-
Configure Pyneal for the GE tutorial demo:
- Communication: Make sure
Pyneal Host IP
is set to127.0.0.1
and thePyneal-Scanner Port
is5555
. - Mask: In the
GE_demo
directory, there is a file nameddummyMask_64-64-18.nii.gz
. Set the mask value in Pyneal to use this file. This mask was pre-made to match the volume dimensions of theGE_demo
scan data. This mask is simply a rectangle positioned in the middle slice of the 3D volume, and is for demonstration purposes only. - Preprocessing: Set
# of timepts
to:60
- Analysis: Select the
Average
option - Output: Set the output directory to
~/pyneal-tutorial/GE_demo/output
. CheckLaunch Dashboard?
.
- Communication: Make sure
-
Start Pyneal by pressing
Submit
.- In the Pyneal Scanner terminal, you will see messages indicating that Pyneal Scanner has succesfully set up a connection to Pyneal and that it is waiting for a new
seriesDir
(which will be created once the scan starts. - In addition, you can open a browser window and enter
127.0.0.1:5558
in the URL bar to see the Pyneal dashboard.
- In the Pyneal Scanner terminal, you will see messages indicating that Pyneal Scanner has succesfully set up a connection to Pyneal and that it is waiting for a new
-
In the first terminal window, where the Scan Simulator tool is running, press
ENTER
to begin the scan. -
As the scan is progressing, each of the three terminal windows will update with new log messages. In addition, you can monitor the progress from dashboard in your browser at
127.0.0.1:5558
. -
As soon at the scan finishes, you can find the Pyneal output at
~/pyneal-tutorial/GE_demo/output/pyneal_001
. This directory will have:- pynealLog.log: log file from the current scan
- receivedFunc.nii.gz: 4D nifi file of the data, as received by Pyneal * results.json: JSON file containing the analysis results from the current scan.
Inside the Philips_demo
folder, there's a directory named scanner
. This directory serves as our mock scanner for this tutorial, and follows a structure similar to what you might find using Philips scanners. There's a single exam directory (Subject001
) that contains two functional series directories: 0001
and 0002
as well as an anatomical image MPRAGE
(for more source data detail, see Appendix: Philips source data)
We will use the Scanner Simulator tool to simulate a new functional series, using 0002
as our source data. The new series will appear in the exam directory alongside the existing series directories, and will be named 0003
.
-
open a new terminal and navigate to the Scanner Simulator tool:
cd ~/pyneal/pyneal_scanner/simulation/scannerSimulators
-
launch
Philips_sim.py
, specifying paths to the source data and output directory, and setting a TR of 2000ms:python Philips_sim.py ~/pyneal-tutorial/Philips_demo/scanner/Subject001/0002 -t 2000 -o ~/pyneal-tutorial/Philips_demo/scanner/Subject001/0003
-
You should see details about the current scan, and an option to press
ENTER
to begin the scan:------------------------- Source slices: ~/pyneal-tutorial/Philips_demo/scanner/Subject001/0002 Output dir: ~/pyneal-tutorial/Philips_demo/scanner/Subject001/0003 Total Volumes: 93 Image Dims: (80, 80, 30) TR: 2000 Press ENTER to begin the "scan"
-
Before starting the simulator, complete the Set up Pyneal Scanner and Set up Pyneal steps.
Configure Pyneal Scanner to watch for new scan data in the exam directory for the Philips mock scanner.
-
open a 2nd terminal window, and navigate to Pyneal Scanner:
cd ~/pyneal/pyneal_scanner
-
create (or edit the existing)
scannerConfig.yaml
file in this directory to set thescannerMake
toPhilips
and thescannerBaseDir
to the mock scanner folder. The contents of thescannerConfig.yaml
file should look like:pynealSocketHost: 127.0.0.1 pynealSocketPort: '5555' scannerBaseDir: ~/pyneal-tutorial/Philips_demo/scanner/Subject001 scannerMake: Philips
-
launch Pyneal Scanner:
python pynealScanner.py
-
You should see details about the current session, and an indication that Pyneal Scanner is attempting to connect to Pyneal:
=============== SCANNER SETTINGS: pynealSocketHost: 127.0.0.1 pynealSocketPort: 5555 scannerBaseDir: ~/pyneal-tutorial/Philips_demo/scanner/Subject001 scannerMake: Philips =============== Session Dir: ~/pyneal-tutorial/Philips_demo/scanner/Subject001 Series Dirs: 0002 73.4 MB 748534 min, 58 s ago 0001 74.0 MB 748522 min, 55 s ago MPRAGE 23.1 MB 748521 min, 44 s ago MainThread - Connecting to pynealSocket...
-
There is nothing more to do in this terminal window. Once Pyneal is set up and the Scan Simulator tool starts, Pyneal Scanner will begin processing new images as they appear and sending the data to Pyneal. You can monitor the progress via the log messages that appear in this terminal.
-
Open a third terminal window, and navigate to Pyneal. Launch Pyneal
cd ~/pyneal python pyneal.py
-
Configure Pyneal for the Philips tutorial demo:
- Communication: Make sure
Pyneal Host IP
is set to127.0.0.1
and thePyneal-Scanner Port
is5555
. - Mask: In the
Philips_demo
directory, there is a file nameddummyMask_80-80-30.nii.gz
. Set the mask value in Pyneal to use this file. This mask was pre-made to match the volume dimensions of thePhilips_demo
scan data. This mask is simply a rectangle positioned in the middle slice of the 3D volume, and is for demonstration purposes only. - Preprocessing: Set
# of timepts
to:93
- Analysis: Select the
Average
option - Output: Set the output directory to
~/pyneal-tutorial/Philips_demo/output
. CheckLaunch Dashboard?
.
- Communication: Make sure
-
Start Pyneal by pressing
Submit
.- In the Pyneal Scanner terminal, you will see messages indicating that Pyneal Scanner has succesfully set up a connection to Pyneal and that it is waiting for a new
seriesDir
(which will be created once the scan starts. - In addition, you can open a browser window and enter
127.0.0.1:5558
in the URL bar to see the Pyneal dashboard.
- In the Pyneal Scanner terminal, you will see messages indicating that Pyneal Scanner has succesfully set up a connection to Pyneal and that it is waiting for a new
-
In the first terminal window, where the Scan Simulator tool is running, press
ENTER
to begin the scan. -
As the scan is progressing, each of the three terminal windows will update with new log messages. In addition, you can monitor the progress from dashboard in your browser at
127.0.0.1:5558
. -
As soon at the scan finishes, you can find the Pyneal output at
~/pyneal-tutorial/Philips_demo/output/pyneal_001
. This directory will have:- pynealLog.log: log file from the current scan
- receivedFunc.nii.gz: 4D nifi file of the data, as received by Pyneal * results.json: JSON file containing the analysis results from the current scan.
Inside the Siemens_demo
folder, there's a directory named scanner
. This directory serves as our mock scanner for this tutorial, and follows a structure similar to what you might find using Siemens scanners. There's a single session directory (data
) that contains all of the dicom files for two functional series (000013
, 000015
) and an anatomical series (for more source data detail, see Appendix: Siemens source data)
We will use the Scanner Simulator tool to simulate a new functional series, using 000013
as our source data. The new series will appear in the session directory alongside the existing series files, and dicom files will contain the series name 000014
.
-
open a new terminal and navigate to the Scanner Simulator tool:
cd ~/pyneal/pyneal_scanner/simulation/scannerSimulators
-
launch
Siemens_sim.py
, specifying paths to the source directory, series numbers, output directory, and setting a TR of 1000ms:python Siemens_sim.py ~/pyneal-tutorial/Siemens_demo/scanner/data 000013 -t 1000 -n 000014
-
You should see details about the current scan, and an option to press
ENTER
to begin the scan:------------------------- Source dir: ~/pyneal-tutorial/Siemens_demo/scanner/data Total Mosaics Found: 60 TR: 1000 Press ENTER to begin the "scan"
-
Before starting the simulator, complete the Set up Pyneal Scanner and Set up Pyneal steps.
Configure Pyneal Scanner to watch for new scan data in the session directory for the Siemens mock scanner.
-
open a 2nd terminal window, and navigate to Pyneal Scanner:
cd ~/pyneal/pyneal_scanner
-
create (or edit the existing)
scannerConfig.yaml
file in this directory to set thescannerMake
toSiemens
and thescannerBaseDir
to the mock scanner folder. The contents of thescannerConfig.yaml
file should look like:pynealSocketHost: 127.0.0.1 pynealSocketPort: '5555' scannerBaseDir: ~/pyneal-tutorial/Siemens_demo/scanner/data scannerMake: Siemens
-
launch Pyneal Scanner:
python pynealScanner.py
-
You should see details about the current session, and an indication that Pyneal Scanner is attempting to connect to Pyneal:
=============== Session Dir: ~/pyneal-tutorial/Siemens_demo/scanner/data Unique Series: 000013 60 files 1113170 min, 51 s ago 000015 60 files 1113170 min, 51 s ago 000017 52 files 1113170 min, 51 s ago MainThread - Connecting to pynealSocket...
-
There is nothing more to do in this terminal window. Once Pyneal is set up and the Scan Simulator tool starts, Pyneal Scanner will begin processing new images as they appear and sending the data to Pyneal. You can monitor the progress via the log messages that appear in this terminal.
-
Open a third terminal window, and navigate to Pyneal. Launch Pyneal
cd ~/pyneal python pyneal.py
-
Configure Pyneal for the Siemens tutorial demo:
- Communication: Make sure
Pyneal Host IP
is set to127.0.0.1
and thePyneal-Scanner Port
is5555
. - Mask: In the
Siemens_demo
directory, there is a file nameddummyMask_64-64-18.nii.gz
. Set the mask value in Pyneal to use this file. This mask was pre-made to match the volume dimensions of theSiemens_demo
scan data. This mask is simply a rectangle positioned in the middle slice of the 3D volume, and is for demonstration purposes only. - Preprocessing: Set
# of timepts
to:60
- Analysis: Select the
Average
option - Output: Set the output directory to
~/pyneal-tutorial/Siemens_demo/output
. CheckLaunch Dashboard?
.
- Communication: Make sure
-
Start Pyneal by pressing
Submit
.- In the Pyneal Scanner terminal, you will see messages indicating that Pyneal Scanner has succesfully set up a connection to Pyneal and that it is waiting for a new
seriesDir
(which will be created once the scan starts. - In addition, you can open a browser window and enter
127.0.0.1:5558
in the URL bar to see the Pyneal dashboard.
- In the Pyneal Scanner terminal, you will see messages indicating that Pyneal Scanner has succesfully set up a connection to Pyneal and that it is waiting for a new
-
In the first terminal window, where the Scan Simulator tool is running, press
ENTER
to begin the scan. -
As the scan is progressing, each of the three terminal windows will update with new log messages. In addition, you can monitor the progress from dashboard in your browser at
127.0.0.1:5558
. -
As soon at the scan finishes, you can find the Pyneal output at
~/pyneal-tutorial/Siemens_demo/output/pyneal_001
. This directory will have:- pynealLog.log: log file from the current scan
- receivedFunc.nii.gz: 4D nifi file of the data, as received by Pyneal * results.json: JSON file containing the analysis results from the current scan.
GE_demo/
├── dummyMask_64-64-18.nii.gz
├── output
└── scanner
└── p1
└── e123
├── s1923
└── s1925
GE scanners typically write new slices dicom files to a directory on the scanner console. The path to that directory can be broken apart like [scannerBaseDir]/[sessionDir]/[seriesDir]
, where
- [scannerBaseDir]: path that remains constant across all scans
- [sessionDir]: directories that can change from session to session, named like
p###/e###
where the specific#
values are unknown in advance. - [seriesDir]: series specific directory named like
s###
where the specific#
values are unknown in advance. Each new scan during a given exam session will be assigned a uniques###
dir.
The GE_demo
folder contains two different series directories, s1923
and s1925
. These are nested in the session directory scanner/p1/e123
Anatomical Image
- FSPGR Bravo
- 162 slices
- Slice dims: 256 x 256
- Voxel dims: 1 x 1 x 1mm
Functional Series - EPI sequence
- TE: 28ms
- TR: 1000ms
- Flip Angle: 90
- Slices/Vol: 18
- Slice dims: 64 x 64 x 18
- Voxel dims: 3 x 3 x 3.8mm
- Total Vols: 60
Philips_demo/
├── dummyMask_80-80-30.nii.gz
├── output
└── scanner
└── Subject001
├── 0001
│ ├── Dump-0000.par
│ ├── Dump-0000.rec
│ ├── Dump-0001.par
│ ├── Dump-0001.rec
│ ...
│ ├── Dump-0185.par
│ └── Dump-0185.rec
├── 0002
│ ├── Dump-0000.par
│ ├── Dump-0000.rec
│ ├── Dump-0001.par
│ ├── Dump-0001.rec
│ ...
│ ├── Dump-0092.par
│ └── Dump-0092.rec
└── MPRAGE
├── Subject001_WIP_MPRAGE_SENSE_2_1.PAR
└── Subject001_WIP_MPRAGE_SENSE_2_1.REC
In a Phillips environment, 3D volumes are typically exported from the scanner and saved to a remote directory during the scan. Each volume appears as a file pair: a par
header file, and corresponding rec
binary file. Each new series during the session will get its own series directory, where all of the par
/rec
files will appear. The series directories for functional series will be named sequentially, starting with 0000
.
Functional Series - EPI sequence
- TR: 1000ms
- Flip Angle: 79
- Slices/Vol: 15
- Slice dims: 80 x 80 x 15
- Voxel dims: 3 x 3 x 4mm
- Total Vols: 186
Functional Series - EPI sequence
- TR: 2000ms
- Flip Angle: 79
- Slices/Vol: 30
- Slice dims: 80 x 80 x 30
- Voxel dims: 3 x 3 x 4mm
- Total Vols: 93
Anatomical Image - EPI sequence
- Volume dims: 176 x 256 x 256
- Voxel dims: 1 x 1 x 1mm
Siemens_demo/
├── dummyMask_64-46-18.nii.gz
├── output
└── scanner
└── data
├── 001_000013_000001.dcm
├── ...
├── 001_000015_000001.dcm
├── ...
├── 001_000017_000001.dcm
└── ...
In a Siemens environment, all dicom files for all series in a given session will be exported to the same directory. That directory is named data
here, but may appear with a different name in an actual scan setting.
Each dicom file represents a 3D volume, with all slices stored in a 2D mosaic format. File names contain both the series number
and volume number
, and follow the pattern:
001_<seriesNumber>_<volumeNumber>.dcm
The Siemens_demo
directory contains dicom files for 3 different series: 000013
, 000015
, and 000017
.
Functional Series - EPI sequence
- TE: 28ms
- TR: 1000ms
- Flip Angle: 90
- Slices/Vol: 18
- Slice dims: 64 x 64 x 18
- Voxel dims: 3 x 3 x 3.8mm
- Total Vols: 60
Functional Series - Multiband EPI sequence
- TE: 37ms
- TR: 800ms
- Flip Angle: 52
- Slices/Vol: 72
- Slice dims: 104 x 104 x 72
- Voxel dims: 2 x 2 x 2mm
- Total Vols: 60
Anatomical Image (Note: vitamin tablet placed on participant's left temple (left relative to the participant) in order to confirm orientation)
- Volume dims: 52 x 180 x 192
- Voxel dims: 3 x 1.33 x 1.33mm