A collection of deep learning architectures and applications ported to the R language and tools for basic medical image processing. Based on keras and tensorflow with cross-compatibility with our python analog ANTsPyNet.
- A large collection of common deep learning architectures for medical imaging that can be initialized
- Various pre-trained deep learning models to perform key medical imaging tasks
- Utility functions to improve training and evaluating of deep learning models on medical images
Installation
You will need R (>=3.2) and C/C++ development tools including CMake (>= 3.16.3).
First, install keras in R
> install.packages(keras)
> keras::install_keras()Then from the command line:
git clone https://github.com/stnava/ITKR.git
git clone https://github.com/ANTsX/ANTsRCore.git
git clone https://github.com/ANTsX/ANTsR.git
R CMD INSTALL ITKR
R CMD INSTALL ANTsRCore
R CMD INSTALL ANTsR
R CMD INSTALL ANTsRNet
Architectures
- AlexNet (2-D, 3-D)
- VGG (2-D, 3-D)
- ResNet (2-D, 3-D)
- ResNeXt (2-D, 3-D)
- WideResNet (2-D, 3-D)
- DenseNet (2-D, 3-D)
- Super-resolution convolutional neural network (SRCNN) (2-D, 3-D)
- Expanded super-resolution (ESRCNN) (2-D, 3-D)
- Denoising auto encoder super-resolution (DSRCNN) (2-D, 3-D)
- Deep denoise super-resolution (DDSRCNN) (2-D, 3-D)
- ResNet super-resolution (SRResNet) (2-D, 3-D)
- Deep back-projection network (DBPN) (2-D, 3-D)
- Super resolution GAN
Applications
-
- Multi-modal brain extraction
- Deep Atropos (Six-tissue brain segmentation)
- Cortical thickness
- Desikan-Killiany-Tourville parcellation
- DeepFLASH (medial temporal lobe parcellation)
- Hippmapp3r (hippocampal segmentation)
- Brain AGE
- Claustrum segmentation
- Hypothalamus segmentation
- Cerebellum morphology
- White matter hyperintensities segmentation
- Perivascular spaces segmentation (SHIVA)
- Brain tumor segmentation
- MRA-TOF vessel segmentation
- Lesion segmentation (WIP)
- Whole head inpainting
Publications
-
Nicholas J. Tustison, Min Chen, Fae N. Kronman, Jeffrey T. Duda, Clare Gamlin, Mia G. Tustison, Michael Kunst, Rachel Dalley, Staci Sorenson, Quanxi Wang, Lydia Ng, Yongsoo Kim, and James C. Gee. The ANTsX Ecosystem for Mapping the Mouse Brain. (biorxiv)
-
Nicholas J. Tustison, Michael A. Yassa, Batool Rizvi, Philip A. Cook, Andrew J. Holbrook, Mithra Sathishkumar, Mia G. Tustison, James C. Gee, James R. Stone, and Brian B. Avants. ANTsX neuroimaging-derived structural phenotypes of UK Biobank. Scientific Reports, 14(1):8848, Apr 2024. (pubmed)
-
Nicholas J. Tustison, Talissa A. Altes, Kun Qing, Mu He, G. Wilson Miller, Brian B. Avants, Yun M. Shim, James C. Gee, John P. Mugler III, and Jaime F. Mata. Image- versus histogram-based considerations in semantic segmentation of pulmonary hyperpolarized gas images. Magnetic Resonance in Medicine, 86(5):2822-2836, Nov 2021. (pubmed)
-
Andrew T. Grainger, Arun Krishnaraj, Michael H. Quinones, Nicholas J. Tustison, Samantha Epstein, Daniela Fuller, Aakash Jha, Kevin L. Allman, Weibin Shi. Deep Learning-based Quantification of Abdominal Subcutaneous and Visceral Fat Volume on CT Images, Academic Radiology, 28(11):1481-1487, Nov 2021. (pubmed)
-
Nicholas J. Tustison, Philip A. Cook, Andrew J. Holbrook, Hans J. Johnson, John Muschelli, Gabriel A. Devenyi, Jeffrey T. Duda, Sandhitsu R. Das, Nicholas C. Cullen, Daniel L. Gillen, Michael A. Yassa, James R. Stone, James C. Gee, and Brian B. Avants for the Alzheimer’s Disease Neuroimaging Initiative. The ANTsX ecosystem for quantitative biological and medical imaging. Scientific Reports. 11(1):9068, Apr 2021. (pubmed)
-
Nicholas J. Tustison, Brian B. Avants, and James C. Gee. Learning image-based spatial transformations via convolutional neural networks: a review, Magnetic Resonance Imaging, 64:142-153, Dec 2019. (pubmed)
-
Nicholas J. Tustison, Brian B. Avants, Zixuan Lin, Xue Feng, Nicholas Cullen, Jaime F. Mata, Lucia Flors, James C. Gee, Talissa A. Altes, John P. Mugler III, and Kun Qing. Convolutional Neural Networks with Template-Based Data Augmentation for Functional Lung Image Quantification, Academic Radiology, 26(3):412-423, Mar 2019. (pubmed)
-
Andrew T. Grainger, Nicholas J. Tustison, Kun Qing, Rene Roy, Stuart S. Berr, and Weibin Shi. Deep learning-based quantification of abdominal fat on magnetic resonance images. PLoS One, 13(9):e0204071, Sep 2018. (pubmed)
-
Cullen N.C., Avants B.B. (2018) Convolutional Neural Networks for Rapid and Simultaneous Brain Extraction and Tissue Segmentation. In: Spalletta G., Piras F., Gili T. (eds) Brain Morphometry. Neuromethods, vol 136. Humana Press, New York, NY doi
Acknowledgements
-
We gratefully acknowledge the support of the NVIDIA Corporation with the donation of two Titan Xp GPUs used for this research.
-
We gratefully acknowledge the grant support of the Office of Naval Research and Cohen Veterans Bioscience.