UltraBench is a standardized benchmark of 14 different convex ultrasound image classification and semantic segmentation tasks (7 classification and 7 segmentation tasks) drawn from the 10 different publicly available datasets covering 11 regions of the body. This repository contains the processing scripts to transform raw datasets from their original sources into a standardized format for training and evaluating machine learning models.
This benchmark was published alongside the article Revisiting Data Augmentation for Ultrasound Images. If you use this benchmark in your research please cite this article and the articles for each of the included datasets. Instructions for how to do so are provided at the end.
A common limitation of studies on ultrasound image analysis using machine learning is a lack of evaluations across multiple domains. However, this is made difficult by a lack of ultrasound image analysis tasks in existing medical image analysis benchmarks, such as MedMNIST, MedSegBench and the Medical Segmentation Decathlon. To address this issue, we created UltraBench. Our aim is to make it easier for researchers by standardizing the preprocessing of many publicly available ultrasound datasets and providing tools (e.g., dataset implementations for common deep learning frameworks) that (a) reduce the effort required to use them and (b) make it easier for researchers to compare results between papers.
Install UltraBench using:
pip install ultrabenchUltraBench is packaged as a commandline application. After installing, run
ultrabench --help to get started.
For more information, checkout the Documentation on the wiki!
The following table contains the list of supported datasets and tasks. Click on the links to checkout the appropriate sections of the documentation or on the article/repository links to visit the original publications. We do not have the rights to republish the datasets, but links to the original articles and datasets are included here and in the Documentation.
| Dataset | Region | Classification Tasks | Segmentation Tasks |
|---|---|---|---|
| Annotated Ultrasound Liver (article, data) | Liver | Liver mass classification | Liver segmentation, Liver mass segmentation |
| Butterfly (data) | Multi-region | Region classification | |
| CAMUS (article, data) | Heart | Image quality classification | Cardiac region segmentation |
| Dataset of B-mode fatty liver ultrasound images (article, data) | Liver | NFLD classification | |
| Gallbladder Cancer Ultrasound (GBCU) (article, data) | Gallbladder | Gallbladder tumor classification | |
| Multi-Modality Ovarian Tumor Ultrasound (MMOTU) (article, data) | Ovaries | Ovarian tumor classifcation | Ovarian tumor segmentation |
| Open Kidney Ultrasound (article, data) | Kidney | Kidney capsule segmentation | |
| Point-of-care Ultrasound (POCUS) (article, data) | Lung | COVID-19 classification | |
| PSFHS (article, data) | Fetus | Fetal head and pubic symphysis segmentation | |
| Stanford Thyroid (data) | Thyroid | Thyroid nodule segmentation |
We would love for this benchmark to grow and flourish into a resource that anyone with an interest in machine learning for ultrasound analysis can pick up and use quickly and easily. Any help addressing bugs, contributing new datasets or tasks, or any other improvements are welcome and appreciated! I only ask that you respect the community guidelines laid out in the CODE_OF_CONDUCT.md. For more information on how to contribute, checkout out the Documentation.
UltraBench is built using uv, so for the best experience make sure to install uv before getting started. Once that's done, you can setup the development environment for the project by following the commands below. To ensure that your code meets the style guidelines etc., this also installs and initializes the optional development dependencies.
uv venv
source .venv/bin/activate
nbstripout --install
pre-commit installIf you have any questions, please open an issue or contact us via email.
If you use UltraBench in your research, please cite our article Revisiting Data Augmentation for Ultrasound Images and the articles for each of the datasets. This ensures that the work of the authors of the original datasets is properly acknowledged, helps more people find the benchmark and encourages us to continue maintaining and improving it!
@article{
tupper2025,
title={Revisiting Data Augmentation for Ultrasound Images},
author={Adam Tupper and Christian Gagn{\'e}},
journal={Transactions on Machine Learning Research},
issn={2835-8856},
year={2025},
url={https://openreview.net/forum?id=iGcxlTLIL5},
note={}
}