The above is a cineloop of cardiac data with conventional delay-and-sum beamforming and ReFoCUS beamforming. We apply clinical-grade post-processing, MimickNet post-processing, and show the unscaled difference between the two.
You can use the model immediately in this google colab notebook
The model from the paper is provided as mimicknet_legacy.h5. We also provide a luminance adjusted version with fewer weights with mimicknet_1568473738-210304.h5 and padded_mimicknet_1568473738-210304.h5. The padded prefix allows for padding to be apart of the model, so no 16 padding logic is required. However, the layer used is incompatible with matlab, thus the matlab version has no pre-padding.
The mimicknet_phantom_verasonics.h5 models were generated with only phantom data from verasonics.
A notebook and sample data is provided under examples for use in the following environments:
Dr. Mark Palmeri's liver and kidney images are provided as test examples (Thanks Mark!)
| Model All | SSIM | PSNR |
|---|---|---|
| MimickNet (BlackBox) | 0.94 ± 0.014 | 31.95 ± 2.04 |
| GrayBox | 0.96 ± 0.012 | 32.86 ± 1.82 |
| MimickNet Phantom | 0.95 ± 0.007 | 33.50 ± 1.43 |
| MimickNet Mark | 0.96 ± 0.005 | 33.12 ± 0.92 |
Results above used non-public invivio + phantom data. Testing is on non-public invivo + phantom data. MimickNet Mark are results only on Mark's liver/cardiac data. MimickNet Phantom are results only on the public phantom data test split.
| Model Phantom | SSIM | PSNR |
|---|---|---|
| MimickNet Phantom | 0.90 ± 0.015 | 31.40 ± 2.76 |
| MimickNet Mark | 0.91 ± 0.005 | 31.43 ± 0.69 |
Results above used only public phantom data for training. MimickNet Phantom are results only on the public phantom data test data. MimickNet Mark are results only on Mark's liver/cardiac data.
The following repo was designed to be run in Google Cloud and makes use of GCS for logging.
python3 -m trainer.blackbox_task
Different hyperparameters can be selected. Different tasks are shown in the root directory of the trainer. blackbox_task.py, and graybox_task.py are for running the blackbox and graybox tasks in the paper: MimickNet, Matching Clinical Post-Processing Under Realistic Black-Box Constraints. blackbox_paper_v1.py, is the exact code run for the first release.
To train this repo within a docker container, first clone the repo and run the following command in the root directory.
docker build -t mimicknet:latest .
Then you can run the docker container with the following command
docker run -it mimicknet:latest
If you have GPUs you can run docker with the --gpus flag.
The current model uses the dataset loader found in ./trainer/utils/dataset.py. This dataset takes in two lists from different domains A and B. The dataset returns elements which are 4D tensors. Each dimension represents [batch, height, width, 1]. You can create your own dataset class, it simply needs to return a 4D tensor, and count the total number of elements. See here on how to use tensorflow datasets.
Contributions are welcome in the form of pull requests. For issues, please provide a minimal example of code which results in an error.
Note that the model provided here is under a different license than the software code.
CC 4.0 Attribution-NonCommercial International
Copyright 2019 Ouwen Huang
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
This work was supported by the National Institute of Biomedical Imaging and Bioengineering under Grant R01- EB026574, and National Institutes of Health under Grant 5T32GM007171-44. The authors would like to thank Siemens Medical Inc. USA for in kind technical support.
@ARTICLE{8977476,
author={O. {Huang} and W. {Long} and N. {Bottenus} and M. {Lerendegui} and G. E. {Trahey} and S. {Farsiu} and M. L. {Palmeri}},
journal={IEEE Transactions on Medical Imaging},
title={MimickNet, Mimicking Clinical Image Post- Processing Under Black-Box Constraints},
year={2020},
volume={39},
number={6},
pages={2277-2286},
doi={10.1109/TMI.2020.2970867}}