v0.2.1 : Update defaults and new utilities (#98)

* Enable thresholding by default

* Update plugin_model_inference.py

* Enable window + default thresholds

* Update V-O defaults

* Add grid search utility

* Document grid search util

* Fix default threshold on first inference open

* Version bump + Wnet3d assumptions docs

* Update README.md

* Bump to 0.2.1 + typos fixes

* Update inference_module_guide.rst

* Update inference_module_guide.rst

* Update utils_module_guide.rst

* Update welcome.rst

---------

Co-authored-by: Mackenzie Mathis <[email protected]>

README.md

@@ -1,4 +1,4 @@
-# CellSeg3D: self-supervised (and supervised) 3D cell segmentation, primarily for mesoSPIM data! 
+# CellSeg3D: self-supervised (and supervised) 3D cell segmentation, primarily for mesoSPIM data!
 [![napari hub](https://img.shields.io/endpoint?url=https://api.napari-hub.org/shields/napari-cellseg3d)](https://www.napari-hub.org/plugins/napari-cellseg3d)
 [![PyPI](https://img.shields.io/pypi/v/napari-cellseg3d.svg?color=green)](https://pypi.org/project/napari-cellseg3d)
 [![Downloads](https://static.pepy.tech/badge/napari-cellseg3d)](https://pepy.tech/project/napari-cellseg3d)
@@ -59,11 +59,17 @@ The strength of our approach is we can match supervised model performance with p
 
 ## News
 
-**New version: v0.2.0**
+**New version: v0.2.1**
 
-- Changed project name to "napari_cellseg3d" to avoid setuptools deprecation
-- Small API changes for training/inference from a script
-- Some fixes to WandB integration ad csv saving after training
+- v0.2.1:
+  - Updated plugin default behaviors across the board to be more readily applicable to demo data
+  - Threshold value in inference is now automatically set by default according to performance on demo data on a per-model basis
+  - Added a grid search utility to find best thresholds for supervised models
+
+- v0.2.0:
+  - Changed project name to "napari_cellseg3d" to avoid setuptools deprecation
+  - Small API changes for training/inference from a script
+  - Some fixes to WandB integration and csv saving after training
 
 Previous additions:
 

docs/source/guides/inference_module_guide.rst

@@ -50,13 +50,13 @@ Interface and functionalities
 
     Inference parameters
 
-* **Loading data** :
+* **Loading data**:
 
   | When launching the module, select either an **image layer** or an **image folder** containing the 3D volumes you wish to label.
   | When loading from folder : All images with the chosen extension ( currently **.tif**) will be labeled.
   | Specify an **output folder**, where the labelled results will be saved.
 
-* **Model selection** :
+* **Model selection**:
 
   | You can then choose from the listed **models** for inference.
   | You may also **load custom weights** rather than the pre-trained ones. Make sure these weights are **compatible** (e.g. produced from the training module for the same model).
@@ -66,19 +66,19 @@ Interface and functionalities
     Currently the SegResNet and SwinUNetR models require you to provide the size of the images the model was trained with.
     Provided weights use a size of 64, please leave it on the default value if you're not using custom weights.
 
-* **Inference parameters** :
+* **Inference parameters**:
 
   * **Window inference**: You can choose to use inference on the entire image at once (disabled) or divide the image (enabled) on smaller chunks, based on your memory constraints.
   * **Window overlap**: Define the overlap between windows to reduce border effects;
     recommended values are 0.1-0.3 for 3D inference.
   * **Keep on CPU**: You can choose to keep the dataset in RAM rather than VRAM to avoid running out of VRAM if you have several images.
   * **Device Selection**: You can choose to run inference on either CPU or GPU. A GPU is recommended for faster inference.
 
-* **Anisotropy** :
+* **Anisotropy**:
 
  For **anisotropic images** you may set the **resolution of your volume in micron**, to view and save the results without anisotropy.
 
-* **Thresholding** :
+* **Thresholding**:
 
   You can perform thresholding to **binarize your labels**.
   All values below the **confidence threshold** will be set to 0.
@@ -87,7 +87,7 @@ Interface and functionalities
   It is recommended to first run without thresholding. You can then use the napari contrast limits to find a good threshold value,
   and run inference later with your chosen threshold.
 
-* **Instance segmentation** :
+* **Instance segmentation**:
 
   | You can convert the semantic segmentation into instance labels by using either the `Voronoi-Otsu`_, `Watershed`_ or `Connected Components`_ method, as detailed in :ref:`utils_module_guide`.
   | Instance labels will be saved (and shown if applicable) separately from other results.
@@ -98,7 +98,7 @@ Interface and functionalities
 .. _Voronoi-Otsu: https://haesleinhuepf.github.io/BioImageAnalysisNotebooks/20_image_segmentation/11_voronoi_otsu_labeling.html
 
 
-* **Computing objects statistics** :
+* **Computing objects statistics**:
 
   You can choose to compute various stats from the labels and save them to a **`.csv`** file for later use.
   Statistics include individual object details and general metrics.
@@ -109,7 +109,7 @@ Interface and functionalities
   * Sphericity
 
 
-  Global metrics :
+  Global metrics:
 
   * Image size
   * Total image volume (pixels)
@@ -118,7 +118,7 @@ Interface and functionalities
   * The number of labeled objects
 
 
-* **Display options** :
+* **Display options**:
 
   When running inference on a folder, you can choose to display the results in napari.
   If selected, you may choose the display quantity, and whether to display the original image alongside the results.
@@ -151,7 +151,16 @@ Unsupervised model - WNet3D
 | The `WNet3D model` is a fully self-supervised model used to segment images without any labels.
 | It functions similarly to the above models, with a few notable differences.
 
-.. _WNet3D model: https://arxiv.org/abs/1711.08506
+WNet3D has been tested on:
+
+* **MesoSPIM** data (whole-brain samples of mice imaged by mesoSPIM microscopy) with nuclei staining.
+* Other microscopy (i.e., confocal) data with:
+    * **Sufficient contrast** between objects and background.
+    * **Low to medium crowding** of objects. If all objects are adjacent to each other, instance segmentation methods provided here may not work well.
+
+Noise and object size are less critical, though objects still have to fit within the field of view of the model.
+
+.. _WNet3D model: https://elifesciences.org/reviewed-preprints/99848
 
 .. note::
     Our provided, pre-trained model uses an input size of 64x64x64. As such, window inference is always enabled

docs/source/guides/utils_module_guide.rst

@@ -11,6 +11,21 @@ See `Usage section <https://adaptivemotorcontrollab.github.io/CellSeg3d/welcome.
 
 You may specify the results directory for saving; afterwards you can run each action on a folder or on the currently selected layer.
 
+Default Paths for Saving Results
+________________________________
+
+Each utility saves results to a default directory under the user's home directory. The default paths are as follows:
+
+* Artifact Removal: ``~/cellseg3d/artifact_removed``
+* Fragmentation: ``~/cellseg3d/fragmented``
+* Anisotropy Correction: ``~/cellseg3d/anisotropy``
+* Small Object Removal: ``~/cellseg3d/small_removed``
+* Semantic Label Conversion: ``~/cellseg3d/semantic_labels``
+* Instance Label Conversion: ``~/cellseg3d/instance_labels``
+* Thresholding: ``~/cellseg3d/threshold``
+* Statistics: ``~/cellseg3d/stats``
+* Threshold Grid Search: ``~/cellseg3d/threshold_grid_search``
+
 Available actions
 __________________
 
@@ -89,6 +104,18 @@ Global metrics :
 | Clears labels that are larger than a given threshold.
 | This is useful for removing artifacts that are larger than the objects of interest.
 
+11. Find the best threshold
+-----------------------
+| Finds the best threshold for separating objects from the background.
+| Requires a prediction from a model and GT labels as input.
+
+.. caution::
+    If the input prediction is not from the plugin, it will be remapped to the 0-1 range.
+
+| The threshold is found by maximizing the Dice coefficient between the thresolded prediction and the binarized GT labels.
+
+| The value for the best threshold will be displayed, and the prediction will be thresholded and saved with this value.
+
 Source code
 ___________
 

docs/welcome.rst

@@ -9,17 +9,14 @@ Welcome to CellSeg3D!
 Use CellSeg3D to:
 
 * Review labeled cell volumes from whole-brain samples of mice imaged by mesoSPIM microscopy [1]_
-* Train and use segmentation models from the MONAI project [2]_ or implement your own custom 3D segmentation models using PyTorch.
+* Train and use segmentation models from the MONAI project [2]_ 
+* Train and use our WNet3D unsupervised model
+* Or implement your own custom 3D segmentation models using PyTorch!
 
-No labeled data? Try our unsupervised model, based on the `WNet`_ model, to automate your data labelling.
-
-The models provided should be adaptable to other tasks related to detection of 3D objects,
-outside of whole-brain light-sheet microscopy.
-This applies to the unsupervised model as well, feel free to try to generate labels for your own data!
 
 .. figure:: https://images.squarespace-cdn.com/content/v1/57f6d51c9f74566f55ecf271/0d16a71b-3ff2-477a-9d83-18d96cb1ce28/full_demo.gif?format=500w
    :alt: CellSeg3D demo
-   :width: 500
+   :width: 800
    :align: center
 
    Demo of the plugin
@@ -145,14 +142,14 @@ Other useful napari plugins
 
 Acknowledgments & References
 ---------------------------------------------
-This plugin has been developed by Cyril Achard and Maxime Vidal, supervised by Mackenzie Mathis for the `Mathis Laboratory of Adaptive Motor Control`_.
+If you find our code or ideas useful, please cite:
+
+Achard Cyril, Kousi Timokleia, Frey Markus, Vidal Maxime, Paychère Yves, Hofmann Colin, Iqbal Asim, Hausmann Sebastien B, Pagès Stéphane, Mathis Mackenzie Weygandt (2024) 
+CellSeg3D: self-supervised 3D cell segmentation for microscopy eLife https://doi.org/10.7554/eLife.99848.1
 
-We also greatly thank Timokleia Kousi for her contributions to this project and the `Wyss Center`_ for project funding.
 
-The TRAILMAP models and original weights used here were ported to PyTorch but originate from the `TRAILMAP project on GitHub`_.
-We also provide a model that was trained in-house on mesoSPIM nuclei data in collaboration with Dr. Stephane Pages and Timokleia Kousi.
 
-This plugin mainly uses the following libraries and software:
+This plugin additionally uses the following libraries and software:
 
 * `napari`_
 
@@ -162,9 +159,9 @@ This plugin mainly uses the following libraries and software:
 
 * `pyclEsperanto`_ (for the Voronoi Otsu labeling) by Robert Haase
 
-* A new unsupervised 3D model based on the `WNet`_ by Xia and Kulis [3]_
 
-.. _Mathis Laboratory of Adaptive Motor Control: http://www.mackenziemathislab.org/
+
+.. _Mathis Laboratory of Adaptive Intelligence: http://www.mackenziemathislab.org/
 .. _Wyss Center: https://wysscenter.ch/
 .. _TRAILMAP project on GitHub: https://github.com/AlbertPun/TRAILMAP
 .. _napari: https://napari.org/
@@ -178,4 +175,3 @@ This plugin mainly uses the following libraries and software:
 
 .. [1] The mesoSPIM initiative: open-source light-sheet microscopes for imaging cleared tissue, Voigt et al., 2019 ( https://doi.org/10.1038/s41592-019-0554-0 )
 .. [2] MONAI Project website ( https://monai.io/ )
-.. [3] W-Net: A Deep Model for Fully Unsupervised Image Segmentation, Xia and Kulis, 2018 ( https://arxiv.org/abs/1711.08506 )

napari_cellseg3d/__init__.py

@@ -1,2 +1,3 @@
 """napari-cellseg3d - napari plugin for cell segmentation."""
-__version__ = "0.2.0"
+
+__version__ = "0.2.1"

napari_cellseg3d/code_models/instance_segmentation.py

@@ -1,4 +1,5 @@
 """Instance segmentation methods for 3D images."""
+
 import abc
 from dataclasses import dataclass
 from functools import partial
@@ -128,14 +129,14 @@ def record_parameters(self):
         """Records all the parameters of the instance segmentation method from the current values of the widgets."""
         if len(self.sliders) > 0:
             for slider in self.sliders:
-                self.recorded_parameters[
-                    slider.label.text()
-                ] = slider.slider_value
+                self.recorded_parameters[slider.label.text()] = (
+                    slider.slider_value
+                )
         if len(self.counters) > 0:
             for counter in self.counters:
-                self.recorded_parameters[
-                    counter.label.text()
-                ] = counter.value()
+                self.recorded_parameters[counter.label.text()] = (
+                    counter.value()
+                )
 
     def run_method_from_params(self, image):
         """Runs the method on the image with the RECORDED parameters set in the widget.
@@ -327,10 +328,8 @@ def binary_connected(
     )
     semantic = np.squeeze(volume)
     foreground = np.where(semantic > thres, volume, 0)  # int(255 * thres)
-    segm = label(foreground)
-    segm = remove_small_objects(segm, thres_small)
-
-    return segm
+    seg = label(foreground)
+    return remove_small_objects(seg, thres_small)
 
 
 def binary_watershed(
@@ -419,15 +418,10 @@ def clear_small_objects(image, threshold, is_file_path=False):
     if is_file_path:
         image = imread(image)
 
-    # print(threshold)
-
     labeled = label(image)
 
     result = remove_small_objects(labeled, threshold)
 
-    # print(np.sum(labeled))
-    # print(np.sum(result))
-
     if np.sum(labeled) == np.sum(result):
         print("Warning : no objects were removed")
 
@@ -551,9 +545,9 @@ def __init__(self, widget_parent=None):
         )
 
         self.sliders[0].label.setText("Foreground probability threshold")
-        self.sliders[
-            0
-        ].tooltips = "Probability threshold for foreground object"
+        self.sliders[0].tooltips = (
+            "Probability threshold for foreground object"
+        )
         self.sliders[0].setValue(500)
 
         self.sliders[1].label.setText("Seed probability threshold")
@@ -652,9 +646,9 @@ def __init__(self, widget_parent=None):
         )
 
         self.sliders[0].label.setText("Foreground probability threshold")
-        self.sliders[
-            0
-        ].tooltips = "Probability threshold for foreground object"
+        self.sliders[0].tooltips = (
+            "Probability threshold for foreground object"
+        )
         self.sliders[0].setValue(800)
 
         self.counters[0].label.setText("Small objects removal")
@@ -715,18 +709,18 @@ def __init__(self, widget_parent=None):
             widget_parent=widget_parent,
         )
         self.counters[0].label.setText("Spot sigma")  # closeness
-        self.counters[
-            0
-        ].tooltips = "Determines how close detected objects can be"
+        self.counters[0].tooltips = (
+            "Determines how close detected objects can be"
+        )
         self.counters[0].setMaximum(100)
-        self.counters[0].setValue(2)
+        self.counters[0].setValue(0.65)
 
         self.counters[1].label.setText("Outline sigma")  # smoothness
-        self.counters[
-            1
-        ].tooltips = "Determines the smoothness of the segmentation"
+        self.counters[1].tooltips = (
+            "Determines the smoothness of the segmentation"
+        )
         self.counters[1].setMaximum(100)
-        self.counters[1].setValue(2)
+        self.counters[1].setValue(0.65)
 
         self.counters[2].label.setText("Small object removal")
         self.counters[2].tooltips = (

napari_cellseg3d/code_models/models/TEMPLATE_model.py

@@ -2,6 +2,7 @@
 
 Please note that custom model implementations are not fully supported out of the box yet, but might be in the future.
 """
+
 from abc import ABC, abstractmethod
 
 
@@ -11,6 +12,7 @@ class ModelTemplate_(ABC):
     weights_file = (
         "model_template.pth"  # specify the file name of the weights file only
     )
+    default_threshold = 0.5  # specify the default threshold for the model
 
     @abstractmethod
     def __init__(

napari_cellseg3d/code_models/models/model_SegResNet.py

@@ -1,11 +1,13 @@
 """SegResNet wrapper for napari_cellseg3d."""
+
 from monai.networks.nets import SegResNetVAE
 
 
 class SegResNet_(SegResNetVAE):
     """SegResNet_ wrapper for napari_cellseg3d."""
 
     weights_file = "SegResNet_latest.pth"
+    default_threshold = 0.3
 
     def __init__(
         self, input_img_size, out_channels=1, dropout_prob=0.3, **kwargs

napari_cellseg3d/code_models/models/model_SwinUNetR.py

@@ -1,4 +1,5 @@
 """SwinUNetR wrapper for napari_cellseg3d."""
+
 from monai.networks.nets import SwinUNETR
 
 from napari_cellseg3d.utils import LOGGER
@@ -10,6 +11,7 @@ class SwinUNETR_(SwinUNETR):
     """SwinUNETR wrapper for napari_cellseg3d."""
 
     weights_file = "SwinUNetR_latest.pth"
+    default_threshold = 0.4
 
     def __init__(
         self,

napari_cellseg3d/code_models/models/model_TRAILMAP_MS.py

@@ -1,4 +1,5 @@
 """TRAILMAP model, reimplemented in PyTorch."""
+
 from napari_cellseg3d.code_models.models.unet.model import UNet3D
 from napari_cellseg3d.utils import LOGGER as logger
 
@@ -7,6 +8,7 @@ class TRAILMAP_MS_(UNet3D):
     """TRAILMAP_MS wrapper for napari_cellseg3d."""
 
     weights_file = "TRAILMAP_MS_best_metric.pth"
+    default_threshold = 0.15
 
     # original model from Liqun Luo lab, transferred to pytorch and trained on mesoSPIM-acquired data (mostly TPH2 as of July 2022)