Merge pull request #125 from scijava/scijava-ops-flimlib/init

Port FlimJ-Ops to SciJava Ops

docs/ops/doc/ScriptingInFiji.md

@@ -12,10 +12,9 @@ To run Ops, scripts require an `OpEnvironment`. The easiest way to obtain an `Op
 
 ## Obtaining inputs
 
-Scripts using SciJava Ops obtain inputs like any other SciJava script, and the lines below will provide us with an `Img` input parameter and an `Img` output parameter, as well as a `ThreadService` which we will use later.
+Scripts using SciJava Ops obtain inputs like any other SciJava script, and the lines below will provide us with an `Img` input parameter and an `Img` output parameter.
 
 ```text
-#@ ThreadService ts
 #@ Img imgInput
 #@output Img out
 ```
@@ -35,10 +34,11 @@ The below script can be pasted into the Script Editor. **Ensure that the Script
 
 ```text
 #@ OpEnvironment ops
-#@ ThreadService ts
 #@ Img imgInput
 #@output Img out
 
 // Call some Ops!
 out = ops.op("filter.gauss").arity2().input(imgInput, new Double(3.0)).apply()
 ```
+
+Scripting in Fiji is a convenient gateway to accessing SciJava Ops. To see more, check out some examples, such as [image deconvolution](examples/deconvolution.rst) or [FLIM analysis](examples/example_flim_analysis.rst)! 

docs/ops/doc/examples/example_flim_analysis.rst

@@ -0,0 +1,164 @@
+=============
+FLIM Analysis
+=============
+
+In this example we will use SciJava Ops within Fiji to perform `FLIM`_ analysis, which is used in many situations including photosensitizer detection and `FRET`_ measurement.
+
+.. image:: https://media.imagej.net/scijava-ops/1.0.0/flim_example_input.gif
+    :width: 49%
+.. image:: https://media.imagej.net/scijava-ops/1.0.0/flim_example_pseudocolored.png
+    :width: 49%
+
+We use a sample of `FluoCells™ Prepared Slide #1`_, imaged by `Jenu Chacko <https://loci.wisc.edu/staff/chacko-jenu/>`_ using Openscan-LSM and SPwith multiphoton excitation and a 40x WI lens.
+
+  FluoCells™ Prepared Slide #1 contains bovine pulmonary artery endothelial cells (BPAEC). MitoTracker™ Red CMXRos was used to stain the mitochondria in the live cells, with accumulation dependent upon membrane potential. Following fixation and permeabilization, F-actin was stained with Alexa Fluor™ 488 phalloidin, and the nuclei were counterstained with the blue-fluorescent DNA stain DAPI.
+
+The sample data can be downloaded `here <https://media.imagej.net/scijava-ops/1.0.0/flim_example_data.sdt>`_ and can be loaded into Fiji with `Bio-Formats`_ using ``File → Open``. When presented with the ``Bio-Formats Import Options`` screen, it may be helpful to select ``Metadata viewing → Display metadata`` to determine values necessary for analysis. Then, select ``OK``. The data may take a minute to load.
+
+Within the script, the `Levenberg-Marquardt algorithm <https://en.wikipedia.org/wiki/Levenberg%E2%80%93Marquardt_algorithm>`_ fitting Op of SciJava Ops FLIM is used to fit the data.
+
+Basic analysis
+---------------------
+
+Script execution requires a number of parameters, which may be useful for adapting this script to other datasets. For this dataset, we use the following values:
+
++--------------------------------------+-------+
+| Parameter                            | Value |
++======================================+=======+
+| Time Base                            | 12.5  |
++--------------------------------------+-------+
+| Time Bins                            | 256   |
++--------------------------------------+-------+
+| Lifetime Axis                        | 2     |
++--------------------------------------+-------+
+| Intensity Threshold                  | 18    |
++--------------------------------------+-------+
+| Bin Kernel Radius                    | 1     |
++--------------------------------------+-------+
+
+The script above will display the fit results, as well as a *pseudocolored* output image. To visualize , it should be contrasted using ImageJ's B&C plugin (``Ctrl + Shift + C``). Using that plugin, the minimum and maximum can be set by selecting the ``Set`` option, and providing ``0`` as the minimum and ``3`` as the maximum.
+
+The results are shown in the panels below, and are described from left to right:
+
+* The first initial fluorescence parameter A\ :subscript:`1`
+
+* The first fluorescence lifetime τ\ :subscript:`1`.
+
+* The pseudocolored result, an HSV image where
+
+  * Hue is a function of τ\ :subscript:`1`, where the function is a LUT
+
+  * Value is a function of A\ :subscript:`1`
+
+.. image:: https://media.imagej.net/scijava-ops/1.0.0/flim_example_a1.png
+    :width: 32%
+
+.. image:: https://media.imagej.net/scijava-ops/1.0.0/flim_example_tau1.png
+    :width: 32%
+
+.. image:: https://media.imagej.net/scijava-ops/1.0.0/flim_example_pseudocolored.png
+    :width: 32%
+
+The pseudocolored result shows a clear separation of fluorophores, which could be segmented and further processed.
+
+Subsampling Within ROIs
+-----------------------
+
+Curve fitting can be an intensive process, requiring significant resources to process larger datasets. For this reason, there can be significant benefit in restricting computation to Regions of Interest (ROIs), and SciJava Ops FLIM allows ROIs to restrict computation for all fitting Ops.
+
+The provided script allows users to specify ROIs by drawing selections using the ImageJ UI. These selections are converted to ImgLib2 ``RealMask`` objects, which are then optionally passed to the Op.
+
+In the panels below, we show script execution with computation restricted to the area around a single cell. In the top left panel, we can see the original dataset, annotated with an elliptical selection using the ImageJ UI. In the top right, bottom left, and bottom right panels, we see the A\ :subscript:`1` component, τ\ :subscript:`1` component, and pseudocolored results, respectively, all limited to the area within the selection.
+
+.. image:: https://media.imagej.net/scijava-ops/1.0.0/flim_example_input_roi.png
+    :width: 49%
+
+.. image:: https://media.imagej.net/scijava-ops/1.0.0/flim_example_a1_roi.png
+    :width: 49%
+
+.. image:: https://media.imagej.net/scijava-ops/1.0.0/flim_example_tau1_roi.png
+    :width: 49%
+
+.. image:: https://media.imagej.net/scijava-ops/1.0.0/flim_example_pseudocolored_roi.png
+    :width: 49%
+
+
+.. tabs::
+
+    .. code-tab:: scijava-groovy
+
+        #@ OpEnvironment ops
+        #@ ROIService roiService
+        #@ Img input
+        #@ Float (description="The total time (ns) (timeBase in metadata)", label = "Time Base") timeBase
+        #@ Integer (description="The number of time bins (timeBins in metadata)", label = "Time Bins") timeBins
+        #@ Integer (description="The index of the lifetime axis (from metadata)", label = "Lifetime Axis", value=2) lifetimeAxis
+        #@ Float (description="The minimal pixel intensity (across all time bins) threshold for fitting", label = "Intensity Threshold") iThresh
+        #@ Integer (description="The radius of the binning kernel", label = "Bin Kernel Radius", value=0, min=0) kernelRad
+        #@OUTPUT Img A1
+        #@OUTPUT Img Tau1
+        #@OUTPUT Img pseudocolored
+
+        import net.imglib2.roi.Regions
+        import java.lang.System
+
+        // Utility function to collapse all ROIs into a single mask for FLIM fitting
+        def getMask() {
+            // No ROIs
+            if (!roiService.hasROIs(input)) {
+                return null
+            }
+            // 1+ ROIs
+            rois = roiService.getROIs(input)
+            mask = rois.children()remove(0).data()
+            for(roi: rois.children()) {
+                mask = mask.or(roi.data())
+            }
+            return mask;
+        }
+
+        import net.imglib2.type.numeric.real.DoubleType
+        def getPercentile(img, mask, percentile) {
+            if (mask != null) {
+                img = Regions.sampleWithRealMask(mask, img)
+            }
+            return ops.op("stats.percentile").arity2()
+                .input(img, percentile)
+                .outType(DoubleType.class)
+                .apply()
+                .getRealFloat()
+        }
+
+        start = System.currentTimeMillis()
+
+        // The FitParams contain a set of reasonable defaults for FLIM curve fitting
+        import org.scijava.ops.flim.FitParams
+        param = new FitParams()
+        param.transMap = input
+        param.ltAxis = lifetimeAxis
+        param.iThresh = iThresh
+        // xInc is the difference (ns) between two bins
+        param.xInc = timeBase / timeBins
+
+        // Fit curves
+        kernel = ops.op("create.kernelSum").arity1().input(1 + 2 * kernelRad).apply()
+        lma = ops.op("flim.fitLMA").arity3().input(param, getMask(), kernel).apply()
+
+        // The fit results paramMap is a XYC image, with result attributes along the Channel axis
+        fittedImg = lma.paramMap
+        // For LMA, we have Z, A1, and Tau1 as the three attributes
+        A1 = ops.op("transform.hyperSliceView").arity3().input(fittedImg, lifetimeAxis, 1).apply()
+        Tau1 = ops.op("transform.hyperSliceView").arity3().input(fittedImg, lifetimeAxis, 2).apply()
+
+        // Finally, generate a pseudocolored result
+        cMin = getPercentile(Tau1, mask, 5.0)
+        cMax = getPercentile(Tau1, mask, 95.0)
+        pseudocolored = ops.op("flim.pseudocolor").arity3().input(lma, cMin, cMax).apply()
+
+        end = System.currentTimeMillis()
+        println("Finished fitting in " + (end - start) + " milliseconds")
+
+.. _`Bio-Formats` : https://www.openmicroscopy.org/bio-formats/
+.. _`FLIM` : https://en.wikipedia.org/wiki/Fluorescence-lifetime_imaging_microscopy
+.. _`FluoCells™ Prepared Slide #1` : https://www.thermofisher.com/order/catalog/product/F36924
+.. _`FRET` : https://en.wikipedia.org/wiki/F%C3%B6rster_resonance_energy_transfer

docs/ops/doc/index.rst

@@ -23,6 +23,7 @@ The combination of these libraries allows declarative image analysis workflows,
    CallingOps
    SearchingForOps
    Benchmarks
+   ScriptingInFiji
 
 .. toctree::
    :maxdepth: 2
@@ -31,6 +32,7 @@ The combination of these libraries allows declarative image analysis workflows,
    examples/deconvolution
    examples/example_gaussian_subtraction
    examples/opencv_denoise
+   examples/example_flim_analysis
 
 .. toctree::
    :maxdepth: 2

pom.xml

@@ -54,6 +54,7 @@
 		<module>scijava-meta</module>
 		<module>scijava-ops-api</module>
 		<module>scijava-ops-engine</module>
+		<module>scijava-ops-flim</module>
 		<module>scijava-ops-image</module>
 		<module>scijava-ops-legacy</module>
 		<module>scijava-ops-indexer</module>

scijava-ops-engine/src/main/java/module-info.java

@@ -532,6 +532,7 @@
 		org.scijava.ops.engine.adapt.functional.InplacesToFunctions.Inplace16_14ToFunction16,
 		org.scijava.ops.engine.adapt.functional.InplacesToFunctions.Inplace16_15ToFunction16,
 		org.scijava.ops.engine.adapt.functional.InplacesToFunctions.Inplace16_16ToFunction16,
+		org.scijava.ops.engine.conversionLoss.impl.IdentityLossReporter,
 		org.scijava.ops.engine.eval.DefaultEval,
 		org.scijava.ops.engine.stats.Mean.MeanFunction;
 

scijava-ops-engine/src/main/java/org/scijava/ops/engine/matcher/convert/ConvertedOpInfo.java

@@ -461,8 +461,9 @@ private double calculatePriority(OpEnvironment env) {
 		List<Type> originalInputs = info.inputTypes();
 		List<Type> inputs = inputTypes();
 		for (int i = 0; i < inputs.size(); i++) {
-			penalty += determineLoss(env, Nil.of(inputs.get(i)), Nil.of(originalInputs
-				.get(i)));
+			var from = inputs.get(i);
+			var to = Types.mapVarToTypes(originalInputs.get(i), typeVarAssigns);
+			penalty += determineLoss(env, Nil.of(from), Nil.of(to));
 		}
 
 		Type opOutput = info.outputType();

scijava-ops-engine/src/main/resources/META-INF/services/org.scijava.ops.spi.Op

@@ -234,5 +234,6 @@ org.scijava.ops.engine.adapt.functional.InplacesToFunctions$Inplace16_13ToFuncti
 org.scijava.ops.engine.adapt.functional.InplacesToFunctions$Inplace16_14ToFunction16
 org.scijava.ops.engine.adapt.functional.InplacesToFunctions$Inplace16_15ToFunction16
 org.scijava.ops.engine.adapt.functional.InplacesToFunctions$Inplace16_16ToFunction16
+org.scijava.ops.engine.conversionLoss.impl.IdentityLossReporter
 org.scijava.ops.engine.eval.DefaultEval
 org.scijava.ops.engine.stats.Mean$MeanFunction

scijava-ops-engine/src/test/java/org/scijava/ops/engine/impl/ServiceLoaderDiscoveryIntegrationTest.java

@@ -47,14 +47,14 @@ public class ServiceLoaderDiscoveryIntegrationTest {
 	public void opDiscoveryRegressionIT() {
 		final Discoverer d = Discoverer.using(ServiceLoader::load);
 		final List<Op> discoveries = d.discover(Op.class);
-		Assertions.assertEquals(236, discoveries.size());
+		Assertions.assertEquals(237, discoveries.size());
 
 		@SuppressWarnings("unused")
 		final OpInfoGenerator g = new OpClassOpInfoGenerator();
 		final List<OpInfo> infos = discoveries.stream() //
 			.flatMap(c -> g.generateInfosFrom(c).stream()) //
 			.collect(Collectors.toList());
-		Assertions.assertEquals(236, infos.size());
+		Assertions.assertEquals(237, infos.size());
 	}
 
 	@Test

scijava-ops-engine/templates/main/java/module-info.vm

@@ -128,6 +128,7 @@ module org.scijava.ops.engine {
 		org.scijava.ops.engine.adapt.functional.InplacesToFunctions.Inplace${arity}_${a}ToFunction${arity},
 #end
 #end
+		org.scijava.ops.engine.conversionLoss.impl.IdentityLossReporter,
 		org.scijava.ops.engine.eval.DefaultEval,
 		org.scijava.ops.engine.stats.Mean.MeanFunction;
 

scijava-ops-flim/LICENSE.txt

@@ -0,0 +1,24 @@
+Copyright (c) 2024, SciJava developers.
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without modification,
+are permitted provided that the following conditions are met:
+
+1. Redistributions of source code must retain the above copyright notice, this
+   list of conditions and the following disclaimer.
+
+2. Redistributions in binary form must reproduce the above copyright notice,
+   this list of conditions and the following disclaimer in the documentation
+   and/or other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
+ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
+LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+POSSIBILITY OF SUCH DAMAGE.

scijava-ops-flim/README.md

@@ -0,0 +1,70 @@
+# SciJava Ops FLIM: A fluorescence lifetime analysis library for SciJava Ops.
+
+SciJava Ops FLIM is a collection of FLIM analysis ops based on [FLIMLib](https://github.com/flimlib/flimlib). It extends the single-transient fitting functions in FLIMLib to dataset-level fitting ops. Currently supported fitting ops include: RLD, MLA, Global, Phasor, and single-component Bayesian.
+
+Besides curve fitting, SciJava Ops FLIM also provides a variety of pre-processing options such as pixel binning, intensity thresholding, ROI masking as well as post-processing utility ops for e.g. calculating τ<sub>m</sub> (mean lifetime), A<sub>i</sub>% (fractional contribution) and pseudocoloring the result with LUT.
+
+# Example usage
+Open [test2.sdt](test_files/test2.sdt) in [Fiji](https://fiji.github.io/). Execute in [Script Editor](http://imagej.github.io/Using_the_Script_Editor) as Groovy:
+
+```groovy
+#@ UIService ui
+#@ OpEnvironment op
+#@ ImgPlus img
+
+// set up parameters
+import org.scijava.ops.flim.FitParams
+
+param = new FitParams()
+param.transMap = img; // input 3-dimensional (x, y, t) dataset
+param.xInc= 0.040     // time difference between bins (ns)
+param.ltAxis = 2      // time bins lay along axis #2
+
+// op call
+fittedImg = op.unary("flim.fitLMA").input(param).apply().paramMap
+
+// display each parameter
+zImg = op.ternary("transform.hyperSliceView").input(fittedImg, param.ltAxis, 0L).apply()
+AImg = op.ternary("transform.hyperSliceView").input(fittedImg, param.ltAxis, 1L).apply()
+tauImg = op.ternary("transform.hyperSliceView").input(fittedImg, param.ltAxis, 2L).apply()
+
+ui.show("z", zImg)
+ui.show("A", AImg)
+ui.show("tau", tauImg)
+```
+
+After running this script, the output shown below can be seen by first running the script, and then brightness and contrast (Ctrl + Shift + C, select "Set" and bound the contrast to each image's corresponding range below).
+
+(z in [-1, 1], A in [0, 4], tau in[0, 3]):
+
+![example output](images/example%20z.png)![example output](images/example%20A.png)![example output](images/example%20tau.png)
+
+See more examples in [Demo.ipynb](notebooks/Demo.ipynb) and [groovy.md](groovy.md).
+
+# Using from a Java project
+
+To depend on SciJava Ops FLIM, copy the following to your `pom.xml`:
+
+```xml
+  <properties>
+    <scijava-ops-flim.version>0-SNAPSHOT</scijava-ops-flim.version>
+  </properties>
+
+  <dependencies>
+    <dependency>
+      <groupId>org.scijava</groupId>
+      <artifactId>scijava-ops-flim</artifactId>
+      <version>${scijava-ops-flim.version}</version>
+    </dependency>
+  </dependencies>
+```
+
+# See also
+
+- [FLIMLib](https://github.com/flimlib/flimlib): Curve fitting library for FLIM
+    - [Debug tutorial](https://github.com/flimlib/flimlib/wiki/Debugging)
+- [FLIMJ Ops](https://github.com/flimlib/flimj-ops): ImageJ Ops for accessing FLIM
+
+# Citation
+
+Coming soon...

scijava-ops-flim/environment.yml

@@ -0,0 +1,8 @@
+name: scijava-ops-flim
+channels:
+  - conda-forge
+dependencies:
+  - beakerx
+  - jupyter_contrib_nbextensions
+  - numpy
+  - openjdk=11