Import of UNET-3D model trained with everywheredl4mic not working

[mvelean1]

Hello,
I'm trying to train a model using the UNET-3D notebook in everydl4mic, and to export it in deepimageJ.
I managed to make the model work quite good in everywheredl4mic, but fail to use it in imageJ...
I obtain this error :

[DEBUG] Image name: C1-003 VHIP VGLUT2 GAD65 0,5x50x Ventral 2-Airyscan Processing-04...P VGLUT2 GAD65 0,5x50x Ventral 2-Airyscan Processing-04.czi #1 kept stack_crop.tif [DEBUG] Start pre-processing [DEBUG] Run Macro pre-processing [DEBUG] End pre-processing [DEBUG] Running inference on the tensors [DEBUG] (Inference) Patch 1/256 io.bioimage.modelrunner.exceptions.RunModelException: Error executing the Tensorflow 2 model in a separate process. The process was not terminated correctly. at io.bioimage.modelrunner.tensorflow.v2.api030.Tensorflow2Interface.runInterprocessing(Tensorflow2Interface.java:360) at io.bioimage.modelrunner.tensorflow.v2.api030.Tensorflow2Interface.run(Tensorflow2Interface.java:301) at io.bioimage.modelrunner.model.Model.runModel(Model.java:510) at deepimagej.RunnerDL.call(RunnerDL.java:398) at deepimagej.RunnerDL.call(RunnerDL.java:1) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) [DEBUG] Close all the inputs except the input image [DEBUG] End execution

and this error in the terminal

2024-10-23 21:27:51.229409: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:283] SavedModel load for tags { serve }; Status: success: OK. Took 314539 microseconds. Exception in thread "main" org.tensorflow.exceptions.TensorFlowException: Converting GraphDef to Graph has failed with an error: 'Op type not registered 'DisableCopyOnRead' in binary running on maxime-HP-Z4-G4-Workstation. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) tf.contrib.resamplershould be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.' The binary trying to import the GraphDef was built when GraphDef version was 898. The GraphDef was produced by a binary built when GraphDef version was 1395. The difference between these versions is larger than TensorFlow's forward compatibility guarantee, and might be the root cause for failing to import the GraphDef.

Is it a problem of compatibility with tensorflow ?

Thanks a lot in advance !

[carlosuc3m]

Hello @mvelean1 !
Right now we are in the process of a major change of deepImageJ, thus until i upload the new version to the update site, this error will happen!
I will try to have a prototype up by tomorrow!
Ill let you know!

[carlosuc3m]

Hello @mvelean1 I have uploaded a preliminary version of the plugin to the update site. Feel free to use it!
You just need to update deepimagej from the update site!

It might not work for Pytorch in macos but i am working to fix it!

Feel free to provide any feedback.
Regards,
Carlos

[mvelean1]

Hello @carlosuc3m ,

Thanks a lot for your answer ! I tried the update, but it doesn't work for me, the install model button leads to "Plugin or class not found: "DeepImageJ_InstallModel"(java.lang.ClassNotFoundException: DeepimageJ_InstallModel)". The use model button opens a new window but many error messages in the console.
I am using Ubuntu 22.04

[carlosuc3m]

HEllo @mvelean1
I forgot to remove the other plugins! No it is just deepimagej run.
If you have created the model with zerocost and you have the zip, put it in the models folder inside Fiji and then unzip it.
The model should appear once you open deepImageJ run.
REgards,
Carlos

[mvelean1]

I unfortunately get the error

"Exception in thread "Thread-7" java.lang.NoClassDefFoundError: io/bioimage/modelrunner/bioimageio/description/ModelDescriptorFactory
at deepimagej.gui.Gui.lambda$loadLocalModels$0(Gui.java:118)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.ClassNotFoundException: io.bioimage.modelrunner.bioimageio.description.ModelDescriptorFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
... 2 more
"
I tried to remove and reinstall the plugin, also to try it on a fresh FiJi, I get the same error each time. And I get this window :

Thank you !

[carlosuc3m]

Sorry @mvelean1 some dependency was missing!
I have just updated the update site again. If you can please try it now!

[mvelean1]

Good morning,
I tried again after update, now there's the enclosed error message when I start the plugin (i've put it in a text file since it's a bit long).
bug_deepimagej.txt
Local models are not detected, and when I click on the right or left arrows

I get

Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
at deepimagej.gui.SearchBar.lambda$performSearch$0(SearchBar.java:116)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
at deepimagej.gui.SearchBar.performSearch(SearchBar.java:118)
at deepimagej.gui.Gui.searchModels(Gui.java:363)
at deepimagej.gui.Gui.lambda$initSearchBar$2(Gui.java:140)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
at java.awt.Component.processMouseEvent(Component.java:6539)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
at java.awt.Component.processEvent(Component.java:6304)
at java.awt.Container.processEvent(Container.java:2239)
at java.awt.Component.dispatchEventImpl(Component.java:4889)
at java.awt.Container.dispatchEventImpl(Container.java:2297)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
at java.awt.Container.dispatchEventImpl(Container.java:2283)
at java.awt.Window.dispatchEventImpl(Window.java:2746)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
at java.awt.EventQueue$4.run(EventQueue.java:733)
at java.awt.EventQueue$4.run(EventQueue.java:731)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)

Thank you !

[carlosuc3m]

could you please show the rdf.yaml file of the model. I am wondering why it is not recognised.

Sorry for all teh inconvenience!

[mvelean1]

rdf.zip

Please find enclosed my rdf.yaml file. Now with a fresh FiJi and the model unziped in the models folder, deepimageJ recognize it, but I get the following error

Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
at deepimagej.gui.SearchBar.lambda$performSearch$0(SearchBar.java:116)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
at deepimagej.gui.SearchBar.performSearch(SearchBar.java:118)
at deepimagej.gui.Gui.searchModels(Gui.java:363)
at deepimagej.gui.Gui.lambda$initSearchBar$2(Gui.java:140)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
at java.awt.Component.processMouseEvent(Component.java:6539)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
at java.awt.Component.processEvent(Component.java:6304)
at java.awt.Container.processEvent(Container.java:2239)
at java.awt.Component.dispatchEventImpl(Component.java:4889)
at java.awt.Container.dispatchEventImpl(Container.java:2297)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
at java.awt.Container.dispatchEventImpl(Container.java:2283)
at java.awt.Window.dispatchEventImpl(Window.java:2746)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
at java.awt.EventQueue$4.run(EventQueue.java:733)
at java.awt.EventQueue$4.run(EventQueue.java:731)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Exception in thread "AWT-EventQueue-0" java.lang.NullPointerException
at deepimagej.gui.SearchBar.lambda$performSearch$0(SearchBar.java:116)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:174)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
at deepimagej.gui.SearchBar.performSearch(SearchBar.java:118)
at deepimagej.gui.Gui.searchModels(Gui.java:363)
at deepimagej.gui.Gui.lambda$initSearchBar$2(Gui.java:140)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:2022)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2348)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:402)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:259)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:252)
at java.awt.Component.processMouseEvent(Component.java:6539)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3324)
at java.awt.Component.processEvent(Component.java:6304)
at java.awt.Container.processEvent(Container.java:2239)
at java.awt.Component.dispatchEventImpl(Component.java:4889)
at java.awt.Container.dispatchEventImpl(Container.java:2297)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4904)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4535)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4476)
at java.awt.Container.dispatchEventImpl(Container.java:2283)
at java.awt.Window.dispatchEventImpl(Window.java:2746)
at java.awt.Component.dispatchEvent(Component.java:4711)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:760)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:84)
at java.awt.EventQueue$4.run(EventQueue.java:733)
at java.awt.EventQueue$4.run(EventQueue.java:731)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:730)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Exception in thread "AWT-EventQueue-0" java.lang.ArithmeticException: / by zero
at deepimagej.gui.ModelSelectionPanel.getWrappedIndex(ModelSelectionPanel.java:187)
at deepimagej.gui.ModelSelectionPanel.redrawModelCards(ModelSelectionPanel.java:151)
at deepimagej.gui.ModelSelectionPanel.setModels(ModelSelectionPanel.java:145)
at deepimagej.gui.Gui.setModelsInGui(Gui.java:348)
at deepimagej.gui.Gui.lambda$clickedLocal$25(Gui.java:489)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Exception in thread "AWT-EventQueue-0" java.lang.ArithmeticException: / by zero
at deepimagej.gui.ModelSelectionPanel.getWrappedIndex(ModelSelectionPanel.java:187)
at deepimagej.gui.ModelSelectionPanel.redrawModelCards(ModelSelectionPanel.java:151)
at deepimagej.gui.ModelSelectionPanel.setModels(ModelSelectionPanel.java:145)
at deepimagej.gui.Gui.setModelsInGui(Gui.java:348)
at deepimagej.gui.Gui.lambda$clickedLocal$26(Gui.java:493)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
[SERVICE-0] {"task":"47536f1f-ba06-4a19-9e26-278472497b7c","requestType":"EXECUTE","inputs":{"modelFolder":"/home/maxime/Downloads/fiji-linux64 (1)/Fiji.app/models/VGLUT2_Lukas"},"script":"loadModel"}
[SERVICE-0] {"responseType":"LAUNCH","task":"47536f1f-ba06-4a19-9e26-278472497b7c"}
[WORKER-0] Exception in thread "Appose-47536f1f-ba06-4a19-9e26-278472497b7c" java.lang.NoClassDefFoundError: org/yaml/snakeyaml/Yaml
[WORKER-0] at io.bioimage.modelrunner.utils.YAMLUtils.load(YAMLUtils.java:59)
[WORKER-0] at io.bioimage.modelrunner.bioimageio.description.ModelDescriptorFactory.readFromLocalFile(ModelDescriptorFactory.java:66)
[WORKER-0] at io.bioimage.modelrunner.tensorflow.v2.api030.Tensorflow2Interface.checkModelUnzipped(Tensorflow2Interface.java:219)
[WORKER-0] at io.bioimage.modelrunner.tensorflow.v2.api030.Tensorflow2Interface.loadModel(Tensorflow2Interface.java:177)
[WORKER-0] at io.bioimage.modelrunner.tensorflow.v2.api030.JavaWorker.executeScript(JavaWorker.java:90)
[WORKER-0] at io.bioimage.modelrunner.tensorflow.v2.api030.JavaWorker.lambda$start$0(JavaWorker.java:104)
[WORKER-0] at java.lang.Thread.run(Thread.java:750)
[WORKER-0] Caused by: java.lang.ClassNotFoundException: org.yaml.snakeyaml.Yaml
[WORKER-0] at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
[WORKER-0] at java.lang.ClassLoader.loadClass(ClassLoader.java:419)
[WORKER-0] at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
[WORKER-0] at java.lang.ClassLoader.loadClass(ClassLoader.java:352)
[WORKER-0] ... 7 more

[carlosuc3m]

HEllo @mvelean1
Is it working for you now?
Regards,
Carlos