Is there a build model_creation example?

[davidackerman]

Previously, there was a model_creation example notebook. It is also referenced in the Bioimagezoo developers guide as being here.

If not, is there another such example of how to create a model from torch?

[FynnBe]

Hi, thanks for reporting the broken link!
Sorry for the inconvenience.

I updated the example links in the bioimageio.core docs here and created bioimage-io/bioimage.io#439 to start updating our central docs.

The model_creation notebook was revised and republished as model_usage.ipynb (as now listed under the examples)

That notebook starts off with loading an existing resource from the zoo to get you started faster.
I once presented how to create a model completely from scratch: notebook slides, but this notebook does not (yet) sufficient comments to really serve as a standalone example (maybe you still find it a useful reference). I plan to make a proper example out of it eventually (#447)

[mattaq31]

Hi, sorry to butt in here, but I'm also attempting to create a Bioimage.io model from a pytorch model and I honestly had no idea where to start before I saw the presentation you've linked.

So just to confirm if I'm reading your presentation correctly - do I need 1) the model source code to load in the model architecture (via ArchitectureFromFileDescr) and 2) the model weights in Torchscript format? I also saw that you saved your weights to .pt format but it seems like it was just for demonstration purposes.

Thanks in advance!

[FynnBe]

Hi @mattaq31 , thank you for your question!
This is great feedback helping to improve our documentation and examples!
under weights there are a few weight formats to choose from: https://bioimage-io.github.io/core-bioimage-io-python/bioimageio/spec/model/v0_5.html#WeightsDescr

We encourage contributors to provide as many as possible, however a single weights format is the minimum. When possible its great to convert pytorch state dicht weights (which require executing the source code for the architecture as I'm sure you're familiar with) to torchscript (which does not require the original source code and specific Python environment making it far easier to deploy).
To continue training or adapt a model in other ways the pytorch state dict weights are easier to work with, so they should still be included even if torchscript weights are available.
We accept scripted and traced torchscript models.

[mattaq31]

Hi @FynnBe thanks for getting back to me so quickly during the holiday period! Also thanks for clarifying the different options available when choosing which weights to upload.

Eventually, I managed to assemble a bioimage.io model for my torchscript weights and proceeded to run the validation script which ran with no errors. For more info, I am preparing a U-Net segmentation model which was trained within PyTorch that accepts single channel images and outputs dual channel images of the same size.

However, I have a couple more questions (apologies in advance for this large dump of text!):

• The validation script is directly testing the model with my provided test input/output arrays, which is fine. However, my ultimate goal is to release my segmentation model for use with DeepImageJ. Are the torchscript weights I provided enough? Is there some way to test whether the model will work within DeepImageJ?
• I have a certain set of pre/post processing requirements:
  • Input images can be either 8 or 16-bit. The 8-bit ones should be normalized by dividing by 255, while the 16-bit ones should be scaled to 0-1 using percentile normalization. I can embed this normalization into my torchscript module, but I won't be able to recognize whether an image is 8- or 16-bit if the input is a tensor. Does a BioImage.IO preprocessing module exist which can take care of these requirements?
  • All input images to the model should have only one channel. RGB inputs need to be converted to grayscale. Does BioImage.IO have a module for this? Alternatively, I can embed the channel squishing within the torchscript module.
  • I also need to pad my images to be divisible by 32. From what I can see, it seems this pre/post processing is not available in BioImage.IO. Is this correct? If so, would you recommend I embed the padding within the torchscript module itself?
• In general, are models in the BioImage.IO zoo setup to work 'out-of-the-box' when an input image is provided? The above requirements are non-trivial and when I previously implemented my torchscript model in QuPath, I preferred to keep the model simple and deal with all the pre/post processing myself within QuPath's java framework. Would you recommend I do the same with DeepImageJ?

In general, the documentation available for preparing a new BioImage.IO model from scratch seems to be very terse, I had to guess many of the arguments to the ModelDescr function based on your presentation example and the brief tutorial here. For example, a question I had was: 'which preprocessing operations are available and what exactly do they do?' and all I could find was a brief mention here with no details.

Thanks again for your help and apologies once again for the large wall of text but I wasn't sure where to reach out otherwise!

[mattaq31]

Hi again @FynnBe, to quickly update, I've managed to validate the model works in DeepImageJ (although that also has its own share of bugs and issues). I'm about to upload the model to BioImage.IO but I am unable to login to the uploader form system (https://bioimageio-uploader.netlify.app/). When I try going ahead and uploading anyway, I'm getting various errors such as:

TypeError: Cannot convert undefined or null to object

Would you be able to let me know if there are any workarounds for this issue?

Thanks!