NDPI: Prevent Null Pointer Exception

[dgault]

This issue was raised on the mailing list and a sample file has been provided: http://lists.openmicroscopy.org.uk/pipermail/ome-users/2018-October/007233.html

The images failed to open due to a NullPointerException on line 176, with the markers variable being null.

To test:

• Ensure builds and tests remain green
• Using the sample file provided on the mailing list test that the file is able to be open and read correctly

[sbesson]

Closing/reopening to include #3251

[sbesson]

@dgault is the sample file available somewhere? Should we configure it alongside our QA repository (and potentially make it public with the agreement of the author)?

[dgault]

I have a copy of the sample file as it no longer appears to be available on the previous link. I will get it configured for the repo tests at least.

[sbesson]

Retested using the sample file uploaded as part of http://lists.openmicroscopy.org.uk/pipermail/ome-users/2019-February/007381.html.

The reader initialization via the initial setId call is no longer failing with a NPE. In an OMERO server with this PR included, smaller resolutions are correctly read but there is an issue while accessing the large resolutions and the thumbnail:

Looking at the server logs, it looks like the change is in the JPEGTurboService in the absence of markers.

java.lang.RuntimeException: java.io.IOException: Restart interval and markers invalid
	at omeis.providers.re.data.PlaneFactory.createPlane(PlaneFactory.java:223) ~[rendering.jar:na]
...
Caused by: java.io.IOException: Restart interval and markers invalid
	at loci.formats.services.JPEGTurboServiceImpl.initialize(JPEGTurboServiceImpl.java:222) ~[formats-bsd.jar:6.0.0-SNAPSHOT]
	at loci.formats.in.NDPIReader.openBytes(NDPIReader.java:167) ~[formats-gpl.jar:6.0.0-SNAPSHOT]
	at loci.formats.ImageReader.openBytes(ImageReader.java:460) ~[formats-api.jar:6.0.0-SNAPSHOT]
	at loci.formats.ChannelFiller.openBytes(ChannelFiller.java:156) ~[formats-bsd.jar:6.0.0-SNAPSHOT]
	at loci.formats.ChannelFiller.openBytes(ChannelFiller.java:148) ~[formats-bsd.jar:6.0.0-SNAPSHOT]
	at loci.formats.ChannelSeparator.openBytes(ChannelSeparator.java:200) ~[formats-bsd.jar:6.0.0-SNAPSHOT]
	at loci.formats.ReaderWrapper.openBytes(ReaderWrapper.java:348) ~[formats-api.jar:6.0.0-SNAPSHOT]
	at ome.io.bioformats.BfPixelsWrapper.getTile(BfPixelsWrapper.java:350) ~[romio.jar:na]
	at ome.io.bioformats.BfPixelBuffer.getTile(BfPixelBuffer.java:499) ~[romio.jar:na]
	at omeis.providers.re.data.PlaneFactory.createPlane(PlaneFactory.java:193) ~[rendering.jar:na]
	... 58 common frames omitted

[dgault]

A similar report has been received in https://forum.image.sc/t/a-couple-of-problems-with-ndpi-files/23194/6 and a sample file has been provided.

I will modify this PR to fix both issues and include for 6.0.1

[dgault]

After a lot of effort to rewrite the JPEGTurboServiceImpl it turns out that you can just ignore the restart markers being empty and the things are much better. The images should display but there is still an issue with the thumbnails however.

[dgault]

The thumbnail issue seems to be a problem with the lowest resolution image, Im not sure if its a bug or if the image is as intended.

[sbesson]

At the level of the non-regression tests, the changes to the JPEGTurboServiceImpl required some configuration changes for two sample JPEG files.

Tested the import of these two jpeg files into a server. Given the size of the planes, this leads to the generation of pyramids server-side:

• without this PR included, the pixel data look like the issue described in https://www.openmicroscopy.org/community/viewtopic.php?f=13&t=8393&p=18693#p18693

• with this PR included

Also tested the import of a few NDPI files including the ones uploaded as part of the original bug report - http://lists.openmicroscopy.org.uk/pipermail/ome-users/2018-October/007233.html

Without this PR

With this PR

• 06-T05322_1A_HEc_FTU.ndpi the issue primarily affects the low levels of resolution of the WSI. Without this PR, the loading fails leading to no thumbnails. With this PR, the plane looks corrupted and generate a thumbnail which seems to indicate
• SN120-18-7_HE.ndpi still fails to import with or without this PR with a NPE at import time when calling openBytes
• the TurboJPEG change seems to introduce a regression for two jpeg images

Thanks for giving this issue another try to try and get it as part of the 6.2.0 release. Given the review, it looks like more investigation is required so I would not hold the release further.

[melissalinkert]

I propose to close this in favor of #3505, since all of the files I could find are now configured in https://github.com/openmicroscopy/data_repo_config/pull/442