ND2 Incorrect Series Count and T Size

[MXGHarryLiu]

Hi BioFormats developers and community members,

I am using BioFormat 5.9.2 in MATLAB r2018b. When I read files with the bfmatlab library, for certain files in the dataset, the interpreted series size and time size got mixed up.
Specifically, in my multi-channel time series dataset which contains 34 files, each file should have one FOV or series. However, when read using reader.getSeriesCount() 6 files return a series count of greater than 1, and correspondingly when using reader.getSizeT() the results for the 6 files are 1 instead of 7.
NIS Viewer reports the time size (T) correctly in all cases, and reader.getDimensionOrder() would give correct result: XYCZT.
I've uploaded some sample files to the OME feedback site. The first two files gave expected values, and ysp010pH4mVsLib2V1_1-1.nd2 is one of the files that gave incorrect results.
If you need more test files, please let me know in the reply.

Thanks for your support,
Harry

[dgault]

Hi @MXGHarryLiu, thank you for uploading the sample files. I was able to test them with Bio-Formats 5.9.2 and the file ysp010pH4mVsLib2V1_1-1.nd2 was not reproducing the problem. I am seeing the following details for it:

Series count = 1
Series #0 :
	Image count = 28
	RGB = false (1) 
	Interleaved = false
	Indexed = true (false color)
	Width = 2048
	Height = 2044
	SizeZ = 1
	SizeT = 7
	SizeC = 4
	Tile size = 2048 x 2044
	Thumbnail size = 128 x 127
	Endianness = intel (little)
	Dimension order = XYCZT (certain)
	Pixel type = uint16
	Valid bits per pixel = 16
	Metadata complete = true
	Thumbnail series = false

Are you definitely using version 5.9.2 also? Also does setting the option nativend2.chunkmap to false (see https://docs.openmicroscopy.org/bio-formats/5.9.2/formats/options.html) solve the problem? This has been a workaround for a similar issue in the past.

[MXGHarryLiu]

Hi @dgault,
Thanks for the prompt reply.
I reinstalled the bfmatlab library and made sure that I was using the latest version 5.9.2.
However, all three files I've uploaded were the correct files... Sorry that I mixed up the files since they have similar names, so I uploaded the missing file that generated the issue: ysp010pH4mVsLib2V2_1-1.nd2. The issue number is 27289.
In addition, I tried your method:

path = 'ysp010pH4mVsLib2V2_1-1.nd2';
l = loci.formats.in.DynamicMetadataOptions;
l.setBoolean('nativend2.chunkmap', java.lang.Boolean(false))
reader = bfGetReader();
reader.setMetadataOptions(l);
reader.setId(path);
reader.getSeriesCount()
reader.getSizeT()

The last two lines still give me mixed series and time count.
Let me know if I implemented the fix correctly.
Thanks a lot for your help!
Harry

[melissalinkert]

Thank you for uploading a new file, @MXGHarryLiu. I see the same behavior, with Bio-Formats reading 7 series and NIS Elements Viewer showing 7 timepoints. This has been added to a known issue with timepoint/series handling in ND2 files:

https://trello.com/c/LEjj3O3A/281-nd2-timepoints-incorrectly-detected-as-series

Your code looks correct, but I can confirm that changing the nativend2.chunkmap setting will not help with this case.

[kevinjohncutler]

I am also having this issue... just spent a few hours trying to figure out why. I am on the latest version of bfmatlab.

[dgault]

Hi @kevinjohncutler, is this still an issue for you or was it only the 1 bad file mentioned in #3448? If it is still a problem would you be able to upload a sample file for testing to https://www.openmicroscopy.org/qa2/qa/upload/

[kevinjohncutler]

@dgault that is for checking in. It seems like it was indeed just that one file giving me issues. I’ve checked a few others and they seem to be fine.

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/bio-formats-not-opening-all-time-points-of-nd2-file/75025/2

[dgault]

This issue has been reported by another user in https://forum.image.sc/t/unable-to-open-nikon-nd2-with-bio-formats/38904/5

A sample file is available at https://zenodo.org/record/7439387#.Y9fA9OzP10t which replicates the issue with Bio-Formats 6.11.1. None of the ND2 reader options impact the behaviour.

In the case of this sample file there are expected to be 44 timepoints, when read with Bio-Formats the timepoints are split into series, but even then only 20 series are displayed and the remaining timepoints are dropped.

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/bio-formats-not-opening-all-time-points-of-nd2-file/75025/8

[ElpadoCan]

I have a similar issue. I thought I was on the latest version of Bioformats, but I was actually using 6.10.1.

However, the issue is fixed with Bioformats 6.12.0.

[dgault]

Another report of a similar issue has been recived on forum thread https://forum.image.sc/t/issue-with-nd2-time-series-in-bio-formats/92810

2 samples have been provided which are available at inbox/imagesc-92810, one a working file showing a single series and 30 timepoints and the second file showing the issue which results in 30 series each with a single timepoint. I was able to reproduce the failure using the sample file and Bio-Formats 7.2.0, switching the chunkmap option did not have any affect on the outcome.

From debugging it looks as though the broken sample file originally parses the metadata correctly as a single series with sizeT of 30. However for the imageOffsets it also finds 30 unique Z positions. As the number of unique Z positions is equal to sizeT, the reader resets the sizeT to 1 and treats each image as a series. This occurs at https://github.com/ome/bioformats/blob/develop/components/formats-gpl/src/loci/formats/in/ND2Reader.java#L1596

For the working sample file it seems that the reader parses 28 unique Z positions and as this differs from the 30 timepoints it does not attempt ti readjust.

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/issue-with-nd2-time-series-in-bio-formats/92810/2

[imagesc-bot]

This issue has been mentioned on Image.sc Forum. There might be relevant details there:

https://forum.image.sc/t/issue-with-nd2-time-series-in-bio-formats/92810/6