Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reader writer fixes #60

Merged
merged 12 commits into from
Feb 17, 2022
Merged

Reader writer fixes #60

merged 12 commits into from
Feb 17, 2022

Conversation

camFoltz
Copy link
Contributor

@camFoltz camFoltz commented Jan 7, 2022

This PR fixes a lot of the issues that @ieivanov brought up on the issue page.

Specifically:

Writer:

  • Fixes how users define contrast limits. Can be a list of tuples with (start, end, min, max) or (start, end) where min/max will be inferred from the datatype
  • Changes default array name to arr_0 instead of array
  • Changes specifiyng [0,N] as a range for writing data and instead implements pythonic slice(0, N)
  • changed default colormap to 'FFFFFF' and opened up an issue in the napari-ome-zarr plugin
  • ome-metadata sets on the first channel as active (for viewing in napari) rest are by default inactive

Reader:

  • automatic filetype parsing if the data_type is not explicitly provided by the user
  • zarrfile reader can handle the old array name and also arr_0
  • closes TiffFile after opening to avoid a warning and to properly clean up the file.

Other:

  • updated one np.float in background_estimator to np.float64 in order to avoid numpy deprecation

@camFoltz
Copy link
Contributor Author

camFoltz commented Feb 2, 2022

@ieivanov do you think you can take a quick look here so we can merge? I have implemented and tested all of your suggestions (thanks!)

@lihaoyeh
Copy link
Contributor

I can use this branch to load data well. When writing, I faced some issues in the specification of clims.

Here is the code block that I use to write the uPTI data.

writer = WaveorderWriter(output_path, hcs=False, hcs_meta=None, verbose=True)
writer.create_zarr_root('uPTI_physical_2.zarr')
position = 0
chan_names_phys = ['Phase3D', 'Retardance3D', 'Orientation', 'Inclination', 'Optic_sign']
phys_data_array = np.transpose(np.array([phase_PT, np.abs(retardance_pr_PT[0]), azimuth[0], theta[0], p_mat_map]),(0,3,1,2))[np.newaxis,...]
data_shape_phys = phys_data_array.shape
chunk_size_phys = (1,1,1)+phys_data_array.shape[3:]
dtype = 'float32'
writer.init_array(position, data_shape_phys, chunk_size_phys, chan_names_phys, dtype,  position_name='Stitched_physical', overwrite=True)
writer.write(phys_data_array, p=position)

I did not encounter an issue using the previous version of the writer. When running this code with this branch, I encounter the following error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-19-5c76b46745f2> in <module>
     11 # clims = [(-0.15,0.15),(0,0.012),(0,np.pi),(0,np.pi),(0,1)]
     12 dtype = 'float32'
---> 13 writer.init_array(position, data_shape_phys, chunk_size_phys, chan_names_phys, dtype,  position_name='Stitched_physical', overwrite=True)
     14 writer.write(phys_data_array, p=position)

~/project/Polscope/waveorder/waveorder/io/writer.py in init_array(self, position, data_shape, chunk_size, chan_names, dtype, clims, position_name, overwrite)
    145 
    146         self.sub_writer.create_position(position, pos_name)
--> 147         self.sub_writer.init_array(data_shape, chunk_size, dtype, chan_names, clims, overwrite)
    148 
    149     def write(self, data, p, t=None, c=None, z=None):

~/project/Polscope/waveorder/waveorder/io/writer_structures.py in init_array(self, data_shape, chunk_size, dtype, chan_names, clims, overwrite)
     61             raise TypeError('dtype must be instance of either np.dtype or str')
     62 
---> 63         self.set_channel_attributes(chan_names, clims)
     64         self.current_pos_group.zeros('arr_0', shape=data_shape, chunks=chunk_size, dtype=dtype,
     65                            compressor=self.__compressor, overwrite=overwrite)

~/project/Polscope/waveorder/waveorder/io/writer_structures.py in set_channel_attributes(self, chan_names, clims)
    337         for i in range(len(chan_names)):
    338 
--> 339             if len(clims[i]) == 2:
    340                 if 'float' in self.dtype.name:
    341                     clim = (clims[i][0], clims[i][1], -1000, 1000)

TypeError: 'NoneType' object is not subscriptable

It seems to mean that we need to specify clims in this version of the writer. I think that the previous version of the writer is convenient in that you don't have to specify contrast limits for individual channels (it auto-scales to min and max). Then, I supply the clims with the following code piece:
clims = [(np.min(array),np.max(array)) for array in phys_data_array[0]]
I then got another error:
TypeError: Object of type float32 is not JSON serializable
This is because the type of the number in the clims tuple is numpy.float32 and that is not serializable. I then converted those numbers to float. It passes.

With these tests, there are a few questions to be answered:

  1. Do we want the writer to identify clims from images if it is not specified? If no, the current implementation is fine.
  2. We can maybe add some checks on the type of numbers in clims or convert the number into float before doing further writing processes.

I do like the part of this version that uses "FFFFFF" as the default colormap. It visualizes images with a brighter contrast to start with.

@camFoltz
Copy link
Contributor Author

Thanks for catching this @lihaoyeh. We want their to be default clims if the user doesn't specify any. It seems like there is a slight bug and I will fix it.

@camFoltz
Copy link
Contributor Author

@lihaoyeh I also added the catch for the numpy datatypes that aren't serializable and now the clims will automatically be converted to regular floats

Copy link
Contributor

@lihaoyeh lihaoyeh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tested this new version and it works for my test case. I have resolved the conflicts and merged the master into this branch. Please go ahead and merge it to the master.

@camFoltz camFoltz merged commit 41ab852 into master Feb 17, 2022
@camFoltz camFoltz deleted the reader_writer_fixes branch February 23, 2022 17:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants