Conversion to NWB problem

[JohnStout]

Running into an issue with converting NWB files.

I am working with Mac OSX Ventura for what it is worth. Working in the terminal and in the caiman environment.

Followed the instructions to download and unpack the Najafi 2018 dataset, which was done successfully. I'm running into issues with converting to nwb. My pynwb version is 2.3.2.

When running

% python3 scripts/convert_to_nwb.py

I run into the following error:

0%| | 0/135 [00:00<?, ?it/s]/Users/js0403/anaconda3/envs/caiman/lib/python3.10/site-packages/numpy/core/numeric.py:407: RuntimeWarning: invalid value encountered in cast
multiarray.copyto(res, fill_value, casting='unsafe')
/Users/js0403/najafi-2018-nwb/scripts/convert_to_nwb.py:156: RuntimeWarning: invalid value encountered in cast
neuron_type[bad_roi_mask] = moremat['inhibitRois_pix']
18%|███████▍ | 24/135 [03:45<22:53, 12.38s/it]Error adding roi_response_series: stimOffAl
ErrorMsg: 'stimOffAl'

19%|███████▊ | 25/135 [04:00<24:18, 13.26s/it]Error adding roi_response_series: stimOffAl
ErrorMsg: 'stimOffAl'

19%|████████ | 26/135 [04:08<21:24, 11.79s/it]Error adding roi_response_series: stimOffAl
ErrorMsg: 'stimOffAl'

20%|████████▍ | 27/135 [04:25<23:52, 13.26s/it]Error adding roi_response_series: stimOffAl
ErrorMsg: 'stimOffAl'

Each file is processed until it runs into 'stimOffAI' after which, the progress terminates.

[victorgutemberg]

hello @ttngu207 and @dimitri-yatsenko,

I'm running into another issue with the conversion to NWB. I think this might be related to different versions.
Could you please export your pip/conda requirements on the environment you're using to run the code?

Great study by the way! 😄

 32%|███▏      | 43/135 [41:24<1:42:31, 66.87s/it]Error adding roi_response_series: stimOffAl
                ErrorMsg: 'stimOffAl'

 33%|███▎      | 44/135 [42:16<1:34:34, 62.36s/it]Error adding roi_response_series: stimOffAl
                ErrorMsg: 'stimOffAl'

 53%|█████▎    | 72/135 [1:13:51<1:04:37, 61.55s/it]
Traceback (most recent call last):
  File "/home/workstation/Projects/najafi-2018-nwb/scripts/convert_to_nwb.py", line 41, in <module>
    moremat, postmat = (sio.loadmat(file_pair[x], struct_as_record=False, squeeze_me=True)
  File "/home/workstation/Projects/najafi-2018-nwb/scripts/convert_to_nwb.py", line 41, in <genexpr>
    moremat, postmat = (sio.loadmat(file_pair[x], struct_as_record=False, squeeze_me=True)
  File "/home/workstation/anaconda3/envs/torch/lib/python3.10/site-packages/scipy/io/matlab/_mio.py", line 227, in loadmat
    matfile_dict = MR.get_variables(variable_names)
  File "/home/workstation/anaconda3/envs/torch/lib/python3.10/site-packages/scipy/io/matlab/_mio5.py", line 333, in get_variables
    res = self.read_var_array(hdr, process)
  File "/home/workstation/anaconda3/envs/torch/lib/python3.10/site-packages/scipy/io/matlab/_mio5.py", line 291, in read_var_array
    return self._matrix_reader.array_from_header(header, process)
  File "_mio5_utils.pyx", line 665, in scipy.io.matlab._mio5_utils.VarReader5.array_from_header
  File "_mio5_utils.pyx", line 712, in scipy.io.matlab._mio5_utils.VarReader5.array_from_header
  File "_mio5_utils.pyx", line 965, in scipy.io.matlab._mio5_utils.VarReader5.read_struct
  File "_mio5_utils.pyx", line 663, in scipy.io.matlab._mio5_utils.VarReader5.read_mi_matrix
  File "_mio5_utils.pyx", line 694, in scipy.io.matlab._mio5_utils.VarReader5.array_from_header
  File "_mio5_utils.pyx", line 768, in scipy.io.matlab._mio5_utils.VarReader5.read_real_complex
  File "_mio5_utils.pyx", line 445, in scipy.io.matlab._mio5_utils.VarReader5.read_numeric
  File "_mio5_utils.pyx", line 350, in scipy.io.matlab._mio5_utils.VarReader5.read_element
  File "_streams.pyx", line 171, in scipy.io.matlab._streams.ZlibInputStream.read_string
  File "_streams.pyx", line 147, in scipy.io.matlab._streams.ZlibInputStream.read_into
  File "_streams.pyx", line 134, in scipy.io.matlab._streams.ZlibInputStream._fill_buffer
zlib.error: Error -3 while decompressing data: incorrect data check