Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't serialize optional fps=None attribute when writing ds to file #420

Open
luiztauffer opened this issue Feb 17, 2025 · 2 comments
Open
Labels
bug Something isn't working

Comments

@luiztauffer
Copy link

Describe the bug
When loading a dataset and not providing a positive float value for fps, it gets automatically assigned a None value.
This is problematic because None is non-serializable when writing to NetCDF file.
Setting load_poses.from_file(..., fps="None") doesn't work, because movement enforces fps to be a positive float.

To Reproduce

from movement.io import load_poses
from movement.sample_data import fetch_dataset_paths


paths_dict = fetch_dataset_paths(filename="DLC_single-mouse_EPM.predictions.csv", with_video=False)

ds = load_poses.from_file(
    file_path=paths_dict["poses"],
    source_software="DeepLabCut",
    #fps=...,  # Explicitly omitting fps here
)
print(ds.fps)

ds.to_netcdf(path="file.nc", engine="scipy")

raises:

TypeError: Invalid value for attr 'fps': None. For serialization to netCDF files, its value must be of one of the following types: str, Number, ndarray, number, list, tuple, bytes

Expected behaviour
Either that:

  • fps is truly optional and doesn't get added to the dataset
  • fps is optionally a string for the dataset attributes validation. and gets assigned the value of "None" in the case above
  • fps is required when loading a dataset from file
@niksirbi
Copy link
Member

Thanks for flagging this @luiztauffer, I had also encountered this while experimenting with saving to netCDF (see this comment) and I agree this is annoying.

There are two kinds of attributes that cause problems with serialization during netCDF saving:

  • None values, as you've described
  • Pathlib.Path objects (When loading data from file, we store the file path as a an attribute)

For the None problem, I lean towards your first suggestion—do not add fps do the dataset if it's not supplied (i.e. it's absence indicates None).

To solve the pathlib.Path problem, we should just convert the paths to string before adding them to the attributes.

Do these workarounds sound sensible to you?

@luiztauffer
Copy link
Author

@niksirbi yes these suggestions sound like a good solution to me!
Btw, I don't think I've encountered the problem with Paths yet, but it does sound reasonable to keep them as strings to avoid this type of error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: 🤔 Triage
Development

No branches or pull requests

2 participants