Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Timedelta64 data cannot be round-tripped to netCDF files without a warning #10099

Open
shoyer opened this issue Mar 5, 2025 · 3 comments · May be fixed by #10101
Open

Timedelta64 data cannot be round-tripped to netCDF files without a warning #10099

shoyer opened this issue Mar 5, 2025 · 3 comments · May be fixed by #10101

Comments

@shoyer
Copy link
Member

shoyer commented Mar 5, 2025

What is your issue?

We added a future warning about not decoding time units to timedelta64 in #9966 (cc @spencerkclark, @kmuehlbauer).

Unfortunately, this warning is raised by default when reading timedelta64 serialized data to disk. This makes it much harder to use this dtype (which is quite useful for storing the "lead time" dimension in weather forecasts), and means that if we ever do finalize this deprecation warning it will break a lot of users.

I would love to see special handling of timedelta64 data, similar to what I described here: #1621 (comment). In particular, we could write a dtype='timedelta64' attribute (possibly also with a specified precision) when writing a dataset to disk, which could be interpreted as np.timedelta64 data when reading the data with Xarray. This would allow us to at least ensure that datasets with timedelta64 data that are written to Zarr/netCDF now will always be able to be read faithfullly in the future.

To reproduce:

import xarray
import numpy as np

deltas = np.array([1, 2, 3], dtype='timedelta64[D]').astype('timedelta64[ns]')
ds = xarray.Dataset({'lead_time': deltas})
xarray.open_dataset(ds.to_netcdf())

This issues:
FutureWarning: In a future version of xarray decode_timedelta will default to False rather than None. To silence this warning, set decode_timedelta to True, False, or a 'CFTimedeltaCoder' instance.

@shoyer shoyer added the needs triage Issue that has not been reviewed by xarray team member label Mar 5, 2025
@slevang
Copy link
Contributor

slevang commented Mar 5, 2025

+1 this warning has also exploded across my code base and would be a major breaking change if the deprecation cycle is finished.

@spencerkclark spencerkclark linked a pull request Mar 6, 2025 that will close this issue
3 tasks
@spencerkclark spencerkclark removed the needs triage Issue that has not been reviewed by xarray team member label Mar 6, 2025
@spencerkclark
Copy link
Member

Sorry for adding this warning somewhat hastily—I'm not surprised it is coming up frequently. To the extent that anyone is relying on the existing way of decoding / encoding np.timedelta64 values, e.g. in old files, some amount of breakage feels unavoidable in addressing #1621, but I fully agree it would be nice to retain this functionality seamlessly for new files. See #10101 for an initial attempt at this.

@kmuehlbauer
Copy link
Contributor

That particular FutureWarning was suggested by me, to inform interested parties about that change which was agreed upon in #1621. But, we should have thought a bit more on a better solution, taking @shoyer's comments into account. That was clearly overlooked, sorry. Thanks @spencerkclark for already tackling this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants