Numpy 2.0

[da-wad]

Numpy 2.0 is coming: numpy/numpy#24300

There will need to be a zfpy release built for this, as 1.0.0 wheels will (very likely) not be compatible with numpy 2.x

Is anyone looking into this yet?

[lindstro]

@da-wad Thanks for the heads up. We've not yet looked at the impact of Numpy 2.0. Are you aware of anything in particular that will require changes to zfpy?

[da-wad]

Nothing more than what is in the summary text in that issue.

But: "the NumPy C ABI will change in 2.0, so any compiled extension modules that rely on NumPy are likely to break, they need to be recompiled." may mean that compiling new wheels is all we need to do.

However, I think this highlights the need for some distance between zfpy and zfp versioning...

[jakirkham]

JFYI 3 weeks ago, NumPy 2.0.0rc1 was tagged. Packages were built for conda & wheels: numpy/numpy#24300 (comment)

Would add NumPy has put together a migration guide. More details are in the release notes

If zfpy make use of NumPy's C API (and produces wheels that use it), having a release of zfpy with wheels built against NumPy 2.0.0rc1 would be helpful to ensure NumPy 1 & 2 compatible wheels (as wheels built against NumPy 1 won't be compatible with NumPy 2). More details in this NumPy 2 ABI doc

Also as NumPy is tracking ecosystem support for NumPy 2.0, it would be helpful to share zfpy's current support status (with any plans) in this issue: numpy/numpy#26191

[lindstro]

@jakirkham Thanks for the heads up. We are currently looking for someone to help out with maintaining zfpy and building wheels as we're short on staff and expertise in this area. Any contributions would be welcome and appreciated.

[jakirkham]

Would recommend using whatever build script is used currently and try NumPy 2 instead (maybe drop oldest-supported-numpy if that is in use)

Given the light usage of NumPy in zfpy, it is possible that is all that is needed

[seberg]

Had a bit of a look around, and I think @jakirkham is right. Assuming the current build setup works (it doesn't look very "formal"): all that should be needed is replacing oldest-support-numpy with numpy>=2.0.0rc1 and you are done.
(Of course testing with NumPy 2 may show other incompatibilities.)

[jaimergp]

I added #231, let's see what happens.

[jaimergp]

Update from #231 (comment). I see two warnings:

/opt/hostedtoolcache/Python/3.12.3/x64/lib/python3.12/site-packages/numpy/_core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: #warning "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]
   17 | #warning "Using deprecated NumPy API, disable it with " \
      |  ^~~~~~~

I don't think those are new (due to 2.0) though. Tests are passing, so I think it's ok?

[seberg]

Yes they are ancient, but we didn't enforce them now, there was just little point.

[frobnitzem]

This is breaking with numpy 2.0.0. Looks like a change to numpy's array struct.

% uname -po
x86_64 GNU/Linux

% pip install zfpy==1.0.0
Collecting zfpy==1.0.0
  Using cached zfpy-1.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (648 bytes)
Using cached zfpy-1.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (729 kB)
Installing collected packages: zfpy
Successfully installed zfpy-1.0.0

% python3
Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import zfpy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "zfpy.pyx", line 1, in init zfpy
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
>>> import numpy as np
>>> np.__version__
'2.0.0'

If I build using (after updating setup.py / pyproject.toml)

cd build
cmake -DCMAKE_INSTALL_PREFIX=$VIRTUAL_ENV ..
make -j install
LDFLAGS=-Wl,-rpath,$VIRTUAL_ENV/lib' pip install .

the problem disappears.

[jakirkham]

Yes this is a known issue. The builds are not compatible with NumPy 2. Would stick with NumPy 1 for now if you need zfpy