Installation fails on Windows

[lqgentner]

Hi, I am having problems installing shxarray on Windows. The issue is related to building the C++ files at src/builtin_backend with the Visual Studio build tools.

Steps to reproduce: Run pip install . or python setup.py install to trigger the build

Error message

Building wheels for collected packages: shxarray
Building wheel for shxarray (pyproject.toml) ... error
error: subprocess-exited-with-error

× Building wheel for shxarray (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [88 lines of output]
    running bdist_wheel
    running build
    running build_py
    copying src\shxarray\__init__.py -> build\lib.win-amd64-cpython-311\shxarray
    copying src\shxarray\core\admin.py -> build\lib.win-amd64-cpython-311\shxarray\core
    copying src\shxarray\core\cf.py -> build\lib.win-amd64-cpython-311\shxarray\core
    copying src\shxarray\core\logging.py -> build\lib.win-amd64-cpython-311\shxarray\core
    copying src\shxarray\core\shcomputebase.py -> build\lib.win-amd64-cpython-311\shxarray\core
    copying src\shxarray\core\shxarbase.py -> build\lib.win-amd64-cpython-311\shxarray\core
    copying src\shxarray\core\sh_indexing.py -> build\lib.win-amd64-cpython-311\shxarray\core
    copying src\shxarray\core\time.py -> build\lib.win-amd64-cpython-311\shxarray\core
    copying src\shxarray\core\xr_accessor.py -> build\lib.win-amd64-cpython-311\shxarray\core
    copying src\shxarray\core\__init__.py -> build\lib.win-amd64-cpython-311\shxarray\core
    copying src\shxarray\earth\constants.py -> build\lib.win-amd64-cpython-311\shxarray\earth
    copying src\shxarray\earth\snrei.py -> build\lib.win-amd64-cpython-311\shxarray\earth
    copying src\shxarray\earth\__init__.py -> build\lib.win-amd64-cpython-311\shxarray\earth
    copying src\shxarray\geoslurp\deg1n2.py -> build\lib.win-amd64-cpython-311\shxarray\geoslurp
    copying src\shxarray\geoslurp\gravity.py -> build\lib.win-amd64-cpython-311\shxarray\geoslurp
    copying src\shxarray\geoslurp\icgem.py -> build\lib.win-amd64-cpython-311\shxarray\geoslurp
    copying src\shxarray\geoslurp\icgemdset.py -> build\lib.win-amd64-cpython-311\shxarray\geoslurp
    copying src\shxarray\io\binv_legacy.py -> build\lib.win-amd64-cpython-311\shxarray\io
    copying src\shxarray\io\gsmv6.py -> build\lib.win-amd64-cpython-311\shxarray\io
    copying src\shxarray\io\icgem.py -> build\lib.win-amd64-cpython-311\shxarray\io
    copying src\shxarray\io\shascii.py -> build\lib.win-amd64-cpython-311\shxarray\io
    copying src\shxarray\io\shiobackend.py -> build\lib.win-amd64-cpython-311\shxarray\io
    copying src\shxarray\io\__init__.py -> build\lib.win-amd64-cpython-311\shxarray\io
    copying src\shxarray\kernels\anisokernel.py -> build\lib.win-amd64-cpython-311\shxarray\kernels
    copying src\shxarray\kernels\axial.py -> build\lib.win-amd64-cpython-311\shxarray\kernels
    copying src\shxarray\kernels\ddk.py -> build\lib.win-amd64-cpython-311\shxarray\kernels
    copying src\shxarray\kernels\factory.py -> build\lib.win-amd64-cpython-311\shxarray\kernels
    copying src\shxarray\kernels\gauss.py -> build\lib.win-amd64-cpython-311\shxarray\kernels
    copying src\shxarray\kernels\gravfunctionals.py -> build\lib.win-amd64-cpython-311\shxarray\kernels
    copying src\shxarray\kernels\isokernelbase.py -> build\lib.win-amd64-cpython-311\shxarray\kernels
    copying src\shxarray\kernels\__init__.py -> build\lib.win-amd64-cpython-311\shxarray\kernels
    running egg_info
    writing src\shxarray.egg-info\PKG-INFO
    writing dependency_links to src\shxarray.egg-info\dependency_links.txt
    writing entry points to src\shxarray.egg-info\entry_points.txt
    writing requirements to src\shxarray.egg-info\requires.txt
    writing top-level names to src\shxarray.egg-info\top_level.txt
    reading manifest template 'MANIFEST.in'
    adding license file 'LICENSE'
    writing manifest file 'src\shxarray.egg-info\SOURCES.txt'
    copying src\builtin_backend\.gitignore -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\Gaunt.hpp -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\Helpers.hpp -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\Legendre.hpp -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\Legendre_nm.hpp -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\Wigner3j.hpp -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\Ynm.hpp -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\analysis.pyx -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\gaunt.pxd -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\gaunt.pyx -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\legendre.pxd -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\legendre.pyx -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\shlib.cpp -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\shlib.pyx -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\synthesis.pyx -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\wigner3j.pxd -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\wigner3j.pyx -> build\lib.win-amd64-cpython-311\builtin_backend
    copying src\builtin_backend\ynm.pyx -> build\lib.win-amd64-cpython-311\builtin_backend
    running build_ext
    building 'shxarray.shlib' extension
    "C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -IC:\Users\luisg\AppData\Local\Temp\pip-build-env-irtlzc2o\overlay\Lib\site-packages\numpy\core\include -I. -IC:\Users\luisg\AppData\Local\Programs\Python\Python311\include -IC:\Users\luisg\AppData\Local\Programs\Python\Python311\Include "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Tools\MSVC\14.41.34120\include" "-IC:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" /EHsc /Tpsrc/builtin_backend/shlib.cpp /Fobuild\temp.win-amd64-cpython-311\Release\src/builtin_backend/shlib.obj /openmp
    shlib.cpp
    C:\Users\luisg\Repositories\shxarray\src\builtin_backend\Wigner3j.hpp(201): error C2143: syntax error: missing ';' before 'Wigner3j<ftype>::downward_scales'
    C:\Users\luisg\Repositories\shxarray\src\builtin_backend\Wigner3j.hpp(201): error C3858: 'Wigner3j<ftype>::scales': cannot be redeclared in current scope
    C:\Users\luisg\Repositories\shxarray\src\builtin_backend\Wigner3j.hpp(51): note: see declaration of 'Wigner3j<ftype>::scales'
    C:\Users\luisg\Repositories\shxarray\src\builtin_backend\Wigner3j.hpp(201): error C2062: type 'int' unexpected
    C:\Users\luisg\Repositories\shxarray\src\builtin_backend\Wigner3j.hpp(201): error C2143: syntax error: missing ';' before '{'
    C:\Users\luisg\Repositories\shxarray\src\builtin_backend\Wigner3j.hpp(201): error C2447: '{': missing function header (old-style formal list?)
    C:\Users\luisg\Repositories\shxarray\src\builtin_backend\Wigner3j.hpp(229): error C2143: syntax error: missing ';' before 'Wigner3j<ftype>::upward_scales'
    C:\Users\luisg\Repositories\shxarray\src\builtin_backend\Wigner3j.hpp(229): error C3858: 'Wigner3j<ftype>::scales': cannot be redeclared in current scope
    C:\Users\luisg\Repositories\shxarray\src\builtin_backend\Wigner3j.hpp(51): note: see declaration of 'Wigner3j<ftype>::scales'
    C:\Users\luisg\Repositories\shxarray\src\builtin_backend\Wigner3j.hpp(229): error C2062: type 'int' unexpected
    C:\Users\luisg\Repositories\shxarray\src\builtin_backend\Wigner3j.hpp(229): error C2143: syntax error: missing ';' before '{'
    C:\Users\luisg\Repositories\shxarray\src\builtin_backend\Wigner3j.hpp(229): error C2447: '{': missing function header (old-style formal list?)
    src/builtin_backend/shlib.cpp(25863): warning C4244: '=': conversion from 'Py_ssize_t' to 'int', possible loss of data     
    src/builtin_backend/shlib.cpp(27891): warning C4244: '=': conversion from 'Py_ssize_t' to 'int', possible loss of data     
    src/builtin_backend/shlib.cpp(27901): warning C4244: '=': conversion from 'Py_ssize_t' to 'int', possible loss of data     
    src/builtin_backend/shlib.cpp(27917): warning C4244: '=': conversion from 'Py_ssize_t' to 'int', possible loss of data     
    src/builtin_backend/shlib.cpp(28049): warning C4244: '=': conversion from 'Py_ssize_t' to 'int', possible loss of data     
    src/builtin_backend/shlib.cpp(30638): warning C4244: '=': conversion from 'Py_ssize_t' to 'int', possible loss of data     
    src/builtin_backend/shlib.cpp(30648): warning C4244: '=': conversion from 'Py_ssize_t' to 'int', possible loss of data     
    src/builtin_backend/shlib.cpp(30756): warning C4244: '=': conversion from 'Py_ssize_t' to 'int', possible loss of data     
    src/builtin_backend/shlib.cpp(41876): warning C4551: function call missing argument list
    src/builtin_backend/shlib.cpp(43601): warning C4551: function call missing argument list
    error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\2022\\BuildTools\\VC\\Tools\\MSVC\\14.41.34120\\bin\\HostX86\\x64\\cl.exe' failed with exit code 2
    [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for shxarray
Failed to build shxarray
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (shxarray)

Version numbers:

• Python: 3.11.9
• Visual Studio Build Tools 2022: 17.11.5
• pip: 24.3.1
• setuptools: 65.5.0

Unfortunatley my knowledge of C++ is limited, so I cannot diagnose this issue any further.

I currently use shxarray via the Windows Subsystem for Linux (WSL), but would appreciate a fix so that Windows users can run shxarray natively on Windows in the future.

Thanks in advance!

[strawpants]

Thanks for reporting this! I've not yet tried building the package on windows myself, but I'm planning to build a binary windows package using github actions in the future. Hopefully this will (partly) address this issue too.

The problem probably lies in that cython generated the shlib.cpp file which may not be compatible with visual studio. Regenerating the shlib.cpp file during a fresh build would require setting the environment variable USE_CYTHON=1, before pip install . , so that is something which could be tried out.