Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cannot install cryptography on Fedora 41+, CentOS Stream 10 #11690

Closed
CtrlZmaster opened this issue Oct 2, 2024 · 3 comments · Fixed by #11714
Closed

Cannot install cryptography on Fedora 41+, CentOS Stream 10 #11690

CtrlZmaster opened this issue Oct 2, 2024 · 3 comments · Fixed by #11714

Comments

@CtrlZmaster
Copy link
Contributor

Basically the same issue as #11331, result of OpenSSL engine deprecation. Fedora 41 is in beta now and CentOS Stream 10 is starting to be added to various CI tools, so I think this issue is getting more impactful.

Pip will build from source after specifying --no-binary or if wheel for an architecture is not available. Upstream cryptography is not installable by pip for s390x and ppc64le architectures on these distributions at all. There was #11328 which was closed without merging even though it looked like it worked and there were no objections. Could it be merged, please?

Fedora 41

  • Versions of Python, cryptography, cffi, pip, and setuptools
    you're using
    • Python 3.13.0rc2
    • Pip 24.2
    • Cryptography 43.0.1
    • No other packages installed for reproducing, cffi 1.17.1 and setuptools 75.1.0 were downloaded by pip
  • How you installed cryptography
    • pip install --verbose --no-binary :all: cryptography
  • Clear steps for reproducing your bug
    • Install a Fedora 41 Beta Server VM
    • dnf install gcc libffi-devel openssl-devel rust cargo python3-devel python3-pip pkg-config
    • pip install --verbose --no-binary :all: cryptography
[root@f41-a ~]# pip install --verbose --no-binary :all: cryptography
Using pip 24.2 from /usr/lib/python3.13/site-packages/pip (python 3.13)
Collecting cryptography
  Using cached cryptography-43.0.1.tar.gz (686 kB)
  Running command pip subprocess to install build dependencies
  Using pip 24.2 from /usr/lib/python3.13/site-packages/pip (python 3.13)
  Collecting maturin<2,>=1
    Using cached maturin-1.7.4-cp313-cp313-linux_x86_64.whl
  Collecting cffi>=1.12
    Using cached cffi-1.17.1-cp313-cp313-linux_x86_64.whl
  Collecting setuptools!=74.0.0,!=74.1.0,!=74.1.1
    Using cached setuptools-75.1.0-py3-none-any.whl
  Collecting pycparser (from cffi>=1.12)
    Using cached pycparser-2.22-py3-none-any.whl
  Installing collected packages: setuptools, pycparser, maturin, cffi
  Successfully installed cffi-1.17.1 maturin-1.7.4 pycparser-2.22 setuptools-75.1.0
  WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Getting requirements to build wheel ... done
  Running command Preparing metadata (pyproject.toml)
  📦 Including license file "/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/LICENSE"
  📦 Including license file "/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/LICENSE.APACHE"
  📦 Including license file "/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/LICENSE.BSD"
  🍹 Building a mixed python/rust project
  🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.7
  🐍 Not using a specific python interpreter
  📡 Using build options features, locked from pyproject.toml
  cryptography-43.0.1.dist-info
  Checking for Rust toolchain....
  Running `maturin pep517 write-dist-info --metadata-directory /tmp/pip-modern-metadata-nw3quv22 --interpreter /usr/bin/python3`
  Preparing metadata (pyproject.toml) ... done
Collecting cffi>=1.12 (from cryptography)
  Using cached cffi-1.17.1-cp313-cp313-linux_x86_64.whl
Collecting pycparser (from cffi>=1.12->cryptography)
  Using cached pycparser-2.22-py3-none-any.whl
Building wheels for collected packages: cryptography
  Running command Building wheel for cryptography (pyproject.toml)
  Running `maturin pep517 build-wheel -i /usr/bin/python3 --compatibility off`
  📦 Including license file "/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/LICENSE"
  📦 Including license file "/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/LICENSE.APACHE"
  📦 Including license file "/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/LICENSE.BSD"
  🍹 Building a mixed python/rust project
  🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.7
  🐍 Not using a specific python interpreter
  📡 Using build options features, locked from pyproject.toml
     Compiling target-lexicon v0.12.15
     Compiling proc-macro2 v1.0.86
     Compiling unicode-ident v1.0.12
     Compiling cc v1.1.6
     Compiling pyo3-build-config v0.22.2
     Compiling quote v1.0.36
     Compiling syn v2.0.71
     Compiling pkg-config v0.3.30
     Compiling vcpkg v0.2.15
     Compiling once_cell v1.19.0
     Compiling openssl-sys v0.9.103
     Compiling libc v0.2.155
     Compiling cfg-if v1.0.0
     Compiling pyo3-macros-backend v0.22.2
     Compiling pyo3-ffi v0.22.2
     Compiling autocfg v1.3.0
     Compiling memoffset v0.9.1
     Compiling openssl v0.10.66
     Compiling foreign-types-shared v0.1.1
     Compiling heck v0.5.0
     Compiling foreign-types v0.3.2
     Compiling pyo3 v0.22.2
     Compiling bitflags v2.6.0
     Compiling unindent v0.2.3
     Compiling indoc v2.0.5
     Compiling cryptography-key-parsing v0.1.0 (/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/src/rust/cryptography-key-parsing)
     Compiling cryptography-cffi v0.1.0 (/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/src/rust/cryptography-cffi)
     Compiling cryptography-openssl v0.1.0 (/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/src/rust/cryptography-openssl)
  The following warnings were emitted during compilation:

  warning: [email protected]: /tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/src/rust/target/release/build/cryptography-cffi-aa5ee4323e7684cb/out/_openssl.c:638:10: fatal error: openssl/engine.h: No such file or directory
  warning: [email protected]:   638 | #include <openssl/engine.h>
  warning: [email protected]:       |          ^~~~~~~~~~~~~~~~~~
  warning: [email protected]: compilation terminated.

  error: failed to run custom build command for `cryptography-cffi v0.1.0 (/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/src/rust/cryptography-cffi)`

  Caused by:
    process didn't exit successfully: `/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/src/rust/target/release/build/cryptography-cffi-6a99f89988fe3b02/build-script-build` (exit status: 1)
    --- stdout
    cargo:rustc-check-cfg=cfg(python_implementation, values("CPython", "PyPy"))
    cargo:rerun-if-env-changed=PYO3_PYTHON
    cargo:rerun-if-changed=../../_cffi_src/
    cargo:rerun-if-changed=../../cryptography/__about__.py
    cargo:rustc-cfg=python_implementation="CPython"
    OUT_DIR = Some(/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/src/rust/target/release/build/cryptography-cffi-aa5ee4323e7684cb/out)
    TARGET = Some(x86_64-unknown-linux-gnu)
    OPT_LEVEL = Some(3)
    HOST = Some(x86_64-unknown-linux-gnu)
    cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
    CC_x86_64-unknown-linux-gnu = None
    cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu
    CC_x86_64_unknown_linux_gnu = None
    cargo:rerun-if-env-changed=HOST_CC
    HOST_CC = None
    cargo:rerun-if-env-changed=CC
    CC = None
    cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
    RUSTC_WRAPPER = None
    cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
    CRATE_CC_NO_DEFAULTS = None
    DEBUG = Some(false)
    CARGO_CFG_TARGET_FEATURE = Some(fxsr,sse,sse2)
    cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
    CFLAGS_x86_64-unknown-linux-gnu = None
    cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
    CFLAGS_x86_64_unknown_linux_gnu = None
    cargo:rerun-if-env-changed=HOST_CFLAGS
    HOST_CFLAGS = None
    cargo:rerun-if-env-changed=CFLAGS
    CFLAGS = None
    OUT_DIR = Some(/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/src/rust/target/release/build/cryptography-cffi-aa5ee4323e7684cb/out)
    cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
    cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
    CRATE_CC_NO_DEFAULTS = None
    CARGO_CFG_TARGET_FEATURE = Some(fxsr,sse,sse2)
    cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
    CFLAGS_x86_64-unknown-linux-gnu = None
    cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
    CFLAGS_x86_64_unknown_linux_gnu = None
    cargo:rerun-if-env-changed=HOST_CFLAGS
    HOST_CFLAGS = None
    cargo:rerun-if-env-changed=CFLAGS
    CFLAGS = None
    OUT_DIR = Some(/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/src/rust/target/release/build/cryptography-cffi-aa5ee4323e7684cb/out)
    cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
    cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
    CRATE_CC_NO_DEFAULTS = None
    CARGO_CFG_TARGET_FEATURE = Some(fxsr,sse,sse2)
    cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
    CFLAGS_x86_64-unknown-linux-gnu = None
    cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
    CFLAGS_x86_64_unknown_linux_gnu = None
    cargo:rerun-if-env-changed=HOST_CFLAGS
    HOST_CFLAGS = None
    cargo:rerun-if-env-changed=CFLAGS
    CFLAGS = None
    OUT_DIR = Some(/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/src/rust/target/release/build/cryptography-cffi-aa5ee4323e7684cb/out)
    cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
    cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
    CRATE_CC_NO_DEFAULTS = None
    CARGO_CFG_TARGET_FEATURE = Some(fxsr,sse,sse2)
    cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
    CFLAGS_x86_64-unknown-linux-gnu = None
    cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
    CFLAGS_x86_64_unknown_linux_gnu = None
    cargo:rerun-if-env-changed=HOST_CFLAGS
    HOST_CFLAGS = None
    cargo:rerun-if-env-changed=CFLAGS
    CFLAGS = None
    OUT_DIR = Some(/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/src/rust/target/release/build/cryptography-cffi-aa5ee4323e7684cb/out)
    cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
    cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
    CRATE_CC_NO_DEFAULTS = None
    CARGO_CFG_TARGET_FEATURE = Some(fxsr,sse,sse2)
    cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
    CFLAGS_x86_64-unknown-linux-gnu = None
    cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
    CFLAGS_x86_64_unknown_linux_gnu = None
    cargo:rerun-if-env-changed=HOST_CFLAGS
    HOST_CFLAGS = None
    cargo:rerun-if-env-changed=CFLAGS
    CFLAGS = None
    cargo:warning=/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/src/rust/target/release/build/cryptography-cffi-aa5ee4323e7684cb/out/_openssl.c:638:10: fatal error: openssl/engine.h: No such file or directory
    cargo:warning=  638 | #include <openssl/engine.h>
    cargo:warning=      |          ^~~~~~~~~~~~~~~~~~
    cargo:warning=compilation terminated.

    --- stderr


    error occurred: Command "cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "/usr/include" "-I" "/usr/include/python3.13" "-Wall" "-Wextra" "-Wconversion" "-Wno-error=sign-conversion" "-Wno-unused-parameter" "-fmacro-prefix-map=/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/src/rust/target/release/build/cryptography-cffi-aa5ee4323e7684cb/out=." "-DPy_LIMITED_API=0x030700f0" "-o" "/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/src/rust/target/release/build/cryptography-cffi-aa5ee4323e7684cb/out/3b3415f8e17501e8-_openssl.o" "-c" "/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/src/rust/target/release/build/cryptography-cffi-aa5ee4323e7684cb/out/_openssl.c" with args cc did not execute successfully (status code exit status: 1).


  warning: build failed, waiting for other jobs to finish...
  💥 maturin failed
    Caused by: Failed to build a native library through cargo
    Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_ENVIRONMENT_SIGNATURE="cpython-3.13-64bit" PYO3_PYTHON="/usr/bin/python3" PYTHON_SYS_EXECUTABLE="/usr/bin/python3" "cargo" "rustc" "--features" "pyo3/abi3-py37" "--message-format" "json-render-diagnostics" "--locked" "--manifest-path" "/tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc/src/rust/Cargo.toml" "--release" "--lib"`
  Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/usr/bin/python3', '--compatibility', 'off'] returned non-zero exit status 1
  error: subprocess-exited-with-error
  
  × Building wheel for cryptography (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /usr/bin/python3 /usr/lib/python3.13/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmpfe0e2w5w
  cwd: /tmp/pip-install-efxjzd2r/cryptography_4757ed757203406189be02ed0ab66efc
  Building wheel for cryptography (pyproject.toml) ... error
  ERROR: Failed building wheel for cryptography

CentOS Stream 10

  • Versions of Python, cryptography, cffi, pip, and setuptools
    you're using
    • Python 3.12.6
    • Pip 23.3.2
    • Cryptography 43.0.1
    • No other packages installed for reproducing, cffi 1.17.1 and setuptools 75.1.0 were downloaded by pip
  • How you installed cryptography
    • pip install --verbose --no-binary :all: cryptography
  • Clear steps for reproducing your bug
    • Install a CentOS Stream 10 VM
    • dnf install gcc libffi-devel openssl-devel rust-devel cargo python3-devel python3-pip pkg-config
    • pip install --verbose --no-binary :all: cryptography
[root@c10s-a ~]# pip install --verbose --no-binary :all: cryptography
Using pip 23.3.2 from /usr/lib/python3.12/site-packages/pip (python 3.12)
Collecting cryptography
  Using cached cryptography-43.0.1.tar.gz (686 kB)
  Running command pip subprocess to install build dependencies
  Collecting maturin<2,>=1
    Using cached maturin-1.7.4-cp312-cp312-linux_x86_64.whl
  Collecting cffi>=1.12
    Using cached cffi-1.17.1-cp312-cp312-linux_x86_64.whl
  Collecting setuptools!=74.0.0,!=74.1.0,!=74.1.1
    Using cached setuptools-75.1.0-py3-none-any.whl
  Collecting pycparser (from cffi>=1.12)
    Using cached pycparser-2.22-py3-none-any.whl
  Installing collected packages: setuptools, pycparser, maturin, cffi
  Successfully installed cffi-1.17.1 maturin-1.7.4 pycparser-2.22 setuptools-75.1.0
  WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
  Installing build dependencies ... done
  Running command Getting requirements to build wheel
  Getting requirements to build wheel ... done
  Running command Preparing metadata (pyproject.toml)
  📦 Including license file "/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/LICENSE"
  📦 Including license file "/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/LICENSE.APACHE"
  📦 Including license file "/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/LICENSE.BSD"
  🍹 Building a mixed python/rust project
  🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.7
  🐍 Not using a specific python interpreter
  📡 Using build options features, locked from pyproject.toml
  cryptography-43.0.1.dist-info
  Checking for Rust toolchain....
  Running `maturin pep517 write-dist-info --metadata-directory /tmp/pip-modern-metadata-cnrj5wet --interpreter /usr/bin/python3`
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: cffi>=1.12 in /usr/local/lib64/python3.12/site-packages (from cryptography) (1.17.1)
Requirement already satisfied: pycparser in /usr/local/lib/python3.12/site-packages (from cffi>=1.12->cryptography) (2.22)
Building wheels for collected packages: cryptography
  Running command Building wheel for cryptography (pyproject.toml)
  Running `maturin pep517 build-wheel -i /usr/bin/python3 --compatibility off`
  📦 Including license file "/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/LICENSE"
  📦 Including license file "/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/LICENSE.APACHE"
  📦 Including license file "/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/LICENSE.BSD"
  🍹 Building a mixed python/rust project
  🔗 Found pyo3 bindings with abi3 support for Python ≥ 3.7
  🐍 Not using a specific python interpreter
  📡 Using build options features, locked from pyproject.toml
     Compiling target-lexicon v0.12.15
     Compiling proc-macro2 v1.0.86
     Compiling unicode-ident v1.0.12
     Compiling cc v1.1.6
     Compiling pyo3-build-config v0.22.2
     Compiling quote v1.0.36
     Compiling syn v2.0.71
     Compiling pkg-config v0.3.30
     Compiling once_cell v1.19.0
     Compiling vcpkg v0.2.15
     Compiling openssl-sys v0.9.103
     Compiling libc v0.2.155
     Compiling cfg-if v1.0.0
     Compiling pyo3-ffi v0.22.2
     Compiling pyo3-macros-backend v0.22.2
     Compiling autocfg v1.3.0
     Compiling memoffset v0.9.1
     Compiling openssl v0.10.66
     Compiling heck v0.5.0
     Compiling foreign-types-shared v0.1.1
     Compiling foreign-types v0.3.2
     Compiling pyo3 v0.22.2
     Compiling bitflags v2.6.0
     Compiling cryptography-key-parsing v0.1.0 (/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/src/rust/cryptography-key-parsing)
     Compiling indoc v2.0.5
     Compiling unindent v0.2.3
     Compiling cryptography-cffi v0.1.0 (/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/src/rust/cryptography-cffi)
     Compiling cryptography-openssl v0.1.0 (/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/src/rust/cryptography-openssl)
  The following warnings were emitted during compilation:

  warning: [email protected]: /tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/src/rust/target/release/build/cryptography-cffi-3885bb56678b35ca/out/_openssl.c:638:10: fatal error: openssl/engine.h: No such file or directory
  warning: [email protected]:   638 | #include <openssl/engine.h>
  warning: [email protected]:       |          ^~~~~~~~~~~~~~~~~~
  warning: [email protected]: compilation terminated.

  error: failed to run custom build command for `cryptography-cffi v0.1.0 (/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/src/rust/cryptography-cffi)`

  Caused by:
    process didn't exit successfully: `/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/src/rust/target/release/build/cryptography-cffi-05f3521f571e1fe0/build-script-build` (exit status: 1)
    --- stdout
    cargo:rustc-check-cfg=cfg(python_implementation, values("CPython", "PyPy"))
    cargo:rerun-if-env-changed=PYO3_PYTHON
    cargo:rerun-if-changed=../../_cffi_src/
    cargo:rerun-if-changed=../../cryptography/__about__.py
    cargo:rustc-cfg=python_implementation="CPython"
    OUT_DIR = Some(/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/src/rust/target/release/build/cryptography-cffi-3885bb56678b35ca/out)
    TARGET = Some(x86_64-unknown-linux-gnu)
    OPT_LEVEL = Some(3)
    HOST = Some(x86_64-unknown-linux-gnu)
    cargo:rerun-if-env-changed=CC_x86_64-unknown-linux-gnu
    CC_x86_64-unknown-linux-gnu = None
    cargo:rerun-if-env-changed=CC_x86_64_unknown_linux_gnu
    CC_x86_64_unknown_linux_gnu = None
    cargo:rerun-if-env-changed=HOST_CC
    HOST_CC = None
    cargo:rerun-if-env-changed=CC
    CC = None
    cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
    RUSTC_WRAPPER = None
    cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
    CRATE_CC_NO_DEFAULTS = None
    DEBUG = Some(false)
    CARGO_CFG_TARGET_FEATURE = Some(avx,avx2,bmi1,bmi2,cmpxchg16b,f16c,fma,fxsr,lzcnt,movbe,popcnt,sse,sse2,sse3,sse4.1,sse4.2,ssse3,xsave)
    cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
    CFLAGS_x86_64-unknown-linux-gnu = None
    cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
    CFLAGS_x86_64_unknown_linux_gnu = None
    cargo:rerun-if-env-changed=HOST_CFLAGS
    HOST_CFLAGS = None
    cargo:rerun-if-env-changed=CFLAGS
    CFLAGS = None
    OUT_DIR = Some(/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/src/rust/target/release/build/cryptography-cffi-3885bb56678b35ca/out)
    cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
    cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
    CRATE_CC_NO_DEFAULTS = None
    CARGO_CFG_TARGET_FEATURE = Some(avx,avx2,bmi1,bmi2,cmpxchg16b,f16c,fma,fxsr,lzcnt,movbe,popcnt,sse,sse2,sse3,sse4.1,sse4.2,ssse3,xsave)
    cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
    CFLAGS_x86_64-unknown-linux-gnu = None
    cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
    CFLAGS_x86_64_unknown_linux_gnu = None
    cargo:rerun-if-env-changed=HOST_CFLAGS
    HOST_CFLAGS = None
    cargo:rerun-if-env-changed=CFLAGS
    CFLAGS = None
    OUT_DIR = Some(/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/src/rust/target/release/build/cryptography-cffi-3885bb56678b35ca/out)
    cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
    cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
    CRATE_CC_NO_DEFAULTS = None
    CARGO_CFG_TARGET_FEATURE = Some(avx,avx2,bmi1,bmi2,cmpxchg16b,f16c,fma,fxsr,lzcnt,movbe,popcnt,sse,sse2,sse3,sse4.1,sse4.2,ssse3,xsave)
    cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
    CFLAGS_x86_64-unknown-linux-gnu = None
    cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
    CFLAGS_x86_64_unknown_linux_gnu = None
    cargo:rerun-if-env-changed=HOST_CFLAGS
    HOST_CFLAGS = None
    cargo:rerun-if-env-changed=CFLAGS
    CFLAGS = None
    OUT_DIR = Some(/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/src/rust/target/release/build/cryptography-cffi-3885bb56678b35ca/out)
    cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
    cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
    CRATE_CC_NO_DEFAULTS = None
    CARGO_CFG_TARGET_FEATURE = Some(avx,avx2,bmi1,bmi2,cmpxchg16b,f16c,fma,fxsr,lzcnt,movbe,popcnt,sse,sse2,sse3,sse4.1,sse4.2,ssse3,xsave)
    cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
    CFLAGS_x86_64-unknown-linux-gnu = None
    cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
    CFLAGS_x86_64_unknown_linux_gnu = None
    cargo:rerun-if-env-changed=HOST_CFLAGS
    HOST_CFLAGS = None
    cargo:rerun-if-env-changed=CFLAGS
    CFLAGS = None
    OUT_DIR = Some(/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/src/rust/target/release/build/cryptography-cffi-3885bb56678b35ca/out)
    cargo:rerun-if-env-changed=CC_ENABLE_DEBUG_OUTPUT
    cargo:rerun-if-env-changed=CRATE_CC_NO_DEFAULTS
    CRATE_CC_NO_DEFAULTS = None
    CARGO_CFG_TARGET_FEATURE = Some(avx,avx2,bmi1,bmi2,cmpxchg16b,f16c,fma,fxsr,lzcnt,movbe,popcnt,sse,sse2,sse3,sse4.1,sse4.2,ssse3,xsave)
    cargo:rerun-if-env-changed=CFLAGS_x86_64-unknown-linux-gnu
    CFLAGS_x86_64-unknown-linux-gnu = None
    cargo:rerun-if-env-changed=CFLAGS_x86_64_unknown_linux_gnu
    CFLAGS_x86_64_unknown_linux_gnu = None
    cargo:rerun-if-env-changed=HOST_CFLAGS
    HOST_CFLAGS = None
    cargo:rerun-if-env-changed=CFLAGS
    CFLAGS = None
    cargo:warning=/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/src/rust/target/release/build/cryptography-cffi-3885bb56678b35ca/out/_openssl.c:638:10: fatal error: openssl/engine.h: No such file or directory
    cargo:warning=  638 | #include <openssl/engine.h>
    cargo:warning=      |          ^~~~~~~~~~~~~~~~~~
    cargo:warning=compilation terminated.

    --- stderr


    error occurred: Command "cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-m64" "-I" "/usr/include" "-I" "/usr/include/python3.12" "-Wall" "-Wextra" "-Wconversion" "-Wno-error=sign-conversion" "-Wno-unused-parameter" "-fmacro-prefix-map=/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/src/rust/target/release/build/cryptography-cffi-3885bb56678b35ca/out=." "-DPy_LIMITED_API=0x030700f0" "-o" "/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/src/rust/target/release/build/cryptography-cffi-3885bb56678b35ca/out/f85f21c44af9c842-_openssl.o" "-c" "/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/src/rust/target/release/build/cryptography-cffi-3885bb56678b35ca/out/_openssl.c" with args cc did not execute successfully (status code exit status: 1).


  warning: build failed, waiting for other jobs to finish...
  💥 maturin failed
    Caused by: Failed to build a native library through cargo
    Caused by: Cargo build finished with "exit status: 101": `env -u CARGO PYO3_ENVIRONMENT_SIGNATURE="cpython-3.12-64bit" PYO3_PYTHON="/usr/bin/python3" PYTHON_SYS_EXECUTABLE="/usr/bin/python3" "cargo" "rustc" "--features" "pyo3/abi3-py37" "--message-format" "json-render-diagnostics" "--locked" "--manifest-path" "/tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad/src/rust/Cargo.toml" "--release" "--lib"`
  Error: command ['maturin', 'pep517', 'build-wheel', '-i', '/usr/bin/python3', '--compatibility', 'off'] returned non-zero exit status 1
  error: subprocess-exited-with-error
  
  × Building wheel for cryptography (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /usr/bin/python3 /usr/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py build_wheel /tmp/tmpibb3_33o
  cwd: /tmp/pip-install-yu819v4f/cryptography_3f49648c66844330a99094e9c78232ad
  Building wheel for cryptography (pyproject.toml) ... error
  ERROR: Failed building wheel for cryptography
Failed to build cryptography
ERROR: Could not build wheels for cryptography, which is required to install pyproject.toml-based projects
@alex
Copy link
Member

alex commented Oct 2, 2024

That PR was closed without merging because we had follow up questions before it could be merged and never got a response.

Those questions remain, namely: Why RedHat's behavior here is different from upstream's no-engine, which we support and test against.

If we can get a clear answer to that question, we can proceed, but from my perspective this is blocked on Red Hat, not us.

@CtrlZmaster
Copy link
Contributor Author

Ah, my bad, I read the discussion wrong in #11328.

I do not feel fully qualified to answer, I am just a cryptography user, so by no means an expert on OpenSSL. But I read the change proposal for Fedora and the discussion for a rejected proposal to remove OpenSSL engines. I think that I see what is going on here and I understand this step from maintenance/packaging perspective.

First, there were concerns that providers still have issues and prevent full switchover from engines. That prompted another approach (the second accepted proposal), deprecating engines but still keeping them for packages that cannot switch over to providers. So OpenSSL is not built with --no-engine. Instead, the engine headers are simply moved from package openssl-devel to openssl-devel-engines which is marked as deprecated. And then OPENSSL_NO_ENGINE is defined mimicking the build without engine support. Fedora Packaging Guidelines prevent additions of new packages with deprecated dependencies. Packages that are ready to switch can do it now. Then there are existing packages that might not know about the deprecation of engines or cannot switch to providers yet. This approach will give them time to coordinate the switch after their builds will start to fail (this has already happened). So, they can easily fix for now by depending on openssl-devel-engines and start replacing them.

I would also like to point out that this is not a "Red Hat behavior" or a Red Hat decision. This approach was chosen by Fedora community, during a Fedora Change process and approved by FESCo (Fedora Engineering Steering Committee). CentOS Stream 10 is simply following the Fedora approach as its downstream.

@alex
Copy link
Member

alex commented Oct 6, 2024

With that context, I guess we'd be happy to have the PR re-sent.

@CtrlZmaster CtrlZmaster mentioned this issue Oct 8, 2024
Merged
CtrlZmaster added a commit to CtrlZmaster/knet-ci-tools that referenced this issue Oct 10, 2024
@CtrlZmaster
use downstream cryptography
cf4c1c2 
Cryptography is currently broken upstream due to deprecation of OpenSSL
engines in Fedora 41+ and CentOS Stream 10. I manage to merge a fix
upstream, but that will take some time until they release a new version.
For more details, see the GitHub issue:
pyca/cryptography#11690
CtrlZmaster added a commit to CtrlZmaster/knet-ci-tools that referenced this issue Oct 10, 2024
@CtrlZmaster
use downstream cryptography
f30050d 
Cryptography is currently broken upstream due to deprecation of OpenSSL
engines in Fedora 41+ and CentOS Stream 10. I managed to merge a fix
upstream, but that will take some time until they release a new version.
For more details, see the GitHub issue:
pyca/cryptography#11690
CtrlZmaster added a commit to CtrlZmaster/knet-ci-tools that referenced this issue Oct 10, 2024
@CtrlZmaster
use downstream cryptography
fcc82f9 
Cryptography is currently broken upstream due to deprecation of OpenSSL
engines in Fedora 41+ and CentOS Stream 10. I managed to merge a fix
upstream, but that will take some time until they release a new version.
For more details, see the GitHub issue:
pyca/cryptography#11690
@CtrlZmaster CtrlZmaster mentioned this issue Oct 10, 2024
Merged
CtrlZmaster added a commit to CtrlZmaster/knet-ci-tools that referenced this issue Oct 11, 2024
@CtrlZmaster
use downstream packages as fallback
66d0248 
Cryptography is currently broken upstream due to deprecation of OpenSSL
engines in Fedora 41+ and CentOS Stream 10. I managed to merge a fix
upstream, but it will take some time until they release a new version.
For more details, see the GitHub issue:
pyca/cryptography#11690

When upstream installation fails, --use-system-packages allows using
downstream packages as a fallback. Downstream packages have an advantage
of being tuned to the distribution, which can resolve these kinds of
issues.

Some context for not having --use-system-packages from Fabio:
it was to stress the build system in different ways
one env might have python3-foo installed, another doesn't
causing different results
say today: we test pcs build X and it works
tomorrow some other packages pulls in python3-something that goes from embedded to system
and then pcs starts to fail
it creates a rather unpredictable build env
on rhel8 we could use system stuff, and we needed it IIRC
others can have all embedded
CtrlZmaster added a commit to CtrlZmaster/knet-ci-tools that referenced this issue Oct 15, 2024
@CtrlZmaster
use downstream packages
f2c76c4 
Cryptography is currently broken upstream due to deprecation of OpenSSL
engines in Fedora 41+ and CentOS Stream 10. I managed to merge a fix
upstream, but it will take some time until they release a new version.
For more details, see the GitHub issue:
pyca/cryptography#11690

Before this commit, some packages were installed from pip. This is not
desired, pcs dependencies are given as rpm packages, not python packages.
Presumably, even upstream users use downstream packages if they build
pcs. Downstream packages have an advantage of being tuned for the
distribution, which can resolve the kinds of issues as cryptography has
now. Additionally, the pip installed packages were not the ones that
can be bundled by autotools, so pcs is still tested with the newest
possible bundled dependencies.

Some context for not having --system-site-packages from Fabio:
it was to stress the build system in different ways
one env might have python3-foo installed, another doesn't
causing different results
say today: we test pcs build X and it works
tomorrow some other packages pulls in python3-something that goes from embedded to system
and then pcs starts to fail
it creates a rather unpredictable build env
on rhel8 we could use system stuff, and we needed it IIRC
others can have all embedded
fabbione pushed a commit to kronosnet/ci-tools that referenced this issue Oct 21, 2024
@CtrlZmaster @fabbione
use downstream packages
146f882 
Cryptography is currently broken upstream due to deprecation of OpenSSL
engines in Fedora 41+ and CentOS Stream 10. I managed to merge a fix
upstream, but it will take some time until they release a new version.
For more details, see the GitHub issue:
pyca/cryptography#11690

Before this commit, some packages were installed from pip. This is not
desired, pcs dependencies are given as rpm packages, not python packages.
Presumably, even upstream users use downstream packages if they build
pcs. Downstream packages have an advantage of being tuned for the
distribution, which can resolve the kinds of issues as cryptography has
now. Additionally, the pip installed packages were not the ones that
can be bundled by autotools, so pcs is still tested with the newest
possible bundled dependencies.

Some context for not having --system-site-packages from Fabio:
it was to stress the build system in different ways
one env might have python3-foo installed, another doesn't
causing different results
say today: we test pcs build X and it works
tomorrow some other packages pulls in python3-something that goes from embedded to system
and then pcs starts to fail
it creates a rather unpredictable build env
on rhel8 we could use system stuff, and we needed it IIRC
others can have all embedded
CtrlZmaster added a commit to CtrlZmaster/knet-ci-tools that referenced this issue Oct 22, 2024
@CtrlZmaster
use downstream packages
37afbde 
Cryptography is currently broken upstream due to deprecation of OpenSSL
engines in Fedora 41+ and CentOS Stream 10. I managed to merge a fix
upstream, but it will take some time until they release a new version.
For more details, see the GitHub issue:
pyca/cryptography#11690

Before this commit, some packages were installed from pip. This is not
desired, pcs dependencies are given as rpm packages, not python packages.
Presumably, even upstream users use downstream packages if they build
pcs. Downstream packages have an advantage of being tuned for the
distribution, which can resolve the kinds of issues as cryptography has
now. Additionally, the pip installed packages were not the ones that
can be bundled by autotools, so pcs is still tested with the newest
possible bundled dependencies.

Some context for not having --system-site-packages from Fabio:
it was to stress the build system in different ways
one env might have python3-foo installed, another doesn't
causing different results
say today: we test pcs build X and it works
tomorrow some other packages pulls in python3-something that goes from embedded to system
and then pcs starts to fail
it creates a rather unpredictable build env
on rhel8 we could use system stuff, and we needed it IIRC
others can have all embedded
CtrlZmaster added a commit to CtrlZmaster/knet-ci-tools that referenced this issue Oct 22, 2024
@CtrlZmaster
use downstream packages
8d631db 
Cryptography is currently broken upstream due to deprecation of OpenSSL
engines in Fedora 41+ and CentOS Stream 10. I managed to merge a fix
upstream, but it will take some time until they release a new version.
For more details, see the GitHub issue:
pyca/cryptography#11690

Before this commit, some packages were installed from pip. This is not
desired, pcs dependencies are given as rpm packages, not python packages.
Presumably, even upstream users use downstream packages if they build
pcs. Downstream packages have an advantage of being tuned for the
distribution, which can resolve the kinds of issues as cryptography has
now. Additionally, the pip installed packages were not the ones that
can be bundled by autotools, so pcs is still tested with the newest
possible bundled dependencies.

Some context for not having --system-site-packages from Fabio:
it was to stress the build system in different ways
one env might have python3-foo installed, another doesn't
causing different results
say today: we test pcs build X and it works
tomorrow some other packages pulls in python3-something that goes from embedded to system
and then pcs starts to fail
it creates a rather unpredictable build env
on rhel8 we could use system stuff, and we needed it IIRC
others can have all embedded
fabbione pushed a commit to kronosnet/ci-tools that referenced this issue Oct 22, 2024
@CtrlZmaster @fabbione
use downstream packages
95d44d2 
Cryptography is currently broken upstream due to deprecation of OpenSSL
engines in Fedora 41+ and CentOS Stream 10. I managed to merge a fix
upstream, but it will take some time until they release a new version.
For more details, see the GitHub issue:
pyca/cryptography#11690

Before this commit, some packages were installed from pip. This is not
desired, pcs dependencies are given as rpm packages, not python packages.
Presumably, even upstream users use downstream packages if they build
pcs. Downstream packages have an advantage of being tuned for the
distribution, which can resolve the kinds of issues as cryptography has
now. Additionally, the pip installed packages were not the ones that
can be bundled by autotools, so pcs is still tested with the newest
possible bundled dependencies.

Some context for not having --system-site-packages from Fabio:
it was to stress the build system in different ways
one env might have python3-foo installed, another doesn't
causing different results
say today: we test pcs build X and it works
tomorrow some other packages pulls in python3-something that goes from embedded to system
and then pcs starts to fail
it creates a rather unpredictable build env
on rhel8 we could use system stuff, and we needed it IIRC
others can have all embedded
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Don't include engine.h when OPENSSL_NO_ENGINE is defined CtrlZmaster/python-cryptography
2 participants
@alex @CtrlZmaster