keystone-engine: add build-keystone-engine.yml for riscv64 wheels - #1775
Merged
Conversation
Ports keystone-engine 0.9.2's Python bindings, following upstream's own python-publish.yml. setup.py builds the vendored LLVM 3.9 MC-layer/codegen fork (Arm, Arm64, Hexagon, Mips, PowerPC, Sparc, SystemZ, X86 + EVM) via CMake (make-share.sh) and loads the resulting libkeystone.so via ctypes; no ext_modules and bdist_wheel universal=1 collapse the build to one cp312 job per libc (manylinux + musllinux).
CIBW_TEST_SOURCES copies bindings/python/sample.py to the test dir preserving its project-relative path, not flattened to the test dir root, so the test command needs the same bindings/python/ prefix.
Both manylinux and musllinux riscv64 images carry a newer cmake (>= 4) ahead on PATH by default, which hard-rejects the vendored CMakeLists.txt's cmake_policy(SET CMP0051 OLD) call outright. Install cmake from the image's own package manager (< 4, still honours it) and put it first on PATH. Also replace the vendored llvm/cmake/config.guess (stamped 2011, predates riscv64) with automake's copy, which already recognises it - otherwise CMake's GetHostTriple.cmake fails with 'unable to guess system type'.
config-ix.cmake's LLVM_NATIVE_ARCH table predates RISC-V's addition to upstream LLVM and has no riscv64 case, so cmake dies with 'Unknown architecture riscv64' before target selection. Add a riscv64 branch mapping to an inert placeholder, matching how any host arch absent from keystone's own 8-target build already behaves (native JIT stays unavailable, which keystone's own code never uses).
…arget
llvm-build (utils/llvm-build/llvmbuild/main.py) validates --native-target
against real LLVMBuild target components independently of config-ix.cmake's
own LLVM_TARGETS_TO_BUILD check, so the placeholder from the previous patch
('RISCV', not a real component in this tree) failed there with 'invalid
native target: RISCV (not in project)'. llvm-build already special-cases
the literal string 'Unknown' to mean no native target; use that instead.
Prepending /usr/bin onto PATH to win over the pre-baked >= 4 cmake also shadowed musllinux's system python, which broke cibuildwheel's own 'python on PATH must match its managed interpreter' check. Shadow only cmake via a dedicated one-binary PATH entry instead. Also replace the CIBW_TEST_COMMAND: the manylinux leg got far enough to run it and hit KS_ERR_ASM_INVALIDOPERAND on sample.py's own b"add eax, 15h" call (default non-RADIX16 syntax rejects the bare hex suffix - a numeric-literal parsing quirk in the demo script itself, not riscv64-specific). Inline the rest of sample.py's known-good calls instead of running the script as-is.
The manylinux leg built, tested, and repaired cleanly. musllinux fails at the auditwheel-repair step: setup.py's bdist_wheel --plat-name insertion hardcodes 'manylinux1_' + platform.machine() regardless of actual libc (a bug capstone's near-identical bindings/python setup.py already fixed upstream, capstone#2445, by dropping the hardcoded prefix), so auditwheel refuses to repair a GLIBC-tagged wheel into a musllinux one. Upstream's own python-publish.yml never builds musllinux either - dropping it here isn't a reduced port, it matches upstream's actual Linux matrix exactly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
keystone-engine0.9.2Compiles keystone's vendored LLVM 3.9 MC-layer/codegen fork (Arm, Arm64, Hexagon, Mips, PowerPC, Sparc, SystemZ, X86, EVM) via CMake and packages the resulting libkeystone.so behind the ctypes bindings in bindings/python. Upstream publishes no riscv64 wheel.
Mirrors upstream's
python-publish.yml.Differs from upstream
llvm/cmake/config-ix.cmaketo recognise riscv64 as a host arch - this LLVM 3.9-era snapshot predates RISC-V support in LLVM itself by years.dnfinCIBW_BEFORE_ALL_LINUX-setup.pyshells out to it directly and it is not a declared build dependency; pypi.riseproject.dev lags PyPI's own latest cmake.llvm/cmake/config.guess(stamped 2011) with automake's copy, and setsCMAKE_POLICY_VERSION_MINIMUM=3.5- both needed for cmake compatibility on this arch/toolchain.Matrix: manylinux only, matching upstream's own Linux matrix exactly (
python-publish.ymlnever builds musllinux either).setup.py'sbdist_wheel --plat-nameinsertion hardcodesmanylinux1_+platform.machine()regardless of actual libc - a bug capstone's near-identicalbindings/python/setup.pyalready fixed upstream (capstone#2445) - so auditwheel refuses to repair that GLIBC-tagged wheel into a musllinux one.Testing
bindings/python/sample.pyis the closest thing to a test upstream ships, but one of its own calls (b"add eax, 15h"under the default non-RADIX16 syntax) raisesKS_ERR_ASM_INVALIDOPERANDindependent of any target this build produces - a numeric-literal parsing quirk in the demo script itself, not riscv64-specific.CIBW_TEST_COMMANDinlines the rest of sample.py's known-good calls instead, exercisingks.asm()across every architecture keystone supports (x86, arm, arm64, hexagon, mips, ppc, sparc, systemz).License: OK
Built on cp312-manylinux_riscv64; all eight architecture
asm()calls passed.