Repository navigation
ci: run the jobs in a prebuilt builder image - #332
Merged
Merged
Conversation
build.yml, vcpkg-cache.yml and defender-scan.yml started from archlinux:base-devel and installed the toolchain, osslsigncode, buf and the libCrypt32.a link on every run. .github/builder/Dockerfile holds exactly that; builder-image.yml (workflow_dispatch, packages: write) builds it and pushes ghcr.io/echotools/nevr-runtime-builder:<tag>, tag = first 12 hex of the Dockerfile's sha256. The three workflows name that tag and test_builder_image.py fails when they disagree. Caches: the Buf cache is dropped (the image has buf). The vcpkg key now carries a hash of the MinGW package set, and the seed no longer restores an older entry, which doubled the saved entry with archives that could not match. Co-Authored-By: nevr-work <agents@sprock.io>
Co-Authored-By: nevr-merge <agents@sprock.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
build.yml,vcpkg-cache.ymlanddefender-scan.ymlstarted fromarchlinux:base-develand installed the toolchain on every run (~29 billed minutes forbuild.yml)..github/builder/Dockerfile: exactly what those jobs installed (read from their steps): the pacman set (union of the three lists), thelibCrypt32.alink, osslsigncode 2.14, buf 1.47.2..github/workflows/builder-image.yml:workflow_dispatch,packages: write; builds it and pushesghcr.io/echotools/nevr-runtime-builder:<tag>, tag = first 12 hex of the Dockerfile's sha256 (42010dd4ba8e). An existing tag is not rebuilt.packages: read, GITHUB_TOKEN credentials).tools/tests/test_builder_image.pyfails if a workflow's tag is not the Dockerfile's hash.Needs one manual dispatch before any workflow can run:
gh workflow run builder-image.yml --ref ci/builder-image(or from main after merge). I did not trigger it. The package may also need its visibility/repo access set once on ghcr.io.Caches: does a run hit them? (past-run logs)
buf-v1.47.2-<go>)Cache not found for input keys: buf-v1.47.2-go1.27.2-X:nodwarf5, then saved. Runs 37950708596 and 37943604255 hit, but only on their own PR ref.gh cache listshows 12+ identical entries, one per ref, becausebuild.ymlnever runs onmain.vcpkg-bin2-…)Cache hit,archives restored from the cache entry: 38, butRestored 5 package(s)ofInstalling …/19; 14x64-mingw-staticports rebuilt (protobuf 8.9 min, abseil 1.3 min, curl 50 s), ending atarchives in the binary cache now: 52. Exact hit means no save, so it repeats every run. Seed run 37914950616 restored an older entry as fallback (Restored 0 package(s)) and saved 38 archives (19 unusable), 231 MB -> 547 MB.android-ndk-r26d-linux)Cache hit for: android-ndk-r26d-linux,Pkg.Revision = 26.3.11579264vcpkg-android-…)Cache hit,Restored 3 package(s),archives in the binary cache: 3vcpkg fix, and what I could not prove
I could not reproduce the 14-archive mismatch. In a container experiment (
/var/tmp/work-nevr-runtime/claude-ci/abi/), the image and a barearchlinux:base-develwith the seed's package list produced identical archive hashes forzlibandminhookonx64-mingw-static(11600b49…,74544324…,761a7597…,b3f415de…), and the logged toolchain versions are identical across runs 37943604255 and 38028853975. So the cause is not the package set, the libCrypt32 link or the logged versions. It remains unproven; the likeliest candidate is something unlogged that changed between the seed (10-09 10:00 UTC) and the builds (mingw-w64-binutilsis not in the "Toolchain versions" output). Changes that make a mismatch visible and non-sticky:binutilsis added to the "Toolchain versions" step.The first seeded run after merge should log
Restored 19 package(s)inbuild.yml; if it does not, the cause is still open.No issue: operational CI change requested by the owner's lane order, no defect tracked.
How it was verified
python3 -m unittest tools.tests.test_builder_image tools.tests.test_vcpkg_pin tools.tests.test_release_contract:Ran 27 tests ... OKdocker buildof the Dockerfile locally (previous revision of the comment only): image built, 4.67 GB.yaml.safe_load).just verifyon tip 0cf3b50:verify: OK (mingw-release).Merge note:
build.yml,vcpkg-cache.yml,defender-scan.ymlandtest_vcpkg_pin.pyare also edited by #331 (on-demand triggers); the hunks are adjacent, expect a trivial merge.Branch lifecycle: deleted when this PR merges or closes.