Skip to content

Also use the GitHub Actions cache for CI builds - #446

Merged
lyrixx merged 1 commit into
mainfrom
ci-gha-cache
Oct 6, 2026
Merged

lyrixx merged 1 commit into
mainfrom
ci-gha-cache

Conversation

@lyrixx

@lyrixx lyrixx commented Oct 6, 2026

Copy link
Copy Markdown
Member

Refs #430

The registry cache (type=registry, mode=max) is regularly missed for some layers of the builder image in the CI: no error and no warning, BuildKit just rebuilds the layer and everything after it.

As suggested in the issue, CI builds now also use the GitHub Actions cache backend (type=gha):

  • New infrastructure/docker/docker-compose.ci.yml, loaded by the ci context. It adds a type=gha cache_from and cache_to (mode=max,ignore-error=true) to frontend and builder. Each service and PHP version gets its own scope, otherwise they overwrite each other.
  • ci.yml exposes the GitHub runtime (ACTIONS_RUNTIME_TOKEN, ACTIONS_RESULTS_URL) with crazy-max/ghaction-github-runtime, which buildx needs to reach this cache from a run: step.
  • The registry cache is kept for local development. castor docker:push still pushes it: the gha entries are appended after the registry one, and its --set cache-to replaces the gha cache_to.

Outside GitHub Actions (e.g. castor -c ci start locally), buildx silently skips type=gha entries when there is no token (isActive() in buildx build/opt.go), so nothing breaks.

Side effect: the PHP 8.3 and 8.4 matrix jobs now get a cache too. The registry cache is only pushed for 8.5.

@lyrixx

lyrixx commented Oct 6, 2026

Copy link
Copy Markdown
Member Author

I re-ran the CI to check that a run can read the GitHub Actions cache written by an earlier run (attempt 2 of run 37440875854).

Result: no builder or frontend step was rebuilt, in any of the 3 PHP jobs. That includes builder 2/9 (the apt-get install step that was missing the cache in #430).

The PHP 8.3 and 8.4 jobs have no registry cache, since cache.yml only pushes it for 8.5. So their cached steps can only come from the type=gha cache.

castor start duration (attempt 1 ran with an empty gha cache):

Job Attempt 1 Attempt 2
PHP 8.3 115s 52s
PHP 8.4 98s 59s
PHP 8.5 67s 48s

The few remaining DONE lines aren't rebuilds:

  • the final image pulls its cached layers;
  • BuildKit always re-checks the remote file of ADD https://… (0.1s).

This shows the gha cache is read back between runs. It doesn't prove yet that the intermittent misses from #430 are gone, since they were random. We'll see over the next runs.

The registry cache is regularly missed for some layers of the builder
image in the CI, silently falling back to a full rebuild (#430).

The "ci" context now loads docker-compose.ci.yml, which adds the GitHub
Actions cache backend (type=gha) to the frontend and builder images: every
CI build reads it and writes it back, with one scope per service and PHP
version. The registry cache is kept, it is still needed for local
development, and stays the one pushed by "castor docker:push".

buildx needs the GitHub runtime to reach this cache, which GitHub only
gives to JavaScript actions: a local, dependency-free action
(.github/actions/expose-github-runtime) exposes it to the next steps, and
selects the v2 cache API. Without it (outside GitHub Actions), buildx
silently ignores the type=gha entries.

Co-authored-by: Damien Alexandre <dalexandre@jolicode.com>
@lyrixx

lyrixx commented Oct 6, 2026

Copy link
Copy Markdown
Member Author

Update: no more external dependency. crazy-max/ghaction-github-runtime is replaced by a local, dependency-free action (.github/actions/expose-github-runtime), so the job token never goes through a third-party action.

It re-exports to the next steps the variables buildx needs to reach the GitHub Actions cache (ACTIONS_RUNTIME_TOKEN, ACTIONS_RESULTS_URL, ACTIONS_CACHE_URL). It also sets ACTIONS_CACHE_SERVICE_V2 to true when the runner doesn't, otherwise buildx falls back to the legacy v1 cache API and fails with an HTTP 400.

All of this comes from @damienalexandre's investigation, he's co-author of the commit. Thanks Damien!

@damienalexandre damienalexandre left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed with fresh battle scars from a downstream project where we validated the same mechanics — including the two traps this PR already handles.

Verified working (from this PR's own runs):

  • The CASTOR_CONTEXT: ci workflow env selects the ci context, so docker-compose.ci.yml is loaded: the latest run shows importing cache manifest from gha:... for both services, builder layers CACHED, and two successive runs went 1m54 -> 1m15. The import side works.
  • The per-service + per-PHP-version scope entries are required here: without them, the three matrix jobs would share the default scope and overwrite each other's cache (last writer wins). Good catch.
  • The push() analysis is correct: the merged cache_from keeps the registry entry at index 0, and --set cache-to replaces the gha cache_to during the push bake, so the registry cache for local dev is preserved. We verified the same mechanics downstream.
  • The local action + ACTIONS_CACHE_SERVICE_V2=true matches what we found: without the v2 flag, buildx selects the legacy cache API (build/opt.go addGithubToken()) and every import/export fails with an opaque HTTP 400 ("Our services aren't available right now"). Dependency-free and in-repo is the right call for a token handler.
  • Docs (README + CHANGELOG) look good.

Suggestions (none blocking):

  1. Downstream footgun: this only works because the workflow sets CASTOR_CONTEXT: ci. Generated projects often don't — we run the CI with the default context, so there we hook the compose file on GITHUB_ACTIONS in the default context instead. Consider either that variant or a README warning, since the template gets copied around and the failure mode is a silent no-op.
  2. router is not covered: its COPY traefik + sed layers rebuild on every run (~5-10s). worker_messenger is a bare FROM php-base AS worker so there is nothing extra to cache, but router is cheap to add.
  3. index.cjs writes single-line NAME=value entries to $GITHUB_ENV: fine for these values (tokens and URLs are single-line), but if the file ever gets reused for other variables, the heredoc format (NAME<<DELIM ... DELIM) is the defensive form.
  4. ignore-error=true on cache_to: reasonable default, with a known trade-off — a failing export becomes invisible and the cache just goes stale. It can only cause misses, never wrong layers (records match by content hash), so it is acceptable.
  5. Explicit scopes are branch-agnostic: PR runs write the same keys as main, so a PR can overwrite the cache main reads. Same conclusion as above: worst case is a miss, never corruption. Fine, just worth knowing.

One extra data point from the same downstream project (out of scope here): once the layers are reliably cached, the remaining CI time went to the dependency installs, which we additionally cache with actions/cache (PHP vendors + tools node_modules, keyed on the lockfiles). That is downstream-project territory, but worth a README note someday.

@lyrixx
lyrixx merged commit dc5f4ee into main Oct 6, 2026
5 checks passed
@lyrixx
lyrixx deleted the ci-gha-cache branch October 6, 2026 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants