Skip to content

Restore earlier CI caches when the dependencies change - #9279

Draft
cknitt wants to merge 8 commits into
masterfrom
ci-cache-generations
Draft

cknitt wants to merge 8 commits into
masterfrom
ci-cache-generations

Conversation

@cknitt

@cknitt cknitt commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

Two problems with the hash-keyed caches:

  • A change of the opam files misses every OPAM cache. On the master push after Check for dead code in CI #9246, Setup OCaml took 531 s (Linux x64), 643 s (macOS) and 953 s (Windows). A PR that changes the opam files pays this on every push, because PR runs never save caches.
  • The build states were keyed on hashes: of the opam files, and for coverage also of compiler/**. So every master push saved a new ~300 MB coverage state, and each change of the opam files left a superseded generation of every compiler state behind. This is why the repository's caches exceed the 10 GB limit (10.8 GB).

Changes:

  • OPAM environment, pull requests only: when the exact key misses, setup-ocaml restores the newest environment with the same prefix and updates it. Before installing, it unpins all packages; opam install . --deps-only … re-pins whatever the opam files still pin. opam update --repositories makes new versions available, and opam reinstall --forget-pending leaves the installed packages as they are, as on an exact hit. On Windows, the install reads the bash_env that ocaml/setup-ocaml writes.
  • Pushes never fall back: they build a missing environment from scratch, and only such environments are saved. Packages and pins dropped from the opam files, and skipped upgrades, therefore can't build up in the cache.
  • One job always resolves from scratch: the OCaml 5.0 job never restores an earlier environment, so each dependency change gets one fresh resolution on the PR, as master will after the merge.
  • A new restore-earlier input of setup-ocaml (default true) controls the fallback per job.
  • Two system packages are bypassed: binaryen, which we install by hand, and musl-tools, which setup-linux-dependencies unpacks without dpkg. With depext-bypass set for them before every install, opam neither installs them a second time nor misreads them as removed in a restored environment; the latter had rebuilt all 64 packages of the switch. Other system dependencies are still installed.
  • Build states: the compiler and coverage keys no longer end in a hash, since dune rebuilds whatever the dependencies or sources change. Master deletes each state and saves it again, as it already did for the compiler state, so there is one per job. The keys also work as restore prefixes, so the first runs restore the states saved under the old hashed keys.

A change to opam-repositories needs a new cache-prefix, since a restored environment keeps its repositories; the action now says so in a comment. Superseded caches are not pruned: GitHub evicts the least recently used caches first, and those are never used again.

To test this, a temporary commit appended a comment to tools.opam; a later commit reverts it. In run 42871956970, the restored jobs set up OPAM in 35–61 s, against 6–16 min for a full setup. The OCaml 5.0 job resolved from scratch in 618 s. In the ARM Linux job, luv was unpinned and re-pinned to the same fork commit, and opam install printed "Nothing to do". Coverage passed.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Kbf3gKHFTuMrZosy3Nb5Qc

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Changelog

No changelog entry.

Semantic Diff

6 commits · Full page

Reuse CI caches across dependency changes and delete superseded ones

7fdcd8f · cknitt + Claude · Nov 09, 2026, 13:53 UTC

A change of the opam files missed every OPAM cache, so CI rebuilt the environments from scratch (9 to 16 minutes on master today, on every push of a pull request that changes them), and left the old ones behind: the repository's caches exceed the 10 GB limit, so live ones get evicted. The environment and build state of the previous dependencies are now restored and updated, and master deletes superseded generations when it saves. The coverage build state no longer changes key with every compiler commit.

DO NOT MERGE: change an opam file to test the cache fallback

f70d1c8 · cknitt · Nov 09, 2026, 13:54 UTC

Drops before review.

Brace the prefix in the coverage build state key

4123ff1 · cknitt + Claude · Nov 09, 2026, 14:16 UTC

$prefix6d99… named a variable that doesn't exist, so the key was empty and the coverage job failed to restore its build state.

Don't let opam rebuild a restored switch for its system packages

3b0ece7 · cknitt + Claude · Nov 09, 2026, 14:17 UTC

On Linux, opam took the packages that setup-linux-dependencies unpacks without dpkg for removed and recompiled all 64 packages of the restored switch (6 to 9 minutes); the fallback now skips system dependencies.

Drop the temporary opam change

c554720 · cknitt + Claude · Nov 09, 2026, 14:33 UTC

It made this pull request's CI take the cache fallback path, which restored and updated the environments in 14 to 35 seconds on every OS.

✨ Grant build-compiler the permission to delete caches

560a751 · cknitt + Claude · Nov 09, 2026, 19:57 UTC · new

Codex review: the job relied on the repository's default token permissions for the cache deletions of master pushes, which now fail instead of being ignored.

@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

✅ Compiler benchmarks

Allocated words

In total: 683,548,340 → 683,548,644 (±0.00%).

None of the 261 benchmarks changed by 0.1% or more.

Instructions

In total: 9,664,671,823 → 9,664,671,823 (±0.00%).

None of the 33 benchmarks changed by 0.1% or more.

Output size

In total: 25,569,598 → 25,569,598 (±0.00%).

None of the 13 benchmarks changed by 0.1% or more.

Pull request 560a751 compared with base b82d944 in this run.

@pkg-pr-new

pkg-pr-new Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Open in StackBlitz

rescript

npm i https://pkg.pr.new/rescript@9279

@rescript/belt

npm i https://pkg.pr.new/@rescript/belt@9279

@rescript/darwin-arm64

npm i https://pkg.pr.new/@rescript/darwin-arm64@9279

@rescript/linux-arm64

npm i https://pkg.pr.new/@rescript/linux-arm64@9279

@rescript/linux-x64

npm i https://pkg.pr.new/@rescript/linux-x64@9279

@rescript/runtime

npm i https://pkg.pr.new/@rescript/runtime@9279

@rescript/win32-x64

npm i https://pkg.pr.new/@rescript/win32-x64@9279

commit: af15778

@cknitt cknitt added the changelog:skip No release notes needed label Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown

@cknitt

cknitt commented Nov 9, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-10T18:28:38.493513Z af15778 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c55472024f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/ci.yml Resolved
@cknitt

cknitt commented Nov 9, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 560a751d45

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@codecov

codecov Bot commented Nov 9, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.97%. Comparing base (145df8d) to head (af15778).
⚠️ Report is 7 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #9279      +/-   ##
==========================================
+ Coverage   81.93%   81.97%   +0.03%     
==========================================
  Files         456      456              
  Lines       65517    65613      +96     
==========================================
+ Hits        53679    53783     +104     
+ Misses      11838    11830       -8     

see 11 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

cknitt and others added 4 commits November 10, 2026 13:22
A change of the opam files missed every OPAM cache, so CI rebuilt the
environments from scratch (9 to 16 minutes on master today, on every push
of a pull request that changes them), and left the old ones behind: the
repository's caches exceed the 10 GB limit, so live ones get evicted. The
environment and build state of the previous dependencies are now restored
and updated, and master deletes superseded generations when it saves. The
coverage build state no longer changes key with every compiler commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kbf3gKHFTuMrZosy3Nb5Qc
Signed-off-by: Christoph Knittel <ck@cca.io>
`$prefix6d99…` named a variable that doesn't exist, so the key was empty
and the coverage job failed to restore its build state.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kbf3gKHFTuMrZosy3Nb5Qc
Signed-off-by: Christoph Knittel <ck@cca.io>
On Linux, opam took the packages that setup-linux-dependencies unpacks
without dpkg for removed and recompiled all 64 packages of the restored
switch (6 to 9 minutes); the fallback now skips system dependencies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kbf3gKHFTuMrZosy3Nb5Qc
Signed-off-by: Christoph Knittel <ck@cca.io>
Codex review: the job relied on the repository's default token
permissions for the cache deletions of master pushes, which now fail
instead of being ignored.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kbf3gKHFTuMrZosy3Nb5Qc
Signed-off-by: Christoph Knittel <ck@cca.io>
@cknitt
cknitt force-pushed the ci-cache-generations branch from 560a751 to 2cf79a9 Compare November 10, 2026 13:23
@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Changelog

No changelog entry.

Semantic Diff

8 commits · Full page

Reuse CI caches across dependency changes and delete superseded ones

5d81756 · cknitt + Claude · Nov 09, 2026, 13:53 UTC

A change of the opam files missed every OPAM cache, so CI rebuilt the environments from scratch (9 to 16 minutes on master today, on every push of a pull request that changes them), and left the old ones behind: the repository's caches exceed the 10 GB limit, so live ones get evicted. The environment and build state of the previous dependencies are now restored and updated, and master deletes superseded generations when it saves. The coverage build state no longer changes key with every compiler commit.

Brace the prefix in the coverage build state key

014e1ea · cknitt + Claude · Nov 09, 2026, 14:16 UTC

$prefix6d99… named a variable that doesn't exist, so the key was empty and the coverage job failed to restore its build state.

Don't let opam rebuild a restored switch for its system packages

4a1637f · cknitt + Claude · Nov 09, 2026, 14:17 UTC

On Linux, opam took the packages that setup-linux-dependencies unpacks without dpkg for removed and recompiled all 64 packages of the restored switch (6 to 9 minutes); the fallback now skips system dependencies.

Grant build-compiler the permission to delete caches

2cf79a9 · cknitt + Claude · Nov 09, 2026, 19:57 UTC

Codex review: the job relied on the repository's default token permissions for the cache deletions of master pushes, which now fail instead of being ignored.

Restore earlier OPAM environments for pull requests only and stop pruning

d965a5e · cknitt + Claude · Nov 10, 2026, 14:52 UTC

A restored environment is only ever added to, and master saved it under the new key: a pin dropped from the opam files, a removed dependency or a missed upgrade stayed in every later environment, and only a prefix bump recovered. Pushes now build a missing environment anew, and only pull requests restore an earlier one. The pruning of superseded caches is gone: GitHub evicts the least recently used caches first, which are those, and its errors could fail master builds. On a fallback, opam keeps the installed packages as they are and bypasses only the two system packages it misreads, so new system dependencies are still installed.

Review

  • Why keep the build state's fallback on master? dune rebuilds what the new dependencies change, so a restored state can't go stale.
Unpin restored environments and resolve one job from scratch

3b8e0af · cknitt + Claude · Nov 10, 2026, 18:06 UTC

Review fixes for the pull request fallback. Pins of earlier dependencies are dropped before the install, which pins what the opam files still pin; the OCaml 5.0 job never restores an earlier environment, so a dependency change still gets one fresh resolution before the merge; and Windows installs read the bash_env that ocaml/setup-ocaml writes. The depext bypass now applies to every install, and only environments resolved from scratch are saved. The build state keys no longer end in a hash of the opam files, so master keeps one state per job.

DO NOT MERGE: change an opam file to test the cache fallback

449b4bd · cknitt · Nov 10, 2026, 14:52 UTC

Dropped before merging; it makes this pull request's CI take the fallback path.

✨ Drop the temporary opam change

af15778 · cknitt + Claude · Nov 10, 2026, 18:26 UTC · new

It made this pull request's CI take the fallback path: the restored jobs set up OPAM in 35 to 61 seconds, the OCaml 5.0 job resolved from scratch.

@github-actions

github-actions Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

✅ Compiler benchmarks

Allocated words

In total: 683,223,534 → 683,223,697 (±0.00%).

Generated projects: 923,212,498 → 923,212,548 (±0.00%).

None of the 265 benchmarks changed by 0.1% or more.

Instructions

In total: 9,666,123,297 → 9,666,123,297 (±0.00%).

None of the 33 benchmarks changed by 0.1% or more.

Output size

In total: 25,841,785 → 25,841,785 (±0.00%).

None of the 13 benchmarks changed by 0.1% or more.

Pull request af15778 compared with base dcedd5d in this run.

@github-actions

Copy link
Copy Markdown

…ning

A restored environment is only ever added to, and master saved it under
the new key: a pin dropped from the opam files, a removed dependency or a
missed upgrade stayed in every later environment, and only a prefix bump
recovered. Pushes now build a missing environment anew, and only pull
requests restore an earlier one. The pruning of superseded caches is gone:
GitHub evicts the least recently used caches first, which are those, and
its errors could fail master builds. On a fallback, opam keeps the
installed packages as they are and bypasses only the two system packages
it misreads, so new system dependencies are still installed.

Review: Why keep the build state's fallback on master? dune rebuilds what
the new dependencies change, so a restored state can't go stale.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kbf3gKHFTuMrZosy3Nb5Qc
Signed-off-by: Christoph Knittel <ck@cca.io>
@cknitt cknitt changed the title Reuse CI caches across dependency changes and delete superseded ones Restore earlier CI caches when the dependencies change Nov 10, 2026
cknitt and others added 2 commits November 10, 2026 18:07
Review fixes for the pull request fallback. Pins of earlier dependencies
are dropped before the install, which pins what the opam files still
pin; the OCaml 5.0 job never restores an earlier environment, so a
dependency change still gets one fresh resolution before the merge; and
Windows installs read the bash_env that ocaml/setup-ocaml writes. The
depext bypass now applies to every install, and only environments
resolved from scratch are saved. The build state keys no longer end in a
hash of the opam files, so master keeps one state per job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kbf3gKHFTuMrZosy3Nb5Qc
Signed-off-by: Christoph Knittel <ck@cca.io>
Dropped before merging; it makes this pull request's CI take the fallback path.

Signed-off-by: Christoph Knittel <ck@cca.io>
@cknitt
cknitt force-pushed the ci-cache-generations branch from 2b2f393 to 449b4bd Compare November 10, 2026 18:07
It made this pull request's CI take the fallback path: the restored jobs
set up OPAM in 35 to 61 seconds, the OCaml 5.0 job resolved from scratch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Kbf3gKHFTuMrZosy3Nb5Qc
Signed-off-by: Christoph Knittel <ck@cca.io>
@cknitt

cknitt commented Nov 10, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: af15778cc2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog:skip No release notes needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant