Consolidate caches #11354
Merged
Consolidate caches #11354
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We are over 10GB allowed cache space. Hence, we need a bit of more improved cache handling. Changes: * disabled cache maintenance on Mimir (if this works, we will end up with one cca 400MB cache) * implement "rolling caches", we always get miss and older cache, and will save newer cache * initial-build gets `master-XXX` cache, uses it, uploads it as `cache-initial-build` artifact * full-builds gets `master-XXX` cache, uses it, uploads it as `cache-full-build-XXX` artifact * integration-tests get `master-XXX` cache, uses it, uploads as `cache-integration-tests-XXX` artifact * introduce last "consolidate" job (depends on all these before): it downloads all `cache-*` and saves to cache as `master-XXX`
To make it work on most special OS on the world.
gnodet
approved these changes
Nov 2, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem: Our caches (all of them) are over 10GB allowed cache space. Due this, GH is dropping caches and our jobs end up exposed to HTTP 500 that since migration Central happens quite often. We used the plain
actions/cacheto store Mimir caches for each node (200-400 MB depending on which node we talk about), and we have 3 active branches (3.9, 4.0 and master), that simply totals out the 10 GB limit.This PR makes we have one "cache blob" per OS, so each OS has one cache blob (times three, for 3.9, 4.0 and master).
Changes: