Skip to content

Commit 304791e

Browse files
cstamascstamas
authored
Consolidate caches (#11354)
Problem: Our caches (all of them) are over 10GB allowed cache space. Due this, GH is dropping caches and our jobs end up exposed to HTTP 500 that since migration Central happens quite often. We used the plain `actions/cache` to store Mimir caches for each node (200-400 MB depending on which node we talk about), and we have 3 active branches (3.9, 4.0 and master), that simply totals out the 10 GB limit. This PR makes we have one "cache blob" per OS, so each OS has one cache blob (times three, for 3.9, 4.0 and master). Changes: * implement "always save" pattern (see cache doco) * we keep cache "per lane" (per OS) * 3 kind of builds (initial, full and integration-tests) all use same (per OS) cache at start and at end uploads cache as artifact (1 day retention) * at end there is a matrix job "consolidate caches" (runs on all 3 OSes) that downloads caches and consolidate them and save cache * hence, we will have 3 OS specific caches
1 parent 0f42f5b commit 304791e

File tree

1 file changed

+86
-27
lines changed

1 file changed

+86
-27
lines changed

‎.github/workflows/maven.yml‎

Lines changed: 86 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -62,10 +62,12 @@ jobs:
6262
6363
- name: Restore Mimir caches
6464
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
65-
id: restore-cache
6665
with:
6766
path: ${{ env.MIMIR_LOCAL }}
68-
key: master-${{ runner.os }}-initial
67+
key: master-${{ runner.os }}-${{ github.run_id }}
68+
restore-keys: |
69+
master-${{ runner.os }}-
70+
master-
6971
7072
- name: Set up Maven
7173
shell: bash
@@ -86,12 +88,13 @@ jobs:
8688
shell: bash
8789
run: ls -la apache-maven/target
8890

89-
- name: Save Mimir caches
90-
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
91-
if: ${{ github.event_name != 'pull_request' && !cancelled() && !failure() }}
91+
- name: Upload Mimir caches
92+
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v4
93+
if: ${{ !cancelled() && !failure() }}
9294
with:
95+
name: cache-${{ runner.os }}-initial
96+
retention-days: 1
9397
path: ${{ env.MIMIR_LOCAL }}
94-
key: ${{ steps.restore-cache.outputs.cache-primary-key }}
9598

9699
- name: Upload Maven distributions
97100
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v4
@@ -103,13 +106,24 @@ jobs:
103106
104107
- name: Upload test artifacts
105108
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v4
106-
if: failure() || cancelled()
109+
if: ${{ failure() || cancelled() }}
107110
with:
108-
name: ${{ github.run_number }}-initial
111+
name: initial-logs
112+
retention-days: 1
109113
path: |
110114
**/target/surefire-reports/*
111115
**/target/java_heapdump.hprof
112116
117+
- name: Upload Mimir logs
118+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
119+
if: always()
120+
with:
121+
name: initial-mimir-logs
122+
include-hidden-files: true
123+
retention-days: 1
124+
path: |
125+
~/.mimir/*.log
126+
113127
full-build:
114128
needs: initial-build
115129
runs-on: ${{ matrix.os }}
@@ -153,13 +167,12 @@ jobs:
153167
154168
- name: Restore Mimir caches
155169
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
156-
id: restore-cache
157170
with:
158171
path: ${{ env.MIMIR_LOCAL }}
159-
key: master-full-${{ matrix.os }}-${{ matrix.java }}
172+
key: master-${{ runner.os }}-${{ github.run_id }}
160173
restore-keys: |
161-
master-full-${{ matrix.os }}-
162-
master-full-
174+
master-${{ runner.os }}-
175+
master-
163176
164177
- name: Download Maven distribution
165178
uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v4
@@ -196,22 +209,34 @@ jobs:
196209
shell: bash
197210
run: mvn site -e -B -V -Preporting
198211

199-
- name: Save Mimir caches
200-
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
201-
if: ${{ github.event_name != 'pull_request' && !cancelled() && !failure() }}
212+
- name: Upload Mimir caches
213+
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v4
214+
if: ${{ !cancelled() && !failure() }}
202215
with:
216+
name: cache-${{ runner.os }}-full-build-${{ matrix.java }}
217+
retention-days: 1
203218
path: ${{ env.MIMIR_LOCAL }}
204-
key: ${{ steps.restore-cache.outputs.cache-primary-key }}
205219

206220
- name: Upload test artifacts
207221
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v4
208222
if: failure() || cancelled()
209223
with:
210-
name: ${{ github.run_number }}-full-build-artifact-${{ runner.os }}-${{ matrix.java }}
224+
name: full-build-logs-${{ runner.os }}-${{ matrix.java }}
225+
retention-days: 1
211226
path: |
212227
**/target/surefire-reports/*
213228
**/target/java_heapdump.hprof
214229
230+
- name: Upload Mimir logs
231+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
232+
if: always()
233+
with:
234+
name: full-build-mimir-logs-${{ runner.os }}-${{ matrix.java }}
235+
include-hidden-files: true
236+
retention-days: 1
237+
path: |
238+
~/.mimir/*.log
239+
215240
integration-tests:
216241
needs: initial-build
217242
runs-on: ${{ matrix.os }}
@@ -243,13 +268,12 @@ jobs:
243268
244269
- name: Restore Mimir caches
245270
uses: actions/cache/restore@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
246-
id: restore-cache
247271
with:
248272
path: ${{ env.MIMIR_LOCAL }}
249-
key: master-its-${{ matrix.os }}-${{ matrix.java }}
273+
key: master-${{ runner.os }}-${{ github.run_id }}
250274
restore-keys: |
251-
master-its-${{ matrix.os }}-
252-
master-its-
275+
master-${{ runner.os }}-
276+
master-
253277
254278
- name: Download Maven distribution
255279
uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v4
@@ -282,20 +306,55 @@ jobs:
282306
shell: bash
283307
run: mvn install -e -B -V -Prun-its,mimir
284308

285-
- name: Save Mimir caches
286-
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
287-
if: ${{ github.event_name != 'pull_request' && !cancelled() && !failure() }}
309+
- name: Upload Mimir caches
310+
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v4
311+
if: ${{ !cancelled() && !failure() }}
288312
with:
313+
name: cache-${{ runner.os }}-integration-tests-${{ matrix.java }}
314+
retention-days: 1
289315
path: ${{ env.MIMIR_LOCAL }}
290-
key: ${{ steps.restore-cache.outputs.cache-primary-key }}
291316

292317
- name: Upload test artifacts
293318
uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v4
294-
if: failure() || cancelled()
319+
if: ${{ failure() || cancelled() }}
295320
with:
296-
name: ${{ github.run_number }}-integration-test-artifact-${{ runner.os }}-${{ matrix.java }}
321+
name: integration-test-logs-${{ runner.os }}-${{ matrix.java }}
322+
retention-days: 1
297323
path: |
298324
**/target/surefire-reports/*
299325
**/target/failsafe-reports/*
300326
./its/core-it-suite/target/test-classes/**
301327
**/target/java_heapdump.hprof
328+
329+
- name: Upload Mimir logs
330+
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4
331+
if: always()
332+
with:
333+
name: integration-test-mimir-logs-${{ runner.os }}-${{ matrix.java }}
334+
include-hidden-files: true
335+
retention-days: 1
336+
path: |
337+
~/.mimir/*.log
338+
339+
consolidate-caches:
340+
runs-on: ${{ matrix.os }}
341+
strategy:
342+
fail-fast: false
343+
matrix:
344+
os: [ubuntu-latest, macos-latest, windows-latest]
345+
needs:
346+
- full-build
347+
- integration-tests
348+
steps:
349+
- name: Download Caches
350+
uses: actions/download-artifact@018cc2cf5baa6db3ef3c5f8a56943fffe632ef53 # v4
351+
with:
352+
merge-multiple: true
353+
pattern: 'cache-${{ runner.os }}*'
354+
path: ${{ env.MIMIR_LOCAL }}
355+
- name: Publish cache
356+
uses: actions/cache/save@0057852bfaa89a56745cba8c7296529d2fc39830 # v4
357+
if: ${{ github.event_name != 'pull_request' && !cancelled() && !failure() }}
358+
with:
359+
path: ${{ env.MIMIR_LOCAL }}
360+
key: master-${{ runner.os }}-${{ github.run_id }}

0 commit comments

Comments
 (0)