fix(ci): add matrix sharding to import profiler workflow - #17779
fix(ci): add matrix sharding to import profiler workflow#17779hebaalazzeh wants to merge 6 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request increases the --fail-threshold value from 5000 to 120000 in ci/run_single_test.sh when executing the profiler script. The feedback suggests double-quoting the ${PROFILER_SCRIPT} and ${PACKAGE_NAME} variables to prevent potential word splitting and globbing issues.
2e75bfa to
0b2d9fd
Compare
37448ee to
99f6a31
Compare
99f6a31 to
c13a5e7
Compare
parthea
left a comment
There was a problem hiding this comment.
Can you create an example PR which includes changes to packages so we can see the results? Right now we only see No packages assigned to Shard 0. which is expected, but it would be great to see an example
see: #17936 |
|
In the demo PR, I see these errors. If that's expected, can we make the error more user friendly? https://github.com/googleapis/google-cloud-python/actions/runs/30484508360/job/90686692668?pr=17936 |
Overview
This PR implements static matrix sharding for the
import-profilerworkflow to run modified packages in parallel across 8 shards, speeding up CI execution time on multi-package PRs.Key Changes
.github/workflows/import-profiler.ymlto split modified packages underpackages/across 8 GitHub Actions runner matrix shards using modulo logic (idx % TOTAL_SHARDS).all-import-profilesaggregation job that depends on the matrix run (needs: import-profile) to serve as the single required GitHub status check for branch protection and merge queue rules.