ci: Remove benchmark workflow that comments on PRs - #5972
Draft
runningcode wants to merge 1 commit into
Draft
Conversation
The `app-metrics` job in integration-tests-benchmarks.yml ran the plain and Sentry test apps through SauceLabs on every PR and posted a "Performance metrics" comment with startup-time and size diffs. The numbers were not actionable: recent runs reported 0 B for both app sizes, and the startup diff swung between 22 ms and 143 ms across unrelated changes. Removing the whole workflow also drops its SauceLabs `test` job, which leaves several things with no remaining caller. Removed here as well: - `test-app-plain` and `test-app-sentry`, built only by `app-metrics` - `metrics-test.yml`, the app-metrics test specification - `.sauce/sentry-uitest-android-benchmark.yml` and its `-lite` variant, passed to saucectl only by the `test` job - the `make assembleBenchmarks` alias, called only by the `test` job The `sentry-uitest-android-benchmark` module itself is kept. It no longer runs in CI, but it still builds and can be run locally.
📲 Install BuildsAndroid
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📜 Description
Deletes
.github/workflows/integration-tests-benchmarks.yml.Its
app-metricsjob was the one posting the "Performance metrics 🚀" comment on every PR, viagetsentry/action-app-sdk-overhead-metrics. Removing the whole file also drops the workflow's other job, the SauceLabstestjob (nightly full benchmark + per-PR lite benchmark).That leaves a handful of things with no remaining caller, so they go too:
sentry-android-integration-tests/test-app-plainapp-metricssentry-android-integration-tests/test-app-sentryapp-metricssentry-android-integration-tests/metrics-test.ymlapp-metrics.sauce/sentry-uitest-android-benchmark.ymltestjob.sauce/sentry-uitest-android-benchmark-lite.ymltestjobmake assembleBenchmarkstestjobPlus the corresponding
settings.gradle.kts/build.gradle.ktsmodule entries and thesentry-android-integration-tests/README.mdlinks.Deliberately kept: the
sentry-uitest-android-benchmarkmodule and themake assembleBenchmarkTestReleasetarget. The module no longer runs in CI, but it still builds and can be run locally — deleting it felt like a separate call. Happy to fold that in if you'd rather it go now.💡 Motivation and Context
The PR comment wasn't actionable. Recent runs reported 0 B for both app sizes, and the startup-time diff swung between 22 ms and 143 ms across unrelated changes — wide enough that a real regression would be invisible in it.
💚 How did you test it?
./gradlew projectsand./gradlew spotlessApply apiDumpboth succeed; the removed modules are gone from the project list and no.apifiles changed. Grepped the tree for references to every deleted path — none remain.📝 Checklist
sendDefaultPIIis enabled.🔮 Next steps
Decide whether
sentry-uitest-android-benchmarkshould also be removed, or re-wired into CI under a workflow that doesn't comment on PRs.#skip-changelog