ci: bound benchmark simulator setup and cleanup - #1630
Draft
mrousavy wants to merge 1 commit into
Draft
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
mrousavy
force-pushed
the
ci/bound-benchmark-simulator-lifecycle
branch
from
September 7, 2026 19:02
04aed54 to
c7576d2
Compare
Performance Report
iOS
All Benchmarks
Android
All Benchmarks
Benchmarking Code Diff Raw measurements: performance-report-1 (JSON artifact). Run 34154047323, attempt 1. Download requires GitHub access. Android: measurements, attempt 1, apps, attempt 1. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Simulator creation and
simctl bootcan currently hang before the five-minutebootstatustimer starts. Put all three commands in one setup step with a five-minute Actions deadline, and give final simulator cleanup a one-minute deadline. Export the device ID immediately after creation so cleanup can run when boot fails or times out.After successful setup, explicitly mark the newly created CI simulator as fresh. Skip preinstall terminate/uninstall calls for apps that cannot yet exist. Reused simulators keep that cleanup by default; each benchmark case still launches and terminates its own process.
Validation on
c7576d288e314744bd3f52992924fe3a9cf74c2c: all GitHub checks passed, including TypeScript, lint, Nitrogen on both platforms, both native builds and both device measurement jobs. The iOS job completed all 92 measurements, created/booted the simulator in 38 seconds and deleted it in four seconds. The two expected missing-app termination errors seen in unchanged runs are absent.Local tests cover fresh/reused installations, paired and same-SHA runs, invalid freshness flags and per-case process isolation. Workflow checks verify that create/boot/bootstatus share the setup deadline and final cleanup has its own deadline; GitHub Actions enforces the timeouts. TypeScript, actionlint, ShellCheck and diff checks pass. Rebase patch equivalence was verified with
git range-diff.Timing review:
The current run and controls share the same base and iOS runner/toolchain. These are separate hosted runs, and app extraction moved to after boot in this PR. The latest run is slightly shorter, but startup variation prevents attributing a reliable overall speedup to the change. The demonstrated benefits are bounded setup/cleanup and removing unnecessary commands/errors. Leave open under the requested works-and-speeds-up-CI merge condition.