ci: allow-list completeness lint + two-phase service startup

[tomassrnka]

Two follow-ups from the #3479 speedup thread, one commit each.

1. Lint the missing direction of the compression allow-list

select-tests.sh rejects stale entries, so compression-tests.tsv "cannot rot into silently reduced coverage" — but only in one direction. Nothing caught a new snapshot-path test that nobody adds to the list: it silently never runs under zstd1/lz4.

check-allowlist-completeness.sh (wired into make check-tests-allowlist, which CI already runs) closes that hole:

• Attributes snapshot-lifecycle symbol hits (pause/resume/fork/snapshot client calls, WithAutoPause(true), fsfreeze) to the enclosing top-level test.
• Self-maintaining net: a package-local helper that touches those symbols must itself be added to the symbol list, so tests hiding behind wrappers stay visible — this discovered pauseSandbox, pauseFilesystemOnly, createSnapshotTemplate{,WithCleanup}, startSnapshotInBackground on the first run.
• Each flagged test must be in the tsv or carry // compression-tests:excluded <reason>; a test that is both listed and excluded is an error.
• Packages the tsv covers with * are exempt (already rot-proof).

Running it on main proved the rot is real. Six tests on the snapshot path were missing from the list and are added here: TestSandboxFork_CrossTeamAccess, TestSandboxResume_CrossTeamAccess_{Paused,Running}, TestSandboxConnect, TestSandboxConnect_CrossTeamAccess_Paused (following the list's own precedent of including lifecycle error/authz paths — cf. the already-listed TestSandboxFork_NotFound), and envd's TestCACertTrustedAfterFilesystemOnlyReboot (its rotation twin was listed). Fifteen fixture-only tests (list/detail/kill/timeout/egress surfaces) got exclusion markers with reasons, per the tsv's own criterion.

Grey calls worth a reviewer's eye: the two TestEgressFirewall*PersistsAfterResume tests are marked excluded because the tsv explicitly names egress as an excluded subject — but "persists after resume" is arguably snapshot-content integrity; flip them to listed if you read it the other way.

2. Two-phase service startup

start-services booted otel → orchestrator → API → client-proxy as four serial start+health-wait chains. Now: otel ∥ orchestrator, then API ∥ client-proxy — preserving the orchestrator-healthy-before-API ordering, while the collector (whose consumers' OTLP exporters retry anyway) and the proxy boot in parallel with their phase-mates.

start-service.sh gains start/wait subcommands (the legacy 5-arg form still works) and now prints the service log tail when a health wait times out — previously a timeout produced zero diagnostics.

Expected effect: ~20–30s off every integration shard's Start Services step; the shards' timing in this PR's runs vs. #3479's baseline (services step 0.7–0.8m) gives the measured answer.

🤖 Generated with Claude Code

[cursor]

PR Summary

Low Risk
Changes are limited to CI startup orchestration and integration-test selection/linting; service ordering constraints are preserved and there is no production runtime or auth logic change.

Overview
CI integration jobs now start otel-collector and orchestrator in parallel, then API and client-proxy in parallel after orchestrator is healthy, instead of four serial start-and-wait chains; start-service.sh supports separate start and wait steps and dumps log tails when a health check times out.

Compressed-config test coverage is guarded by check-allowlist-completeness.sh (hooked into check-tests-allowlist): snapshot-path tests must appear in compression-tests.tsv or declare // compression-tests:excluded. The PR adds previously missing lifecycle/auth tests to the list and marks API/egress/list/detail tests that only use pause/resume as fixtures.

^{Reviewed by Cursor Bugbot for commit 09b14ca. Bugbot is set up for automated code reviews on this repo. Configure here.}

[codecov]

❌ 2 Tests Failed:

Tests completed	Failed	Passed	Skipped
3735	2	3733	7

View the top 1 failed test(s) by shortest run time

github.com/e2b-dev/infra/tests/integration/internal/tests/api/templates::TestTemplateBuildDistroFamilies/RPM_family

Stack Traces | 300s run time

=== RUN   TestTemplateBuildDistroFamilies/RPM_family
=== PAUSE TestTemplateBuildDistroFamilies/RPM_family
=== CONT  TestTemplateBuildDistroFamilies/RPM_family
    build_template_test.go:159: test-distro-fedora: [info] Building template 3w2ei4zp24bc6xjxtefj/9e9a9670-4dbd-4744-b5d9-b6e614fc8edf
    build_template_test.go:159: test-distro-fedora: [info] [base] FROM fedora:44 [646eafce87bec29b8bf2b1928e907ea79de9773ee574b53ef4f95f062570e3c1]
    build_template_test.go:159: test-distro-fedora: [info] Base Docker image size: 69 MB
    build_template_test.go:159: test-distro-fedora: [info] Creating file system and pulling Docker image
    build_template_test.go:159: test-distro-fedora: [info] Uncompressing layer sha256:4dd84e24b4ebeafd11df963cf2ee4b8544198650dd964a0f008448a2e2664f2b 69 MB
    build_template_test.go:159: test-distro-fedora: [info] Uncompressing layer sha256:c2e637d475f0569cef59da7bf87f9809d1aebd371db004ce49df1cfdb8899977 14 MB
    build_template_test.go:159: test-distro-fedora: [info] Uncompressing layer sha256:e216621d8a0a7a4486d6a228759b29bc378eff95e6ea9469854b8624b267dd13 137 B
    build_template_test.go:159: test-distro-fedora: [info] Layers extracted
    build_template_test.go:159: test-distro-fedora: [info] Root filesystem structure: .profile, afs, bin, boot, dev, etc, home, lib, lib64, media, mnt, opt, proc, root, run, sbin, srv, sys, tmp, usr, var
    build_template_test.go:159: test-distro-fedora: [info] Provisioning sandbox template
    build_template_test.go:159: test-distro-fedora: [info] Provisioning was successful, cleaning up
    build_template_test.go:159: test-distro-fedora: [info] Sandbox template provisioned
    build_template_test.go:159: test-distro-fedora: [info] [base] DEFAULT USER user [0a390e0f27b4c497ee698a5573d4e30bc39caabaecaa08aacdbd0605adb1cf13]
    distro_build_test.go:85: 
        	Error Trace:	.../api/templates/build_template_test.go:124
        	            				.../api/templates/distro_build_test.go:85
        	Error:      	Received unexpected error:
        	            	Get "http://localhost:.../builds/9e9a9670-4dbd-4744-b5d9-b6e614fc8edf/status?logsOffset=13&level=info": context deadline exceeded
        	Test:       	TestTemplateBuildDistroFamilies/RPM_family
--- FAIL: TestTemplateBuildDistroFamilies/RPM_family (300.39s)

View the full list of 1 ❄️ flaky test(s)

github.com/e2b-dev/infra/tests/integration/internal/tests/api/templates::TestTemplateBuildDistroFamilies

Flake rate in main: 18.97% (Passed 94 times, Failed 22 times)

Stack Traces | 0s run time

=== RUN   TestTemplateBuildDistroFamilies
=== PAUSE TestTemplateBuildDistroFamilies
=== CONT  TestTemplateBuildDistroFamilies
--- FAIL: TestTemplateBuildDistroFamilies (0.00s)

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit c72d995. Configure here.}

[tomassrnka]

Measured results (run 30693622934, all green)

Commit 1 (allow-list lint): validated. The lint runs in every integration shard via check-tests-allowlist, passes on the fixed tree, and fails correctly when a marker is removed. One interaction found the hard way: the repo formatter rewrites the marker to // compression-tests:excluded (hyphen ≠ machine-directive pattern), so the lint accepts the formatted form — that's the third commit.

Commit 2 (two-phase startup): no measured wall win — being honest about it. Start Services came in at 40–46s vs 42–48s baseline (~2s, within noise). The step's cost is dominated by DB health-wait + migrations + seed, not the four service boots — prebuilt binaries turn healthy in 1–2 polls, so the serial waits I parallelized were already near-free. What the commit still delivers: the log tail on health-wait timeout (previously a timeout produced zero diagnostics) and the start/wait split for anyone restructuring later. If you'd rather keep the diff minimal, I can drop the two-phase reordering and keep only the script's subcommands + failure diagnostics — say the word.

Unrelated to this PR but visible in the run: shards queued 2.1–6.2 min because three pushes in ~10 min stacked runs against the runner group's 24-slot ceiling (wall 17.3 min despite healthy shard durations). Same story as the #3479 measurements — the max-concurrency raise is worth doing.

🤖 Generated with Claude Code