test(cache): make stalled-remote-cache a proxy for the remote cache - #795
Merged
Merged
Conversation
vtt stalled-remote-cache used to stall every request to an endpoint of its own. It is now a proxy for the endpoint in VP_REMOTE_CACHE_URL: requests to the routes given with --stall are never answered, and the rest are forwarded, so it can run in front of remote-cache-server or any other backend. ctrl_c_during_fetch and fast_fail_during_fetch stall /fetch with nothing behind the proxy. The new ctrl_c_during_upload case stalls /store in front of the backend, which needs remote-cache-server to leave Ctrl-C to its command. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
wan9chi
added this pull request to stack #796
October 4, 2026 05:12
fspy benchmarklinuxmacoswindows |
wan9chi
added a commit
that referenced
this pull request
Oct 4, 2026
… HTTP/2 (#787) ## Motivation In `read-write` mode, a task waited for its upload to the remote cache before it counted as finished, so every task that depended on it waited too. A slow remote cache slowed down the whole run, even though nothing in the run needed the upload. Once uploads run side by side, HTTP/1.1 also opens a connection for each one; HTTP/2 lets them share one. ## Changes - **Background uploads.** Once a task's result is cached locally, its upload starts in the background and the task finishes right away. When the graph is done, `vp run` prints `Waiting for N remote cache uploads to finish (Ctrl-C to cancel)...` and waits for them before the summary. An upload's error is set later, through the `Arc<OnceLock<UploadError>>` in `CacheUpdateStatus::Updated`, and the summary reads it after the wait. - **Cancellation.** A new interrupt token, which only Ctrl-C cancels, cancels the uploads. Their entries stay in the local cache, and the summary says they weren't uploaded because they were interrupted. Fast-fail still stops lookups but no longer stops uploads, so tasks that succeeded before another one failed are still uploaded. `docs/cancellation.md` covers both. - **HTTP/2.** reqwest's `http2` feature is on. HTTPS endpoints use HTTP/2 if the server accepts it in the TLS handshake and HTTP/1.1 otherwise. `http://` endpoints stay on HTTP/1.1. - **Tests.** Whether an upload is still running when the graph is done depends on how fast the remote cache responds. So e2e steps that upload set `VP_RUN_INTERNAL_HIDE_PENDING_UPLOADS`, which keeps the waiting line out of their output, including the first step of `restore_failure` from #770. The cases that show the waiting line or cancel uploads put `vtt stalled-remote-cache --stall /store` from #795 in front of the backend. Their fetches reach the backend and miss, and their uploads never finish. Like the other backend cases, they need Node.js and are skipped on Windows. New e2e cases: `pending_uploads`, `hide_pending_uploads`, and `fast_fail_during_upload`. `ctrl_c_during_upload` from #795 now shows the waiting line. A unit test checks that the client offers `h2` over TLS. ## Notes for reviewers - **Uploads aren't capped.** Each upload in progress holds its encoded entry in memory and its output archive open, plus its own connection on HTTP/1.1, and none of that counts against `--concurrency-limit`. A large graph with a slow endpoint can pile them up. A cap is left for a follow-up. - **No HTTP/3.** reqwest's `http3` feature needs `--cfg reqwest_unstable` in every build, vite-plus included, adds aws-lc-rs next to ring, bypasses proxies, and is used only when the client forces HTTP/3 for every request. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
The e2e cases for a remote cache that never answers need some requests to stall while others behave normally. For example, an upload case needs its fetch to reach a backend and miss before the upload stalls.
vtt stalled-remote-cachewas its own endpoint and stalled every request, so it couldn't do that. #718 will replaceremote-cache-serverwith the real cache service, which can't stall requests itself. So the stalling has to happen in front of whatever backend a test runs.Changes
vtt stalled-remote-cache [--stall ROUTE]... COMMAND [ARGS...]is now a proxy for the endpoint inVP_REMOTE_CACHE_URL, which must behttp://<host>:<port>/<path>. It runs the command withVP_REMOTE_CACHE_URLset to the proxy./store, is never answered, and it emits astalledmilestone.connection: close, so each one comes in on its own connection and is checked on its own.ctrl_c_during_fetchandfast_fail_during_fetchstall/fetch, with an unreachable endpoint behind the proxy. They still run on every platform without Node.js.ctrl_c_during_uploadcase. It runsremote-cache-server vtt stalled-remote-cache --stall /store vt run build. The fetch reaches the backend and misses, the upload stalls, and Ctrl-C cancels it.remote-cache-servernow ignores Ctrl-C and leaves it to its command, so it still prints its request log afterwards.Notes for reviewers
remote-cache-serverare skipped on Windows anyway.packages/tools/src/remote-cache/cli.ts, so the line that ignores Ctrl-C needs to move into its version.🤖 Generated with Claude Code