feat(cache): authenticate remote cache uploads with GitHub Actions OIDC

CHANGELOG.md

@@ -4,7 +4,7 @@
 - **Changed** The run summary now says a task that wrote a file it also read was `not cached because it modified its inputs`, and the statistics in `vp run --verbose` and `vp run --last-details` use the singular for a count of one, e.g. `1 task • 1 cache miss` ([#783](https://github.com/voidzero-dev/vite-task/pull/783)).
 - **Fixed** An invalid glob in `--filter` no longer shows its error message twice ([#763](https://github.com/voidzero-dev/vite-task/pull/763)).
 - **Changed** The detailed summary from `vp run --verbose` and `vp run --last-details` now shows each underlying cause of an error on its own line ([#761](https://github.com/voidzero-dev/vite-task/pull/761)).
-- **Added** Remote caching. Configure an endpoint with the workspace's `cache: { remote: { url } }` or `VP_REMOTE_CACHE_URL`, and choose access with `--remote-cache=off|read|read-write` or `VP_REMOTE_CACHE`. The default is `read` with an endpoint and `off` without one. After a local cache miss, `vp run` looks the task up in the remote cache and, on a hit, restores its outputs and caches it locally. The task output and the run summary show which hits came from the remote cache. A failed read is just a cache miss, with the failure as its reason. In `read-write` mode, `vp run` also uploads the results of successful, cacheable tasks after caching them locally. Uploads run in the background, so tasks that depend on an uploading task don't wait for it; once all tasks are done, `vp run` waits for the uploads still running and says so. A failed upload doesn't fail the task; the run summary shows a warning instead. Ctrl-C, or a failing task, stops remote cache lookups right away, and a task still being looked up doesn't start. Ctrl-C also cancels the uploads, but a failing task doesn't. Tasks can opt out with `cache: { remote: false }`. Requests to an HTTPS endpoint use HTTP/2 if the server supports it. Requests use the proxy environment variables or, on macOS and Windows, the system proxy settings ([#727](https://github.com/voidzero-dev/vite-task/pull/727), [#755](https://github.com/voidzero-dev/vite-task/pull/755), [#756](https://github.com/voidzero-dev/vite-task/pull/756), [#757](https://github.com/voidzero-dev/vite-task/pull/757), [#764](https://github.com/voidzero-dev/vite-task/pull/764), [#770](https://github.com/voidzero-dev/vite-task/pull/770), [#771](https://github.com/voidzero-dev/vite-task/pull/771), [#772](https://github.com/voidzero-dev/vite-task/pull/772), [#786](https://github.com/voidzero-dev/vite-task/pull/786), [#787](https://github.com/voidzero-dev/vite-task/pull/787)).
+- **Added** Remote caching. Configure an endpoint with the workspace's `cache: { remote: { url } }` or `VP_REMOTE_CACHE_URL`, and choose access with `--remote-cache=off|read|read-write` or `VP_REMOTE_CACHE`. The default is `read` with an endpoint and `off` without one. After a local cache miss, `vp run` looks the task up in the remote cache and, on a hit, restores its outputs and caches it locally. The task output and the run summary show which hits came from the remote cache. A failed read is just a cache miss, with the failure as its reason. In `read-write` mode, `vp run` also uploads the results of successful, cacheable tasks after caching them locally. Uploads run in the background, so tasks that depend on an uploading task don't wait for it; once all tasks are done, `vp run` waits for the uploads still running and says so. A failed upload doesn't fail the task; the run summary shows a warning instead. Ctrl-C, or a failing task, stops remote cache lookups right away, and a task still being looked up doesn't start. Ctrl-C also cancels the uploads, but a failing task doesn't. Tasks can opt out with `cache: { remote: false }`. Requests to an HTTPS endpoint use HTTP/2 if the server supports it. Requests use the proxy environment variables or, on macOS and Windows, the system proxy settings. In a GitHub Actions job with `permissions: id-token: write`, uploads authenticate with a GitHub Actions OIDC token whose audience is the endpoint without a trailing slash. If no token can be obtained, each upload fails with a warning instead ([#727](https://github.com/voidzero-dev/vite-task/pull/727), [#755](https://github.com/voidzero-dev/vite-task/pull/755), [#756](https://github.com/voidzero-dev/vite-task/pull/756), [#757](https://github.com/voidzero-dev/vite-task/pull/757), [#764](https://github.com/voidzero-dev/vite-task/pull/764), [#770](https://github.com/voidzero-dev/vite-task/pull/770), [#771](https://github.com/voidzero-dev/vite-task/pull/771), [#772](https://github.com/voidzero-dev/vite-task/pull/772), [#786](https://github.com/voidzero-dev/vite-task/pull/786), [#787](https://github.com/voidzero-dev/vite-task/pull/787), [#798](https://github.com/voidzero-dev/vite-task/pull/798)).
 - **Fixed** On Windows, environment variable names used by `vp run` now match regardless of ASCII letter case. Assignments in task commands override earlier assignments and inherited variables spelled differently, and `FORCE_COLOR`, `VP_RUN_CONCURRENCY_LIMIT`, and variables requested through `@voidzero-dev/vite-task-client` are found under any spelling ([#747](https://github.com/voidzero-dev/vite-task/pull/747)).
 - **Changed** A task's cache settings now go inside `cache`, e.g. `cache: { env: ["NODE_ENV"], input: ["src/**"] }`; `cache: true` is the same as `cache: {}`. `env`, `untrackedEnv`, `input`, and `output` are no longer supported at the top level of a task ([#749](https://github.com/voidzero-dev/vite-task/pull/749)).
 - **Fixed** Cached tasks on macOS no longer intermittently fail with exit 2 and `oils I/O error (main): No such process` when a fast command finishes before the shell gets scheduled. The bundled shell that runs task commands is updated to Oils 0.38.0, which fixes this race ([#702](https://github.com/voidzero-dev/vite-task/issues/702), [#703](https://github.com/voidzero-dev/vite-task/pull/703)).

crates/vt/src/session/cache/remote.rs

@@ -30,7 +30,7 @@ use vt_plan::{
 };
 use vt_remote_cache::{
     Client, Download, Fetched,
-    auth::{Anonymous, Auth},
+    auth::{Anonymous, Auth, GithubOidc},
 };
 use vt_str::Str;
 use wincode::{
@@ -246,6 +246,11 @@ impl RemoteClients {
 fn build_auth(auth: &RemoteCacheAuth) -> Arc<dyn Auth> {
     match auth {
         RemoteCacheAuth::Anonymous => Arc::new(Anonymous),
+        RemoteCacheAuth::GithubOidc(github_oidc) => Arc::new(GithubOidc::new(
+            &github_oidc.request_url,
+            github_oidc.request_token.expose(),
+            &github_oidc.audience,
+        )),
     }
 }
 
@@ -426,7 +431,7 @@ mod tests {
     use vt_path::{AbsolutePathBuf, RelativePathBuf};
     use vt_plan::{
         cache_metadata::{EnvValueHash, SpawnFingerprint},
-        remote_cache::RemoteCacheAccess,
+        remote_cache::{GithubOidcAuth, RemoteCacheAccess, Secret},
     };
 
     use super::*;
@@ -765,12 +770,22 @@ mod tests {
             serve_stalled(b"HTTP/1.1 500 Internal Server Error\r\ncontent-length: 0\r\n\r\n");
         // Nothing can listen on port 0.
         let unreachable = anonymous("http://127.0.0.1:0/projects/test");
+        let github_oidc_unavailable = ResolvedRemoteCacheConfig {
+            auth: RemoteCacheAuth::GithubOidc(GithubOidcAuth {
+                request_url: Arc::from("http://127.0.0.1:0/token"),
+                request_token: Secret::new(Arc::from("request-token")),
+                audience: Arc::from("http://127.0.0.1:0/projects/test"),
+            }),
+            ..unreachable.clone()
+        };
         let clients = RemoteClients::default();
         let uploads = RemoteUploads::default();
 
-        for (remote_config, message) in
-            [(error_status, "HTTP status 500"), (unreachable, "network error")]
-        {
+        for (remote_config, message) in [
+            (error_status, "HTTP status 500"),
+            (unreachable, "network error"),
+            (github_oidc_unavailable, "failed to authenticate"),
+        ] {
             let error = Arc::new(OnceLock::new());
             uploads.spawn(prepare_upload(&clients, &remote_config).unwrap(), Arc::clone(&error));
             uploads.wait(&CancellationToken::new()).await;

crates/vt_bin/tests/e2e_snapshots/fixtures/remote_cache/snapshots.toml

@@ -443,6 +443,42 @@ steps = [
   ], comment = "The details include the underlying error." },
 ]
 
+[[e2e]]
+name = "github_oidc_unavailable"
+steps = [
+  { argv = [
+    "vt",
+    "run",
+    "build",
+  ], envs = [
+    [
+      "VP_REMOTE_CACHE",
+      "read-write",
+    ],
+    [
+      "VP_REMOTE_CACHE_URL",
+      "http://127.0.0.1:0/projects/test",
+    ],
+    [
+      "ACTIONS_ID_TOKEN_REQUEST_URL",
+      "http://127.0.0.1:0/token?api-version=2.0",
+    ],
+    [
+      "ACTIONS_ID_TOKEN_REQUEST_TOKEN",
+      "request-token",
+    ],
+    [
+      "VP_RUN_INTERNAL_HIDE_PENDING_UPLOADS",
+      "1",
+    ],
+  ], comment = "The job can request GitHub Actions OIDC tokens, so the upload needs one first. Nothing can listen on port 0, so the token request fails, and the upload fails without being sent. The task succeeds." },
+  { argv = [
+    "vt",
+    "run",
+    "--last-details",
+  ], comment = "The details include why the token request failed." },
+]
+
 [[e2e]]
 name = "read_invalid_endpoint"
 steps = [

crates/vt_bin/tests/e2e_snapshots/fixtures/remote_cache/snapshots/github_oidc_unavailable.md

@@ -0,0 +1,43 @@
+# github_oidc_unavailable
+
+## `VP_REMOTE_CACHE=read-write VP_REMOTE_CACHE_URL=http://127.0.0.1:0/projects/test ACTIONS_ID_TOKEN_REQUEST_URL=http://127.0.0.1:0/token?api-version=2.0 ACTIONS_ID_TOKEN_REQUEST_TOKEN=request-token VP_RUN_INTERNAL_HIDE_PENDING_UPLOADS=1 vt run build`
+
+The job can request GitHub Actions OIDC tokens, so the upload needs one first. Nothing can listen on port 0, so the token request fails, and the upload fails without being sent. The task succeeds.
+
+```
+$ vtt write-file dist/output.txt built ○ cache miss: remote cache fetch failed, executing
+
+---
+vt run: remote-cache#build not uploaded to the remote cache: failed to authenticate. (Run `vt run --last-details` for full details)
+```
+
+## `vt run --last-details`
+
+The details include why the token request failed.
+
+```
+
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+    Vite+ Task Runner • Execution Summary
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+
+Statistics:   1 task • 0 cache hits • 1 cache miss
+Performance:  0% cache hit rate
+
+Task Details:
+────────────────────────────────────────────────
+  [1] remote-cache#build: $ vtt write-file dist/output.txt built ✓
+      → Cache miss: remote cache fetch failed
+        ↳ network error
+        ↳ error sending request
+        ↳ client error (Connect)
+        ↳ tcp connect error
+        ↳ <os error>
+      ⚠ Not uploaded to the remote cache: failed to authenticate
+        ↳ GitHub Actions OIDC token request failed
+        ↳ error sending request
+        ↳ client error (Connect)
+        ↳ tcp connect error
+        ↳ <os error>
+━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
+```

crates/vt_graph/run-config.ts

@@ -29,6 +29,10 @@ export type InputBase = "package" | "workspace";
 export type RemoteCacheConfig = {
 /**
  * HTTP or HTTPS namespace endpoint. Overridden by `VP_REMOTE_CACHE_URL`.
+ *
+ * In a GitHub Actions job with `permissions: id-token: write`, uploads
+ * authenticate with a GitHub Actions OIDC token whose audience is the
+ * endpoint without a trailing slash.
  */
 url: string, };
 

crates/vt_graph/src/config/user.rs

@@ -412,6 +412,10 @@ impl ResolvedGlobalCacheConfig {
 #[serde(deny_unknown_fields, rename_all = "camelCase")]
 pub struct UserRemoteCacheConfig {
     /// HTTP or HTTPS namespace endpoint. Overridden by `VP_REMOTE_CACHE_URL`.
+    ///
+    /// In a GitHub Actions job with `permissions: id-token: write`, uploads
+    /// authenticate with a GitHub Actions OIDC token whose audience is the
+    /// endpoint without a trailing slash.
     pub url: Arc<str>,
 }
 

crates/vt_plan/src/error.rs

@@ -184,6 +184,10 @@ pub enum Error {
     #[error("Remote caching requires cache.remote.url or VP_REMOTE_CACHE_URL")]
     MissingRemoteCacheEndpoint,
 
+    /// The value isn't shown, since it can be a credential.
+    #[error("Invalid value for {0}: not valid UTF-8")]
+    NonUtf8RemoteCacheAuthEnv(&'static str),
+
     /// A cycle was detected in the task dependency graph during planning.
     ///
     /// This is caught by `AcyclicGraph::try_from_graph`, which validates that the