Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion crates/socket-patch-cli/CLI_CONTRACT.md
Original file line number Diff line number Diff line change
Expand Up @@ -979,7 +979,7 @@ v5.0 replaces v4's per-purl reverts and whole-ledger reverse replay (`revert_rem
| Key | Shape | Meaning |
|---|---|---|
| `error` | `{code, message}` | Only on `status: "error"` (v5.0, MAJOR: was a string). Codes: `manifest_not_found`, `manifest_invalid`, `manifest_unreadable`, `patch_not_found`, `path_glob_no_match`, `hosted_wiring_contested`, `vendor_ledger_missing`, `rollback_failed`, `lock_held` / `lock_io`, and `path_glob_invalid` (a usage error, exit 2). Per-result `results[*].error` stays a string. |
| `warnings` | `[{code, detail}]` | Run-level warnings, now populated (previously always empty): `reinstall_required`, `hosted_state_not_preservable`, `out_of_scope_copies_restored`, `vendor_state_unreadable`, `cleanup_failed`, `manifest_write_failed`, `legacy_redirect_ledger_kept`, the upstream-restore advisories (`npm_allow_remote_left`, `pnpm_trust_lockfile_left`, `maven_trusted_checksums_left`, `nuget_default_config_left`, `upstream_uv_override_removed`, `upstream_registry_fallback`, `upstream_pnpm_tarball_setting_guessed`, `upstream_gem_stale_cache`), `ownership_not_restored` (a restored file whose ownership could not be put back — see the apply warnings), `rollback_record_superseded` (a manifest record superseded by a live hosted pin, left to the hosted leg — see Manifest cleanup), plus vendored/hosted leg advisories. New codes are additive (MINOR) |
| `warnings` | `[{code, detail}]` | Run-level warnings, now populated (previously always empty): `reinstall_required`, `hosted_state_not_preservable`, `out_of_scope_copies_restored`, `vendor_state_unreadable`, `cleanup_failed`, `manifest_write_failed`, `legacy_redirect_ledger_kept`, the upstream-restore advisories (`npm_allow_remote_left`, `pnpm_trust_lockfile_left`, `maven_trusted_checksums_left`, `nuget_default_config_left`, `upstream_uv_override_removed`, `upstream_registry_fallback`, `upstream_pnpm_tarball_setting_guessed`, `upstream_gem_stale_cache`, `yarn_berry_node_gyp_unresolved`), `ownership_not_restored` (a restored file whose ownership could not be put back — see the apply warnings), `rollback_record_superseded` (a manifest record superseded by a live hosted pin, left to the hosted leg — see Manifest cleanup), plus vendored/hosted leg advisories. New codes are additive (MINOR) |
| `vendored` | `[purl]` | **Meaning narrowed (MAJOR)**: vendor-owned purls the run did NOT act on — today exactly the corrupt-vendor-ledger skip. |
| `vendoredReverted` | `[purl]` | Ledger entries cleanly reverted this run (unwired + artifact deleted + entry dropped; previewed on dry-run) |
| `vendoredPreserved` | `[purl]` | `--preserve-state`: unwired with artifact + ledger entry kept |
Expand Down Expand Up @@ -1284,6 +1284,7 @@ Every `--json` invocation emits a single JSON object that follows the **unified
| `upstream_registry_fallback` | rollback/remove `warnings[]`; vendor advisory event (takeover) | upstream restore: a yarn berry, pnpm or vlt entry is restored from the version document of the registry the project resolves it against (`.yarnrc.yml` `npmRegistryServer`, the pnpm lock's sibling `.npmrc` `registry` / `@scope:registry` or pnpm-workspace.yaml `registry` / `registries`, vlt's node registry); that registry could not be read (e.g. it needs credentials), so the default registry's document was used and the restored tarball URL may not be the mirror's. |
| `upstream_gem_stale_cache` | rollback/remove `warnings[]`; vendor advisory event (takeover) | upstream restore (#1260): a restored gem's `<name>-<version>.gem` is still in Bundler's cache dir (`cache_path`, default `vendor/cache`, resolved as for the Gem stale-install guard) and its sha256 is not the upstream one (the restored `CHECKSUMS` entry, else the rubygems.org compact index), or it could not be checked (`--offline`, a registry error). Bundler installs from that dir first, so a `bundle cache` taken while the hosted pin was live makes every later install fail on the upstream checksum (exit 37) or, on bundler < 2.6 frozen installs, keep installing the patched bytes. The detail names the file. Remedy: delete it, then run `bundle cache` to cache the upstream gem in its place (or `bundle install` if the project does not commit its cache; with the cache dir committed, a frozen install reads only the cache). Read-only: the restore never deletes it. Not raised for an archive whose sha256 matches upstream. |
| `upstream_pnpm_tarball_setting_guessed` | rollback/remove `warnings[]`; vendor advisory event (takeover) | upstream restore (#902): nothing showed which pnpm wrote a hosted `pnpm-lock.yaml` (no unpinned registry entry that shows the setting, no `node_modules/.modules.yaml` install record, no package.json `packageManager` pin; for a Rush lock, no rush.json `pnpmVersion`), so its entries were restored with or without `tarball:` by pnpm 10's reading of `lockfileIncludeTarballUrl` (pnpm-workspace.yaml, else `.npmrc` `lockfile-include-tarball-url`), and pnpm 9 (which reads only `.npmrc`) or pnpm >= 11 (which reads only pnpm-workspace.yaml) would have read it the other way. The detail names the lock, the setting followed, the pnpm that disagrees and the entries. Remedy: pin the pnpm (package.json `packageManager`, or reinstall so the install record names it; rush.json `pnpmVersion` for Rush), or give the two files the same value so every pnpm reads it alike; a rollback or remove can then be redone by restoring the lock from version control and re-running. Not raised when evidence decided, when both files read the same on every pnpm, or when the tarball is one pnpm records regardless. |
| `yarn_berry_node_gyp_unresolved` | rollback/remove `warnings[]`; vendor advisory event (takeover) | upstream restore (#737): yarn's npm resolver gives the restored registry entry an implicit `node-gyp: "npm:latest"` dependency (the version document runs `node-gyp` in a script without declaring it), which the hosted pin dropped because yarn never gives a tarball-locator entry one, and the lock no longer has a `node-gyp@npm:latest` entry (the pin dropped the subtree only node-gyp reached). The entry is restored without it; a plain `yarn install --immutable` accepts that, a hardened or `--refresh-lockfile` install reports the lock modified. Remedy: run `yarn install` once. Not raised while another entry keeps `node-gyp@npm:latest` in the lock: the dependency is then restored too. |
| `legacy_redirect_ledger_kept` | rollback `warnings[]` (+ remove stderr) | v5.0: a pre-v5 `.socket/vendor/redirect-state.json` could not be deleted once no hosted pin was left; the file is inert (never read for planning). Never flips the exit. |
| `vendor_stale_artifact_removed` | `removed` | vendor / scan `--mode vendored`: re-vendor under a newer patch uuid removed the previous uuid's orphaned artifact dir. |
| `vendor_unsupported_ecosystem` | `skipped` | vendor: no vendor backend for this purl's ecosystem (jsr). |
Expand Down Expand Up @@ -1380,6 +1381,7 @@ Every `--json` invocation emits a single JSON object that follows the **unified
| `pypi_hatch_stale_install` | `skipped` (warning) | vendor (hatch): an existing Hatch environment of the project (found under Hatch's data dir, `dirs.env.virtual` or an explicit env `path`) still holds the upstream release; Hatch keeps it on the next `hatch run`, so the detail names the env and the `hatch env remove <env>` / `hatch env prune` remedy. |
| `pypi_pipenv_installer_unknown` | `skipped` (warning) | vendor (pipenv): no `pipenv` answered on PATH; the vendored references assume Pipenv 2018 or later (7–11 cannot consume them — use hosted mode there); `SOCKET_PIPENV_MAJOR` pins the release. |
| `vendor_lock_entry_relocked` | revert `warnings[]` | vendor `--revert` / rollback (pipenv): a relock regenerated the wired entry to a registry reference, or removed it; the record is retired (artifact removed, ledger entry dropped) instead of drift-kept. |
| `vendor_lock_entry_pruned_kept` | revert `warnings[]` | vendor `--revert` / rollback (yarn berry, #737): vendoring dropped a lock entry only the patched package's implicit `node-gyp` dependency reached (or one descriptor of a shared entry), and the lock has re-added or changed that entry since; it is left as it is (not drift: the artifact is still removed). Run `yarn install` if the lock is incomplete. |
| `pypi_{poetry,pdm,pipenv}_no_lockfile` | `failed` | vendor (pypi): a lock-less tool marker with no `requirements.txt` fallback — run `<tool> lock`. |
| `pypi_poetry_integrity_unverified` | `skipped` (warning) | vendor (pypi / poetry): the lock was written by Poetry < 1.4 (0.12 `[metadata.hashes]`, lock 1.0/1.1, or a 2.0 lock without a `@generated by Poetry X.Y.Z` header — 1.3 wrote those). That installer does not verify local wheel hashes (the committed wheel bytes are the protection) and does not replace an already-installed package at the same version; recreate the virtualenv or `pip uninstall` the package before `poetry install`, or upgrade Poetry. |
| `redirect_poetry_stale_install_risk` | `redirect.warnings[]` (warning) | scan `--mode hosted` (poetry): same writer test as above — a warm virtualenv keeps the upstream package after the redirect on Poetry < 1.4 (1.4+ re-installs from the new source); fresh installs pick up the patched wheel. Emitted once per rewritten lock, only on the run that rewrites it. |
Expand Down
141 changes: 140 additions & 1 deletion crates/socket-patch-cli/tests/in_process_redirect.rs
Original file line number Diff line number Diff line change
Expand Up @@ -631,6 +631,11 @@ async fn mock_reference_with_berry(server: &MockServer) {
}

async fn mock_reference_with_berry_url(server: &MockServer, hosted_url: &str) {
mock_reference_with_berry_sha512(server, hosted_url, PATCHED_SHA512).await;
}

/// [`mock_reference_with_berry_url`] granting a tarball of `sha512`.
async fn mock_reference_with_berry_sha512(server: &MockServer, hosted_url: &str, sha512: &str) {
Mock::given(method("POST"))
.and(path(format!("/v0/orgs/{ORG}/patches/package")))
.respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({
Expand All @@ -641,7 +646,7 @@ async fn mock_reference_with_berry_url(server: &MockServer, hosted_url: &str) {
"purl": PURL,
"artifacts": [
{ "kind": "tarball", "url": hosted_url,
"integrity": { "sha512": PATCHED_SHA512 } },
"integrity": { "sha512": sha512 } },
{ "kind": "yarn-berry-zip", "url": "http://patch.test/berry.zip",
"integrity": { "yarnBerry10c0": BERRY_CHECKSUM } }
],
Expand Down Expand Up @@ -861,6 +866,129 @@ async fn scan_redirect_rewrites_crlf_and_bom_yarn_berry_locks_and_rollback_resto
}
}

/// #737: yarn's npm resolver gives a registry entry whose scripts run
/// node-gyp (the registry injects `install: node-gyp rebuild` for any
/// package with a `binding.gyp`) an implicit `node-gyp: "npm:latest"`
/// dependency, which the tarball-locator pin never gets: the pin drops it,
/// and with it every entry only node-gyp reached. Rollback re-resolves the
/// registry entry and puts the dependency back while the lock still
/// resolves node-gyp; once it does not, it warns that `yarn install` must
/// re-add it.
#[tokio::test]
#[serial]
async fn yarn_berry_pin_drops_the_implicit_node_gyp_and_rollback_restores_it() {
let server = MockServer::start().await;
mock_discovery(&server).await;
let hosted_url = HOSTED_URL.replace("http://patch.test", &server.uri());
let tarball = upstream_tarball();
mock_reference_with_berry_sha512(
&server,
&hosted_url,
&vlt_hosted_common::sha512_sri(&tarball),
)
.await;
mock_view(&server).await;
// The served tarball's own manifest (the pin reads it): no node-gyp.
Mock::given(method("GET"))
.and(path(hosted_url.trim_start_matches(&server.uri())))
.respond_with(ResponseTemplate::new(200).set_body_bytes(tarball.clone()))
.mount(&server)
.await;
mock_npm_registry_scripts(
&server,
&vlt_hosted_common::sha512_sri(&tarball),
Some(tarball),
serde_json::json!({ "install": "node-gyp rebuild" }),
)
.await;

let node_gyp = "\"node-gyp@npm:latest\":\n version: 13.1.0\n \
resolution: \"node-gyp@npm:13.1.0\"\n dependencies:\n \
nopt: \"npm:^10.0.0\"\n checksum: 10c0/aa\n languageName: node\n \
linkType: hard\n\n\"nopt@npm:^10.0.0\":\n version: 10.0.1\n \
resolution: \"nopt@npm:10.0.1\"\n checksum: 10c0/bb\n \
languageName: node\n linkType: hard\n";
let other_gyp = "\n\"other-gyp@npm:1.0.0\":\n version: 1.0.0\n \
resolution: \"other-gyp@npm:1.0.0\"\n dependencies:\n \
node-gyp: \"npm:latest\"\n checksum: 10c0/cc\n \
languageName: node\n linkType: hard\n";
for shared in [true, false] {
let label = if shared { "shared" } else { "sole" };
let tmp = tempfile::tempdir().unwrap();
write_berry_project_spelled(tmp.path(), |t| {
let mut t = t
.replace(
&format!("resolution: \"{NAME}@npm:{VERSION}\"\n"),
&format!(
"resolution: \"{NAME}@npm:{VERSION}\"\n dependencies:\n \
node-gyp: \"npm:latest\"\n"
),
)
.replace(
"\n\"consumer@workspace:.\"",
&format!("\n{node_gyp}\n\"consumer@workspace:.\""),
);
if shared {
t = t.replace(
&format!(" {NAME}: \"npm:^{VERSION}\"\n"),
&format!(" {NAME}: \"npm:^{VERSION}\"\n other-gyp: \"npm:1.0.0\"\n"),
);
t.push_str(other_gyp);
}
t
});
let lock_path = tmp.path().join("yarn.lock");
let pristine = std::fs::read_to_string(&lock_path).unwrap();

let env = run_redirect_subprocess_with(
tmp.path(),
&server.uri(),
&["--patch-server-url", &server.uri()],
);
assert_eq!(env["redirect"]["redirected"], 1, "{label}: {env:#}");
assert!(warning_codes(&env).is_empty(), "{label}: {env:#}");
let lock = std::fs::read_to_string(&lock_path).unwrap();
assert!(
lock.contains(&format!(
"\"{NAME}@{hosted_url}\":\n version: {VERSION}\n \
resolution: \"{NAME}@{hosted_url}\"\n checksum: {BERRY_CHECKSUM}\n"
)),
"{label}: the pin has no node-gyp dependency: {lock}"
);
assert_eq!(
lock.contains("\"node-gyp@npm:latest\":") && lock.contains("\"nopt@npm:^10.0.0\":"),
shared,
"{label}: node-gyp's subtree stays only while another entry reaches it: {lock}"
);

let (code, env) = rollback_json_with_origin(tmp.path(), &server, &server.uri());
assert_eq!(code, Some(0), "{label}: rollback: {env:#}");
let restored = std::fs::read_to_string(&lock_path).unwrap();
let checksum = berry_checksum_of(&restored);
let pristine = pristine.replace(
&format!("10c0/{}", "3".repeat(128)),
&format!("10c0/{checksum}"),
);
let warned = env.to_string().contains("yarn_berry_node_gyp_unresolved");
if shared {
assert_eq!(restored, pristine, "{label}: byte-exact rollback");
assert!(!warned, "{label}: {env:#}");
} else {
assert_eq!(
restored,
pristine
.replace(
" dependencies:\n node-gyp: \"npm:latest\"\n checksum: 10c0/",
" checksum: 10c0/"
)
.replace(&format!("{node_gyp}\n"), ""),
"{label}: the registry entry, minus the dependency the lock cannot resolve"
);
assert!(warned, "{label}: {env:#}");
}
}
}

/// #632: a dependency declared through a yarn catalog (`"catalog:"`) is
/// matched by yarn's `resolutions` before the catalog is expanded, so the
/// hosted pin must also route `<name>@catalog:`; `rollback` must drop every
Expand Down Expand Up @@ -1802,12 +1930,23 @@ fn hosted_unwind_json(
/// `tarball` served at the document's `dist.tarball` (a yarn berry restore
/// downloads it to recompute the zip checksum).
async fn mock_npm_registry(server: &MockServer, integrity: &str, tarball: Option<Vec<u8>>) {
mock_npm_registry_scripts(server, integrity, tarball, serde_json::json!({})).await;
}

/// [`mock_npm_registry`] whose version document carries `scripts`.
async fn mock_npm_registry_scripts(
server: &MockServer,
integrity: &str,
tarball: Option<Vec<u8>>,
scripts: serde_json::Value,
) {
let tarball_path = format!("/npm-registry/{NAME}/-/{NAME}-{VERSION}.tgz");
Mock::given(method("GET"))
.and(path(format!("/npm-registry/{NAME}/{VERSION}")))
.respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({
"name": NAME,
"version": VERSION,
"scripts": scripts,
"dist": {
"tarball": format!("{}{tarball_path}", server.uri()),
"integrity": integrity,
Expand Down
Loading
Loading