Skip to content

Commit 6fa6a9e

Browse files
Fix vendored npm-family tarballs dropped by .gitignore (#831) (#837)
* Start fix for #831 Assisted-by: Claude Code:claude-opus-5-5 * Keep vendored npm tarballs out of .gitignore Vendoring into a yarn classic, yarn berry, npm, pnpm or bun project wrote .socket/vendor/npm/<uuid>/<pkg>.tgz without checking whether git would commit it. Under Node.gitignore's `*.tgz`, or a `vendor/` or `.socket/` rule, the scan reported success, the commit dropped the tarball, and every fresh checkout's install failed. The shared tarball staging now refuses with vendor_artifact_gitignored before writing anything when a rule ignores the uuid dir itself. After writing, it adds <uuid>/.gitignore (re-including the tarball against rules like `*.tgz`) and .gitattributes, as vlt already does, and checks the written paths again. Refs #831 Assisted-by: Claude Code:claude-opus-5-5 * Fail vendor --check on unledgered lock references When the vendor ledger and manifest were ignored or dropped from a commit, `vendor --check` found nothing to compare and exited 0, while the lockfile still pointed at .socket/vendor/<eco>/<uuid>/ and every fresh install failed. The check now reads the lockfile references (the same scan repair uses) and reports each one no ledger entry owns as vendor_ledger_missing. Refs #831 Assisted-by: Claude Code:claude-opus-5-5 * Document the vendored tarball's uuid .gitignore The contract's vendoring table now says every npm-family tarball flavor writes <uuid>/.gitignore and .gitattributes next to the tarball, and refuses vendor_artifact_gitignored when git would still drop it. Refs #831 Assisted-by: Claude Code:claude-opus-5-5 * Keep patch uuids out of vendor --check messages CodeQL flagged the new unledgered-reference message for printing the patch uuid. The human line and error detail now name only the ecosystem; the JSON event still carries the uuid and path as repair's event does. Refs #831 Assisted-by: Claude Code:claude-opus-5-5 * Port #851: fix vex alias tests broken by the store-copy merge main's #605 made the name-keyed npm resolver reach alias and peer copies itself, which broke two vex_consumed tests that assumed an alias-blind resolver. Same change as #851, ported so this PR's CI runs green against the current base; it no-ops once #851 lands. Refs #831 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FMZyKmgYNAridSR5eqv999 * Refuse a gitignored vendor dir before the hosted takeover The tarball gitignore refusal only ran inside stage_patch_pack, which the hosted->vendored takeover reaches after restore_upstream has already removed the hosted pin. In a hosted project that ignores .socket/, vendoring then restored the registry entry and refused, leaving the package patched in neither mode. The npm takeover preflight now runs the same uuid-dir probe before the restore, as vlt's preflight already does. Refs #831 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FMZyKmgYNAridSR5eqv999 * Port #878: route Gradle digests through utils::digest main's #865 added a test that fails when production code computes digests inline; the Gradle cache, JVM jar and Maven sidecar code landed with inline sha1/sha256 calls, so main's coverage and test-release jobs fail production_digests_go_through_the_helpers. Same change as #878, ported so this PR's CI runs green against the current base; it no-ops once #878 lands. Refs #831 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FMZyKmgYNAridSR5eqv999 --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 7ce869e commit 6fa6a9e

8 files changed

Lines changed: 697 additions & 18 deletions

File tree

‎crates/socket-patch-cli/CLI_CONTRACT.md‎

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -697,7 +697,7 @@ to **six flavors**.
697697

698698
| eco / flavor | vendored artifact | committed wiring | consumption proof |
699699
|---|---|---|---|
700-
| npm (package-lock) | deterministic patched tarball `[@scope/]<name>-<version>.tgz` | `package-lock.json` only (`npm-shrinkwrap.json` wins when present): every entry matching name+version gets `resolved: "file:…"` + recomputed `integrity`. `package.json` untouched | `npm ci` (integrity-verified). Plain `npm install` preserves the entry; `npm update <pkg>` re-resolves and drops it |
700+
| npm (package-lock) | deterministic patched tarball `[@scope/]<name>-<version>.tgz`, plus `<uuid>/.gitignore` (re-includes the tarball against the project's ignores, such as Node.gitignore's `*.tgz`) and `<uuid>/.gitattributes` (`-text`); every tarball flavor below writes the same pair and refuses `vendor_artifact_gitignored` when git would still drop the tarball | `package-lock.json` only (`npm-shrinkwrap.json` wins when present): every entry matching name+version gets `resolved: "file:…"` + recomputed `integrity`. `package.json` untouched | `npm ci` (integrity-verified). Plain `npm install` preserves the entry; `npm update <pkg>` re-resolves and drops it |
701701
| npm / yarn classic | (same tarball) | `yarn.lock` only: matching blocks get `resolved "file:./…#<sha1>"` + `integrity` (both checksums recomputed; merged-key & `npm:`-alias blocks covered) | `yarn install --frozen-lockfile --offline` (sha1 fragment + sha512 SRI both enforced; byte-stable lock) |
702702
| npm / yarn berry (node-modules linker) | (same tarball) | root `package.json` `resolutions` + `yarn.lock` entry with `checksum: 10c0/<sha512>` of the berry cache-zip (reproduced from the tarball offline). **PnP is refused** (`.pnp.*` → different artifact pipeline) | `yarn install --immutable --check-cache`, cold cache. Refused if `__metadata.cacheKey ≠ 10c0` or a non-default `compressionLevel`. Both files keep their own layout — a CRLF lock (yarn's output on Windows) is spliced in CRLF, `package.json` is re-serialized with its BOM, indent, line ending and trailing-newline shape — so vendor + `--revert` round-trip byte-exactly; a lock or `package.json` MIXING CRLF and LF is refused before any write (`vendor_yarn_berry_mixed_line_endings`) |
703703
| npm / pnpm (lockfileVersion 9) | (same tarball) | root `package.json` `pnpm.overrides` (versioned selector) **+** `pnpm-lock.yaml` surgery (overrides / importer version / packages `resolution.integrity` / snapshots) | `pnpm install --frozen-lockfile --offline`, cold store (integrity-verified; byte-stable on pnpm 9 & 10). Other lockfileVersions: 5.4/6.0 route to the legacy backend below; anything else refused |
@@ -1237,7 +1237,7 @@ Every `--json` invocation emits a single JSON object that follows the **unified
12371237
| `already_vendored` | `skipped` | vendor: artifact + wiring already in sync for this patch uuid. |
12381238
| `unsafe_coordinates` | `failed` | vendor: purl/uuid would escape `.socket/vendor/` (tampered manifest/state); refused before any write. |
12391239
| `revert_failed` | `failed` | vendor --revert: a recorded entry could not be reverted. |
1240-
| `vendor_ledger_missing` | `failed` (artifact-level: `uuid` + `details.{ecosystem,path}`, no purl) | repair (v5.0): a lockfile references `.socket/vendor/<eco>/<uuid>/` but the vendor ledger has no entry for it; repair no longer rebuilds ledger entries from lockfiles. Recovery: restore `.socket/vendor/state.json` from version control and re-run `repair`, or `git checkout -- <lockfile>` and re-vendor. |
1240+
| `vendor_ledger_missing` | `failed` (artifact-level: `uuid` + `details.{ecosystem,path}`, no purl) | repair (v5.0) and `vendor --check`: a lockfile references `.socket/vendor/<eco>/<uuid>/` but the vendor ledger has no entry for it; repair no longer rebuilds ledger entries from lockfiles. Recovery: restore `.socket/vendor/state.json` from version control and re-run `repair`, or `git checkout -- <lockfile>` and re-vendor. |
12411241
| `vendor_wiring_unknown_revert_blocked` | `skipped` (beside the `failed`/`revert_failed` event) | vendor --revert: the ledger entry was reconstructed by a pre-v5 `repair` without wiring records and the live lockfile still resolves through the artifact — the revert refuses (fail-closed) instead of deleting a tarball the lock points at. Recovery: `socket-patch repair`, then restore the pre-vendor lock (or re-lock without the override) and re-run the revert. repair: an npm ledger entry whose `flavor` this release does not know (written by a newer socket-patch) is skipped, never health-checked or rebuilt, and the artifact, wiring and ledger stay as found (a lone `skipped` event; the run's exit is unaffected). Recovery: upgrade socket-patch. |
12421242
| `stale_install` | `skipped` | vex (in-run `scan --mode hosted --vex`): a hosted stale-install probe found positively unpatched installed bytes, so the purl is omitted even under `--vex-no-verify` (see the gem / Python stale-install guards). |
12431243
| `record_unavailable` | `skipped` | vex (manifest-less): a lockfile-wired patch has no local record (manifest, this run's hosted records or a pre-v5 redirect ledger, vendor ledger) and none could be fetched — `--offline`, transport error, 404, or a refused (paid) patch. Omitted, never attested from the `socket-patch.vendor.json` marker. |
@@ -1284,8 +1284,8 @@ Every `--json` invocation emits a single JSON object that follows the **unified
12841284
| `vendor_vlt_legacy_lockfile` | `skipped` (warning) | vendor (vlt): an era-A lock (vlt 0.0.0-19 … 1.0.0-rc.8): a `··` default-registry id, or default-registry ids that are URL segments equal to a scalar `options.registry` with no `·npm·` id (era B writes `·npm·` whatever the scalar). vlt 0.0.0-31 … 1.0.0-rc.5 install the vendored lock but fail to reinstall the vendored `file:` dependency if `vlt-lock.json` is deleted and re-created (the other era-A releases reinstall it; the lock does not say which release reads it). The package is still vendored; remedy: upgrade vlt. |
12851285
| `vendor_vlt_reinstall_required` | `skipped` (advisory; human: `Warning: …`) | vendor / scan / get `--mode vendored` (vlt), wet and dry runs, and in-sync reruns: (a) the run rewires an optional dependency, or an importer's `node_modules/<name>` of an optional dependency still resolves into `node_modules/.vlt/`: from vlt 0.0.0-30 a plain `vlt install` (1.2.0: also `--force`) keeps that installed upstream copy linked; the detail says to run `vlt ci` (or delete `node_modules` and run `vlt install`) to link the vendored copy, and that vlt 0.0.0-30 … 1.0.4 install no optional dependency from the lock of a project that declares only optional dependencies (upgrade to 1.0.5 or later first); (b) otherwise, an importer's link of the dependency still resolves into `node_modules/.vlt/`: the detail names the links (`node_modules/<name>`, `<member dir>/node_modules/<name>`) and says `vlt install` (or `vlt ci`) links the vendored copy — on a warm tree after a plain `vlt install` that is true of every vendored direct dependency; (c) an importer's link resolves into the vendored dir of the patch this run replaces (a new patch uuid), which the run removes: the detail names the links and says `vlt install` (or `vlt ci`) links the new vendored copy; (d) a redownload of the payload (vendor, or `repair` after a corrupt or missing payload) could not keep vlt's links to the package's own dependencies (its old `node_modules/` held more than links): the detail says to run `vlt ci` (or delete `node_modules` and run `vlt install`), since a plain `vlt install` does not re-link them. `repair` moves those links back into the downloaded payload when they are only links. The package is vendored either way; a run whose patch fails to apply emits neither. A wet `vendor --revert` (and the revert a vendored → hosted takeover runs, whose advisory joins `redirect.warnings[]`): (a) the revert moves an `optionalDependencies` spec back from the `file:` dir, or an optional importer's `node_modules/<name>` still resolves into the vendored uuid dir: from vlt 0.0.0-30 a plain `vlt install` keeps that link (dangling once the dir is removed), so the detail says to run `vlt ci` (or delete `node_modules` and run `vlt install`) to link the restored copy, with the same vlt 1.0.5 note; (b) otherwise, an importer's link still resolves into the vendored uuid dir: the detail names the links and says `vlt install` (or `vlt ci`) links the restored copy. A dry-run revert emits neither. |
12861286
| `vendor_flavor_changed` | `failed` | vendor (npm): the purl's vendor ledger entry was written for another lockfile `flavor` than the one the router now detects (for example `npm` → `vlt` after switching package managers). Remedy: `socket-patch vendor --revert` it first, then re-vendor. Refused before any write. |
1287-
| `vendor_artifact_gitignored` | `failed` | vendor (vlt): inside a git work tree, `git check-ignore --no-index` reports the new artifact's uuid directory as ignored by a rule its own `.gitignore` cannot override (such as a root `.socket/` rule; the detail names the rule). Remedy: drop that rule for `.socket/vendor/`. Refused before any write. |
1288-
| `vendor_artifact_gitignore_unchecked` | warning | vendor (vlt): git is installed but could not answer the ignore check for the written vendored directory (it failed to start, ran past 30 s, or `rev-parse` / `check-ignore` exited with an error); the package is vendored and the detail names what failed. Remedy: make sure no ignore rule covers `.socket/` before committing. Git absent, or a project outside any work tree, raises nothing. |
1287+
| `vendor_artifact_gitignored` | `failed` | vendor (vlt and the npm-family tarball flavors: npm, pnpm, bun, yarn classic, yarn berry): inside a git work tree, `git check-ignore --no-index` reports the new artifact's uuid directory as ignored by a rule its own `.gitignore` cannot override (such as a root `.socket/` or `vendor/` rule; the detail names the rule). Remedy: drop that rule for `.socket/vendor/`. Refused before any write. A file rule such as `*.tgz` is overridden by the `<uuid>/.gitignore` vendoring writes; if the written artifact still reads as ignored, the run refuses and removes the uuid dir it created. |
1288+
| `vendor_artifact_gitignore_unchecked` | warning | vendor (vlt and the npm-family tarball flavors): git is installed but could not answer the ignore check for the written vendored directory (it failed to start, ran past 30 s, or `rev-parse` / `check-ignore` exited with an error); the package is vendored and the detail names what failed. Remedy: make sure no ignore rule covers `.socket/` before committing. Git absent, or a project outside any work tree, raises nothing. |
12891289
| `vendor_ledger_entry_missing` | `failed` | vendor (vlt): the only installed copy is vlt's link to a committed vendored directory, but the vendor ledger has no entry for the package; restore `.socket/vendor/state.json` from version control (v5.0: `repair` no longer re-synthesizes it). Replaces the `package_not_installed` skip. |
12901290
| `vendor_variant_ambiguous` | `failed` | vendor / scan / get `--mode vendored` (pypi, gem): the package is not installed and the manifest holds several release variants of it (`?artifact_id=` / `?platform=`), none of which the vendor ledger records, so nothing says which distribution to vendor; install the package or keep one release variant. A variant the ledger records (at any patch uuid) is taken as the wired one and its siblings are left out without an event. |
12911291
| `vendor_artifact_missing` | reason | The recorded artifact is missing; repair requires an online exact redownload. |
@@ -1716,7 +1716,11 @@ wiring: an entry whose lockfile or config no longer references its
17161716
`package-lock.json` / `npm-shrinkwrap.json` entry for the vendored `name@version`
17171717
that `vendor` would rewire but that does not resolve to the vendored artifact
17181718
(#588); the reason names that entry. Missing ledger entries fail with
1719-
`vendor_ledger_missing`. Offline upstream metadata is reported as the run warning
1719+
`vendor_ledger_missing`: a manifest patch with no ledger entry, and a lockfile
1720+
reference to `.socket/vendor/<eco>/<uuid>/` that no ledger entry owns (the
1721+
artifact-level event repair emits: `uuid` plus `details.{ecosystem,path}`, no
1722+
purl), so a checkout whose ledger was ignored or dropped from the commit no
1723+
longer passes with nothing to check. Offline upstream metadata is reported as the run warning
17201724
`vendor_jvm_upstream_unverified`. The check never starts an API client or writes
17211725
lock/recovery files. `--check` conflicts with `--revert`.
17221726

‎crates/socket-patch-cli/src/commands/vendor.rs‎

Lines changed: 54 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1063,11 +1063,13 @@ async fn run_check(args: &VendorArgs) -> i32 {
10631063
}
10641064
env.record(event);
10651065
}
1066-
for key in manifest
1066+
let mut unledgered_uuids: HashSet<&str> = HashSet::new();
1067+
for (key, record) in manifest
10671068
.patches
1068-
.keys()
1069-
.filter(|k| !state.entries.contains_key(*k))
1069+
.iter()
1070+
.filter(|(k, _)| !state.entries.contains_key(*k))
10701071
{
1072+
unledgered_uuids.insert(record.uuid.as_str());
10711073
if !args.common.json {
10721074
eprintln!("{key}: patch has no vendored ledger entry");
10731075
}
@@ -1076,6 +1078,38 @@ async fn run_check(args: &VendorArgs) -> i32 {
10761078
"patch has no vendored ledger entry",
10771079
));
10781080
}
1081+
// A project file still wired to a vendored artifact the ledger does not
1082+
// know (the ledger was ignored or dropped from the commit along with the
1083+
// manifest) leaves every fresh install failing; the manifest keys above
1084+
// cannot see it, so the references are read from the wiring itself.
1085+
let references =
1086+
crate::commands::vendored_backend::repair::scan_vendor_references(root).await;
1087+
for (eco, uuid, rel) in references {
1088+
let ledgered = state
1089+
.entries
1090+
.values()
1091+
.any(|entry| entry.uuid == uuid && entry.ecosystem == eco);
1092+
if ledgered || unledgered_uuids.contains(uuid.as_str()) {
1093+
continue;
1094+
}
1095+
// No ledger entry means no purl to name: like repair, the event
1096+
// carries the uuid and the referenced path instead. The message
1097+
// names only the ecosystem, keeping patch identifiers out of logs.
1098+
let detail = format!(
1099+
"a lockfile references a vendored {eco} artifact under .socket/vendor/{eco}/ but \
1100+
the vendor ledger (.socket/vendor/state.json) has no entry for it; restore \
1101+
state.json from version control"
1102+
);
1103+
if !args.common.json {
1104+
eprintln!("vendor_ledger_missing: {detail}");
1105+
}
1106+
env.record(
1107+
PatchEvent::artifact(PatchAction::Failed)
1108+
.with_uuid(uuid)
1109+
.with_error("vendor_ledger_missing", detail)
1110+
.with_details(serde_json::json!({ "ecosystem": eco, "path": rel })),
1111+
);
1112+
}
10791113
if args.common.json {
10801114
println!("{}", env.to_pretty_json());
10811115
}
@@ -2718,15 +2752,23 @@ pub(crate) async fn vendor_records_reusing(
27182752
.clone();
27192753
let refusal = match project {
27202754
Some(refusal) => Some(refusal),
2721-
None => {
2722-
socket_patch_core::vendor::yarn_berry_vendor_target_preflight(
2723-
&common.cwd,
2724-
candidate,
2725-
pin,
2726-
&restore_opts,
2727-
)
2728-
.await
2729-
}
2755+
None => match socket_patch_core::vendor::npm_tarball_gitignore_preflight(
2756+
&common.cwd,
2757+
&record.uuid,
2758+
)
2759+
.await
2760+
{
2761+
Some(refusal) => Some(refusal),
2762+
None => {
2763+
socket_patch_core::vendor::yarn_berry_vendor_target_preflight(
2764+
&common.cwd,
2765+
candidate,
2766+
pin,
2767+
&restore_opts,
2768+
)
2769+
.await
2770+
}
2771+
},
27302772
};
27312773
if let Some((code, detail)) = &refusal {
27322774
has_errors = true;

0 commit comments

Comments
 (0)