hyp purge <path> reports success and exits 0 while silently retaining rows recorded under an aliased spelling of that directory

[philcunliffe]

hyp purge <path> reports success and exits 0 while silently retaining every row recorded under a different spelling of that directory. The user asked for their captured LLM traffic to be deleted, the command said it worked, and the data is still in the cache.

This is a defect on master. It is not introduced by PR #484 or #482. It is being filed now because PR #484 closes #483, whose body lists this purge behaviour as one of the symptoms, so closing #483 would retire the only place it was written down.

Reproduction (executed on Linux, master @ 1555f13)

Rows are recorded under one spelling of a directory, the purge names the other. Both spellings are built as string literals, so this is fully witnessable off macOS: the subtree predicate is a byte comparison and never touches the filesystem.

$ hyp purge ~/café/proj --yes      # typed NFC; rows were recorded under NFD
purged 0 rows from 0 partitions
$ echo $?
0

rows recorded as	purge argument	exit	stdout	stderr	rows surviving
NFC	NFC (control)	0	purged 1 row from 1 partition	resurrection warning	0
NFD	NFC	0	purged 0 rows from 0 partitions	(empty)	1
NFC	NFD	0	purged 0 rows from 0 partitions	(empty)	1
/home/u/proj	/home/u/Proj	0	purged 0 rows from 0 partitions	(empty)	1

Identical on master (1555f13) and on PR #484's head, so #484 ships no regression here.

Why it is worse than the count suggests

• Exit 0, empty stderr. The outcome is indistinguishable from "that directory had nothing cached". There is no signal a user could act on.
• The failure is quieter than the success. A purge that actually deletes prints the LLP 0104 resurrection warning on stderr. The purge that silently retains prints nothing at all. Silence reads as a clean result.
• It is a deletion promise, not a capture leak. Every other spelling-divergence symptom in Usage-policy matcher never canonicalizes: a symlinked spelling of an ignored directory escapes its .hypignore (all adapters, hyp purge, query visibility) #479 / Usage-policy gate is escapable on macOS by path case or Unicode normalization: realpath does not fold either #483 fails toward privacy (the gate over-suppresses, policy unset refuses to remove an opt-out). This one fails away from it: data the user explicitly asked to destroy stays on disk.

Cause

src/core/cache/purge.js, the subtree branch of buildPredicate:

if (!isEqualOrDescendant(path.resolve(row.cwd), base)) return false

isEqualOrDescendant is a pure lexical prefix test and path.resolve folds neither Unicode normalization nor case. Two processes at different times produce the two strings (a CLI resolving the purge target, versus a client that recorded a cwd), so divergence is ordinary rather than user error. Same root cause as #479 (symlinks) and #483 (case / normalization), different call site.

What already covers part of this

• hyp purge --ignored is not affected once PR Usage-policy gate folds path spellings a volume treats as one directory (NFC verified, case unverified) #484 lands, because that target classifies each row through resolver.resolve(row.cwd) and so inherits Usage-policy gate folds path spellings a volume treats as one directory (NFC verified, case unverified) #484's fold. Verified: with an ignore entry declared NFC and rows recorded NFD, master purges 0 rows and Usage-policy gate folds path spellings a volume treats as one directory (NFC verified, case unverified) #484 purges the row. Marking the directory and running hyp purge --ignored is the workaround until this is fixed.
• PR Usage-policy gate matches directories, not path spellings: canonicalize both sides #482 closes the symlink half of this exact call site, rerouting the subtree predicate through its shared scopeGoverns predicate. After Usage-policy gate matches directories, not path spellings: canonicalize both sides #482 merges, hyp purge <path> folds symlinks but still not normalization or case.

So the residue this issue tracks is precisely: the subtree predicate does not fold the spellings #483 is about, after both PRs land.

The fix is not a one-liner, and that is the point

Do not close this by dropping foldPath into the predicate. At the gate, widening the match is free because the resolved class is max(declared, folded), so a wrong fold can only over-restrict. In a deletion predicate widening deletes. On a Linux volume, caf + U+00E9 and cafe + U+0301 are two directories with two inodes that can both exist in one parent (verified on this host: distinct ino, distinct contents, both listed by readdir), so folding NFC inside the purge predicate would delete cached rows for a genuinely different sibling directory the user never named. Same argument applies to case folding on a case-sensitive volume.

A correct fix needs the fold gated on the volume actually being normalization-insensitive. No such probe exists today: createVolumeCaseProbe (src/core/usage-policy/fold.js) answers only the case question. Options, for whoever picks this up:

1. Add a per-volume normalization-insensitivity probe alongside the case probe (mkdir/stat a decomposed spelling of an existing entry and compare dev/ino), and gate the purge fold on it. Most correct, most work.
2. Restrict purge-side folding to darwin, matching the existing probe's inertness off darwin. Cheap, and no worse than the platform assumption already shipped.
3. Widen the match but require confirmation, i.e. report "N additional rows match a different spelling of this path" and let the user opt in. Preserves the no-surprise-deletion property without needing a probe.

Whichever is chosen, it belongs in the one shared predicate (scopeGoverns, introduced by #482) rather than as a second folding rule at the call site, per #465.

Acceptance

• Purging a directory whose rows were recorded under an aliased spelling either deletes them or tells the user it did not, on a volume that treats the spellings as one directory.
• No over-deletion on a volume that treats them as two: a purge of caf+U+00E9 must not touch rows recorded under cafe+U+0301 on Linux.
• A regression test in test/core/purge-command.test.js covering both directions, with the two spellings written as \u escapes so an editor re-normalizing the file cannot collapse them (see the tripwire in test/core/usage-policy-fold.test.js).

Related

#479 (symlink class, PR #482), #483 (case/normalization class, PR #484), #465 (the duplicated-invariant pattern to avoid), LLP 0050 §normalization "Not covered", LLP 0104.

Filed by the round-2 review of PR #484.

[philcunliffe]

🤖 neutral: stuck on sequencing, not on the finding, plus a cheap partial mitigation you may want first

The defect is real, reproduces on master, and is directly actionable in principle. Neutral is declining to open a fix PR now and recording why, because a fix written today would be obsolete before it merged.

Why not now

The fix belongs in src/core/cache/purge.js's subtree predicate (isEqualOrDescendant, around line 87). PR #482 already rewrites that file and is currently held with neutral:approved, awaiting your merge:

git diff --stat origin/master...origin/fix/issue-479 -- src/core/cache/purge.js
 src/core/cache/purge.js | 24 +++++++++++++++++++-----

#482 introduces scopeGoverns / sameDirectory and wires them into exactly this predicate for symlink spellings. PR #484 (also held, neutral:approved) adds normalization folding in matcher.js but deliberately leaves purge.js untouched, precisely so it would not become a second folding rule.

So a correct fix for this issue is: unify #482's spelling-set predicate with #484's folding, then apply the unified predicate at the purge seam, plus a per-volume normalization-insensitivity probe that does not exist yet (#484 built the analogous case probe, so it is tractable, but it is on an unmerged branch). Writing that against master today means writing against a predicate two approved PRs are about to replace.

Both reviews of #484 independently reached the same conclusion and warned specifically against the tempting shortcut: dropping foldPath into the purge predicate as-is would over-delete, because folding is only safe at the gate where the verdict is max(declared, folded). LLP 0050 previously justified unconditional folding with a claim that turned out to be false (that no targeted filesystem distinguishes normalization forms; disproved by execution, two inodes in one parent), and that false claim is exactly what would have licensed the shortcut.

How this unsticks

Merge #482 and #484. Then drop neutral:stuck here, or reply, and neutral will open the unified fix against a settled predicate.

A cheap partial mitigation, if you want it sooner

The worst property of this bug is not the retention, it is the silence. What a user sees today:

purged 0 rows from 0 partitions      exit 0      stderr empty      rows still on disk

That is indistinguishable from "nothing was cached", and the aggravating detail is that the successful purge is the noisier one, since it prints a resurrection warning while silent retention prints nothing.

A diagnostic-only change would touch no contended code: when a purge matches zero rows for a path that does exist, say so, and point at hyp purge --ignored, which is already correct because it classifies through resolver.resolve() (verified: master leaves the row, #484 deletes it). That workaround was previously documented nowhere.

Neutral has not done this, because it changes user-facing CLI output and that is your call, not something to slip in under a data-retention issue. If you want it, say so and it will be a separate small PR held for review as usual.

Related

• PR Usage-policy gate matches directories, not path spellings: canonicalize both sides #482 (rewrites the predicate), PR Usage-policy gate folds path spellings a volume treats as one directory (NFC verified, case unverified) #484 (Fixes #483, adds folding, filed this issue), Usage-policy matcher never canonicalizes: a symlinked spelling of an ignored directory escapes its .hypignore (all adapters, hyp purge, query visibility) #479, Usage-policy gate is escapable on macOS by path case or Unicode normalization: realpath does not fold either #483, Two independent readers of Codex session_meta enforce the same privacy-relevant invariant; it has silently drifted twice already #465 (the duplicated-invariant pattern to avoid), LLP 0050 §normalization

[philcunliffe]

Both #482 and #484 are merged now, so the sequencing blocker is gone - go ahead with the unified fix.

Decisions:

1. Fix approach: option 1 - add a per-volume normalization-insensitivity probe alongside the existing case probe, and gate the purge-side fold on it. No folding in the deletion predicate on volumes that treat the spellings as distinct directories. Keep it in the shared scopeGoverns predicate per Two independent readers of Codex session_meta enforce the same privacy-relevant invariant; it has silently drifted twice already #465, not a second rule at the call site.

2. Yes to the diagnostic-only mitigation as a separate small PR: when a purge matches zero rows for a path that exists, say so on stderr and point at hyp purge --ignored. Land it independently so the silence is fixed even before the main fix merges.