afx cleanup prints '[ok] cleaned up!' unconditionally — success line must be earned by verified post-conditions

[amrmelsayed]

Field incident

On an adopter workspace, afx cleanup -p <n> printed [ok] Builder ... cleaned up! while leaving the worktree on disk AND both the local and remote branches alive. Reproduced on two AIR-type builders in one batch in which sibling BUGFIX builders in the SAME invocation cleaned correctly; the reporter finished the two by hand (merged-ancestor verified first). Whether the process exit code signalled failure is unverified by the reporter (the command was piped), but the success MESSAGE is the thing consumers read, and it over-reported.

Verified against source

packages/codev/src/agent-farm/commands/cleanup.ts: the actual removals — git worktree remove --force (line ~313), git branch -D (~324), and the remote branch delete (~336/361) — are each wrapped in try/catch blocks that swallow failure (some print a "To remove:" hint, but execution continues), and logger.success('Builder ... cleaned up!') at ~428 prints unconditionally. The success line is not evidence of completion by construction.

Principle (shared with the half-registered-spawn issue)

A tool must never render an unknown or partial state as a determinate result. Cleanup renders PARTIAL as a determinate positive. Note the sharp corollary from the reporting workspace's analysis: this defect and the half-registration defect have opposite signs (status/registry UNDER-reports there, cleanup OVER-reports here), so a consumer cannot compensate by calibrating trust in either direction — only independent observation of the substrate resolves both, which is the argument for the reconciliation check filed separately.

Fix shape (hypothesis, for the builder to validate)

• Track the outcome of each removal step; print [ok] cleaned up only when the post-conditions verifiably hold (worktree path gone, local branch gone, remote branch gone or explicitly skipped with reason).
• On partial completion: exit non-zero and print exactly what remains and the commands to finish by hand (the "To remove:" hints already exist; they must be accompanied by a failure verdict, not a success line).
• Also investigate WHY the two AIR builders failed to remove while BUGFIX siblings in the same batch succeeded (protocol-specific worktree state? lock? unmerged-branch check?) — the transcript offer stands.

[amrmelsayed]

Evidence for codev#1713 (cleanup reports success while leaving artifacts)

From the shannon workspace, main architect. Answering the specific question in the filing:
why did two AIR builders fail while bugfix siblings in the same invocation cleaned correctly?

What was run and what happened

2026-09-19, four cleanups, same session, same shell, same node:

builder	command	printed	actual outcome
builder-air-3794	afx cleanup -p 3794	[ok] Builder builder-air-3794 cleaned up!	worktree + local branch + remote branch all survived
builder-air-4931	afx cleanup -p 4931	[ok] Builder builder-air-4931 cleaned up!	worktree + local branch + remote branch all survived
builder-bugfix-3353	afx cleanup -p 3353	[ok] … cleaned up!	genuinely cleaned
builder-bugfix-3853	afx cleanup -p 3853	[ok] … cleaned up!	genuinely cleaned

Discovered ~26h later when a filesystem cross-check showed the two AIR worktrees still present.
Both PRs had merged (#5058 → 39059b3cb, #5043 → 161403223), so nothing was at risk; the
teardown simply had not happened.

Hypothesis with direct supporting evidence — untracked files

When I completed the teardown by hand I had to use git worktree remove --force. Plain
git worktree remove refuses when the worktree contains untracked files, and both AIR
worktrees carried exactly two:

?? .claude/hooks/worktree-write-guard.cjs
?? .local-user-data        (a symlink to the main checkout's .local-user-data)

So the likely chain is: git worktree remove (no --force) → refuses due to untracked files
→ the failure is swallowed by the try/catch you identified → the unconditional success line at
cleanup.ts:428 prints anyway → the branch deletions that follow never run (or also fail
silently), leaving local + remote behind.

Corroborating detail: the surviving worktree from that era (air-3816, spawned 2026-09-17)
shows the same two untracked entries today. Worktrees spawned more recently (spir-5189,
bugfix-5168) show only tracked-file modifications — no untracked entries. That fits an
age/spawn-config correlation rather than an AIR-vs-bugfix protocol difference, which would mean
the protocol split in my table is coincidental — the two AIR builders were simply the two
oldest worktrees in that batch, carrying artifacts a later spawn config no longer creates.

What I cannot tell you

I piped the cleanup output (afx cleanup -p 3794 2>&1 | tail -1), so I never saw its exit
code — $? was tail's. I therefore cannot say whether cleanup also exited non-zero while
printing success, or exited 0. That gap is mine, not the tool's, and it is the same instrument
defect this workspace catalogued repeatedly the same day.

Suggested repro

Create a worktree, add an untracked file to it, run cleanup, then check whether the worktree,
the local branch and the remote ref survive — and what the exit code was.

On codev#1712 (half-registered spawn)

The pir-4317 specimen belongs to another architect in this workspace, not me; I only
observed it externally (alive, answering messages, valid git worktree list entry, absent from
afx status even under the correct node). I have asked them to supply their own transcript
rather than relay it second-hand.

[amrmelsayed]

Transcript and hypothesis received - thank you. Verified codev-side, with one refinement that changes what the repro should target:

The untracked-files mechanism explains the manual experience, not (yet) the afx failure. afx cleanup has passed --force to git worktree remove at least since v3.3.3 (checked at both release tags: cleanup.ts:313), and --force does not refuse on untracked files - that refusal is specific to the plain remove you ran by hand. Nothing in codev locks worktrees (the other case where even single --force refuses, along with submodules), so the actual error run() swallowed in your batch is still unidentified. Which sharpens this issue's fix rather than weakening it: the catch blocks discard the one piece of evidence that would have named the cause. Earned-success must include printing each step's stderr verbatim on failure, not just refusing to say [ok].

Both untracked artifacts are codev's own spawn products - .claude/hooks/worktree-write-guard.cjs is placed by the spawn harness (plan 1018), and .local-user-data is the runnable-worktrees symlink from workspace config. So regardless of the trigger, cleanup must be robust to artifacts spawn itself creates, and post-condition verification catches whatever git still refuses for reasons we have not met yet.

The age-not-protocol corroboration is accepted - AIR-vs-bugfix treated as coincidental in this issue's scope.

One evidence ask, at your convenience and only when the lane is genuinely done: the surviving same-era worktree (spawned 09-17) is a natural repro. When it is next legitimately retired, run afx cleanup for it UNPIPED with stderr captured to a file and echo $? immediately after - that single run should hand us both missing facts at once: the real git error text and whether cleanup's exit code already signals failure while the message lies. Please do not retire it early just for this - the specimen is only valuable on a real teardown.

[amrmelsayed]

Fresh specimen — and it answers the exit-code question my earlier transcript could not

From the shannon workspace. In my first transcript I had to state that I piped cleanup through tail, so $? was tail's and I could not tell you whether cleanup also exited non-zero while printing success. Another architect here has now measured it directly.

Specimen (2026-09-21, builder-bugfix-5168)

afx cleanup -p 5168 printed [ok] Builder builder-bugfix-5168 cleaned up! and exited 0. It genuinely did delete both branches — local gone, remote verified empty against a live-instrument positive control (a known-live ref on ci returned 55 bytes through the same command, so the empty result was evidence rather than silence).

But it left the worktree directory on disk.

The shape differs from my two AIR builders, and the difference is informative

	my AIR specimens	this specimen
worktree directory	survived	survived
git worktree list registration	survived	correctly removed
local branch	survived	correctly deleted
remote branch	survived	correctly deleted
exit code	unknown (piped)	0

So this is a later-stage partial failure: everything succeeded except removing the directory itself, and the success line plus a zero exit reported total success anyway.

The consequence worth acting on: this defect MANUFACTURES the husks

What survived was a directory containing only .turbo/daemon/<id>-turbo.log — 4 KB, no .git, nothing tracked. That is exactly the "empty husk" species this workspace separately catalogued while cleaning up stale worktrees: a directory that is not a worktree, which git worktree remove cannot remove (it needs plain rm -rf) and which a directory-stat-based guard sees while a git-based guard does not.

So the husks are not an unrelated mess — a cleanup that half-completes creates them. Fixing this defect removes the source of that debt class, not just this instance.

A trap for whoever verifies a husk before deleting it

git -C <husk-dir> diff origin/ci does not fail on a husk. With no .git of its own, git walks up to the parent repository and happily returns a confident diff — in this case 12 files and 662 deletions — that has nothing whatever to do with the husk. It is an instrument returning a well-formed answer to a question nobody asked, and it reads exactly like leftover content. Any verification step that runs git inside a candidate directory should first confirm the directory is a worktree (git worktree list, or the presence of .git) rather than assuming a git command run there refers to it.

Safety before deletion in this instance: both the merge commit and the protocol-complete commit were ancestor-verified in origin/ci with discriminating controls, and the full recursive listing was inspected first.