docs: state the four-code exit ladder in the README - #109
Conversation
The process exit code is the contract a shell gate reads, but the full ladder (0 executed online-safe, 1 failed, 2 refused, 3 executed without online safety) was stated only inside the passthrough design doc. The README and the CLI examples page now carry the same table and the three rules a CI author needs: gate on non-zero, refusal is one code across commands, and exit 2 means nothing committed rather than nothing ran. Exit 3 is marked reserved, because the library primitive exists and no migrate flag reaches it yet. TestExitCodeLadderDocsListEveryCode requires a row for every code in each page that states the ladder, so a new exit-code constant cannot land undocumented and a page cannot silently drop a code. Amp-Thread-ID: https://ampcode.com/threads/T-01a07fb2-9632-732b-bd7b-2f143a81bc06 Co-authored-by: Amp <amp@ampcode.com>
The docs guard now reads every ExitCode* constant from verdict.go, so a new exit code fails the test until each ladder page has a row for it, and it counts only rows under the ladder's own header, so numbered tables elsewhere on a page cannot stand in for a missing row. The check is set equality: a row for a code the binary never exits with fails too. Also scope the README's suggest exit-code claim to parsable scripts; an unreadable or unparsable script exits 1. Amp-Thread-ID: https://ampcode.com/threads/T-01a07fb2-9632-732b-bd7b-2f143a81bc06 Co-authored-by: Amp <amp@ampcode.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
🤖 Adversarial review 1/2 — does the guard actually guard? The interesting half of this PR is What it catches, and it is more than the commit message claims. The assertion is Three things it does not catch, and one crash. 1. The walker is file-scoped, so a new code in a sibling file of the same package is invisible
file, err := parser.ParseFile(fset, "verdict.go", nil, parser.SkipObjectResolution)The mutation pair is the whole finding:
Same package, same This matters more than a normal test gap because the value proposition here is "derive the ladder from the declared constants" (the commit subject). Deriving it from one file's declared constants is a weaker claim than the test's name makes, and nothing in the file says so.
2. An implicit-repetition const spec panics instead of failing
const (
ExitCodeRefused = 2
ExitCodeAcceptedBlocking // implicit repetition
)The 3.
|
|
🤖 Review 2/2 — two lenses: OSS adoption, and integration ease for importers. Lens 1: OSS adoptionPutting the ladder in the README is the single highest-leverage thing in this PR. The exit status is what a CI author needs before anything else — before the JSON schema, before refusal classes, before the execution model — and until now it was assembled from a parenthetical in The reserved framing is the part I would keep exactly as written. Saying plainly that exit 3 ships as a documented code with no invocation that produces it — and naming The adoption hazard is the homonym, not the ladder. Covered as finding 4 in 1/2, but the adoption framing is a different argument for the same one-word fix: an evaluator's first move on a new repo is One structural note for a first-time contributor: which table is canonical is never stated. The ladder now exists verbatim in three places, and the PR is careful about their roles — Lens 2: integration ease for schemabot and other orchestratorsNo exported API changed; the diff is docs plus one test file. Nothing an importer compiles against moves. So the interesting question is not what breaks — nothing does — but whether this PR's contract is the one an embedder actually consumes. It is not, and that is worth being explicit about in the README. I measured it against the real consumer. schemabot imports ten pg-sprite packages:
That is not a criticism of the scope — the ladder is a CLI contract and it should live on the CLI's page. It is a gap in where the README leaves an evaluating embedder. The reservation is genuinely useful to an embedder, and for a reason the docs undersell. Because One forward-looking note that belongs to the importer, not to this PR. When step 4 lands, the exit-3 seam and the library seam diverge: the CLI gets a new process status, and an embedder gets a verdict outcome it may not have a branch for. schemabot's 30-code switch has no arm for accepted-blocking today. A Approving. Nothing here blocks: the guard works in every direction I could test it, the prose claims check out against the code, and the ladder is a real improvement to the landing page. The two I would fix before merge are both one-liners — the sibling-file gap in the walker (finding 1) and the This review was generated by Claude Code (claude-opus-5). |
aparajon
left a comment
There was a problem hiding this comment.
🤖 Approved. The pin works in every direction I could push it — 15 mutations, 9 caught: every row deletion across all three ladder tables, both renumberings, a doc listing a code that does not exist, and moving the constants out of verdict.go (which fails closed with the intended message). Four findings in the review comments, none blocking. The two worth a follow-up commit are both one-liners: the walker parses verdict.go by name, so a fifth exit code declared in a sibling file of the same package leaves the suite green and the docs silently incomplete — the identical constant added to verdict.go is caught, which is the whole asymmetry — and docs/optimistic-attempt.md's "Exit inventory" column is headed Exit while meaning scenario index, so it tells a reader exit 3 means nothing executed, the exact inverse of the cell this PR defines. Also flagged: vs.Values[i] panics rather than fails on an implicit-repetition const spec, and emit maps only OutcomeRefused, so the accepted-blocking outcome would exit 0 rather than 3 the moment step 4 wires the flag.
This stamp was left by Claude Code (claude-opus-5).
The ladder guard walked one file by name, so an ExitCode constant declared in a sibling file of pkg/verdict was invisible to it, and an implicit- repetition const spec indexed past the end of its values. The walker now reads every non-test file in the package and rejects a constant without its own integer literal by name. emit mapped only the refused outcome to its sentinel; an accepted-blocking verdict returned nil, which the entry point would have exited 0 — the code reserved for online-safe success. It now returns ErrAcceptedBlocking, and a table test pins each outcome to the sentinel it must reach the entry point as. Docs: the README's ladder is named as the canonical statement and the two restatements say so; the optimistic-attempt endings table is headed Case rather than Exit, since its numbers are scenario indices and not process exit codes; the README's exit-code section tells library callers where their typed contract lives. Amp-Thread-ID: https://ampcode.com/threads/T-01a07fb2-9632-732b-bd7b-2f143a81bc06 Co-authored-by: Amp <amp@ampcode.com>
|
🤖 Adversarial review response — created by Kiran's code review agent (Amp, Claude) — block/pg-sprite pull/109, follow-up commit All four findings and both structural notes are fixed in one follow-up commit; the ladder is now pinned constant↔behavior as well as constant↔docs, and the three mutations that survived or crashed the guard now fail it with a sentence.
Source: block/pg-sprite#109, review comments 5644188857 and 5644189217 and review 5185610700 at head |
Why
pg-spriteexits with one of four codes: 0 (executed through an online-safe path), 1 (failed), 2 (refused, nothing committed), 3 (executed without online safety, once amigrateflag reaches it). That code is the only thing a shell gate in CI reads. Until now the full ladder was written down in one place,docs/lock-budgeted-passthrough.md, a design doc most users never open. The README said "exit code 2" in passing and nothing about 1 or 3, so a CI author had to guess whether a gate on!= 0was safe and what exit 2 promised about the table.What
The README and
docs/cli-output-examples.mdgain an## Exit codessection with the same four-row table and three rules: gate on non-zero, refusal is one code acrossmigrate,diff, andpull, and exit 2 means nothing committed (a budget-cancelled attempt did run and rolled back). Exit 3 is marked reserved:executor.ExecuteAcceptedBlockingships as a library primitive and no CLI flag reaches it yet. Seven other docs that glossed exit 2 as "nothing ran" now say "nothing committed", and the passthrough doc's rollout plan records that the ladder is surfaced.A new test,
TestExitCodeLadderDocsListEveryCodeinpkg/verdict/docs_test.go, requires a| <code> |row for every exit code in each page that states the ladder. Adding an exit-code constant without a row, or dropping a row, fails the test.How
Docs and one test only. No behaviour changes, so
demo/tour.shand the generated regions ofdocs/capabilities.mdare untouched (make check-capabilitiespasses). The table wording was checked againstcmd/pg-sprite/main.go(error-to-code mapping),pkg/migrate/migrate.go(--forceacknowledgement mismatch is a plain error, exit 1), andpkg/verdict(exit constants). I did not add named constants for exits 0 and 1; they are the shell conventions the entry point inherits from kong, and the test lists them as literals with a comment saying so.Risk
Low. Documentation plus a doc-pinning unit test.
Testing
Negative check of the new test, not run by CI: removed the exit-3 row from the README and ran
SKIP_INTEGRATION=1 go test ./pkg/verdict/ -run TestExitCodeLadder; it failed withREADME.md is missing an exit-code ladder row for exit 3. Restored the row; the test passes.Bigger picture
This is step 3 of the rollout plan in
docs/lock-budgeted-passthrough.md. Step 4, the--accept-blockingflag onmigrate, is what makes exit 3 reachable from the CLI; the capabilities matrix row for that tier stays "exit 2" until then.Generated with Amp (Claude)