Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .github/workflows/evals.yml
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,17 @@ jobs:
file: test/skill-e2e-coverage-audit.test.ts
- name: e2e-triage
file: test/skill-e2e-triage.test.ts
# ship-docsync is whole-file tier-gated (describeE2ETier('gate') keeps
# it out of the periodic shard census), so its row MUST set tier: gate
# — without it the self-gate skips every test and the job reports a
# hollow green (the same silent-skip class as the rehomed monolith
# above, one layer deeper). The Run step exports EVALS_TIER from this
# property; rows without it keep EVALS_TIER empty (= unset: every
# reader is `=== '<tier>'` or truthiness). Enforced by
# test/evals-workflow-matrix.test.ts.
- name: e2e-ship-docsync
file: test/skill-e2e-ship-docsync.test.ts
tier: gate
- name: e2e-routing
file: test/skill-routing-e2e.test.ts
- name: e2e-codex
Expand Down Expand Up @@ -337,6 +348,12 @@ jobs:
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
EVALS_CONCURRENCY: "40"
PLAYWRIGHT_BROWSERS_PATH: /opt/playwright-browsers
# Per-row tier activation for whole-file-gated suites. Empty when the
# row declares no tier — every EVALS_TIER reader treats empty as unset
# (`=== '<tier>'` comparisons and the truthiness check in
# test/helpers/e2e-helpers.ts:70), so untiered rows are byte-for-byte
# unaffected.
EVALS_TIER: ${{ matrix.suite.tier || '' }}
run: EVALS=1 bun test --retry ${{ matrix.suite.retries || 1 }} --concurrent --max-concurrency 40 ${{ matrix.suite.file }}

- name: Upload eval results
Expand Down
41 changes: 41 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,46 @@
# Changelog

## [1.70.1.0] - 2026-08-26

**Ship names its documentation subagent at every decision point.**
**The handoff is now pinned by tests that fail loud if it ever goes quiet.**

`/ship` has dispatched `/document-release` as Step 18 since v0.18.2.0, but the v1.54.0.0 carve moved that step into an on-demand section and the always-loaded skeleton stopped saying "document-release" at any decision point (one mention survived, buried in the re-run checklist). The wiring was intact. The visibility was gone, and nothing tested the handoff. This release restores the visibility and locks it in: the section index, the STOP pointer, the Step 17 handoff line, and a new hoisted doc-sync invariant all name "the /document-release subagent" (subagent-framed on purpose, so an agent dispatches the isolated worker instead of running a weaker inline copy). A free tripwire pins the wording, carve-guard anchors pin each touchpoint independently, and a new gate-tier E2E proves a live agent actually fires the dispatch before creating the PR.

### The numbers that matter

Source: this branch's eval store (`~/.gstack/projects/<slug>/evals/`, runs of `test/skill-e2e-ship-docsync.test.ts`) and `wc -c ship/SKILL.md`.

| Property | Before | After |
|--------|--------|-------|
| Doc-sync subagent named in the Claude-host ship workflow body | 0 mentions at any decision point | 4 (section index, STOP pointer, Step 17 handoff, hoisted invariant) |
| Tests pinning the ship→document-release handoff | none | 5 free tripwire tests + 3 per-touchpoint carve anchors + 1 gate E2E |
| Live dispatch proof | never measured | 9/9 runs fire the dispatch before PR creation ($0.63-1.04, 234-319s each, sonnet-4-6) |
| Always-loaded skeleton cost | 91,267 B | 91,764 B (+497 B, cap raised to 92,300) |

Nine out of nine live runs is the line that matters. The E2E asserts on the actual tool-call stream, with a dispatch-specific matcher that a subagent merely quoting section text cannot satisfy, and a timeout-tolerant exit check that never softens the dispatch assert itself.

### What this means for gstack users

When you run `/ship`, the docs sync step is no longer an invisible line in a file the agent may summarize past. It is named at the exact moments the agent decides what to do next, and a merge-blocking test fails if any future edit makes it invisible again. A failed docs subagent still never blocks your ship. Nothing to configure. Upgrade and ship.

### Itemized changes

#### Fixed

- `/ship`'s Claude-host skeleton names "the /document-release subagent" at all three Step 18 decision points (manifest trigger rendering into the section index and STOP pointer, the Step 17 handoff line, and a hoisted doc-sync invariant beside the PR-title invariant). The invariant states the contract plainly: the dispatch itself is never skipped; only a failed subagent is non-blocking.

#### Added

- `test/ship-document-release-dispatch.test.ts`: free tripwire pinning the carved Step 18 contract (imperative, `subagent_type`, JSON return keys, non-blocking clause), the three skeleton touchpoints, the invariant-above-STOP ordering, the E2E matcher's four marker strings, and the inlined Step 18 → Step 19 ordering in the codex/factory goldens.
- `test/skill-e2e-ship-docsync.test.ts` (`ship-docsync`, gate tier): a live agent runs the sliced Step 17→19 ship tail in a hermetic git fixture; hard assert that an Agent dispatch matching the Step 18 prompt markers appears in the tool-call stream before any `gh pr create`. Fixture fails loud on step-marker drift, pins its git branch against operator config, asserts every setup command, and neutralizes the credential pre-push guard's question branch.

#### For contributors

- Carve-guards ship entry: three non-overlapping per-touchpoint anchors (gerund, imperative, third-person: no anchor subsumes another, so each is independently enforced), the carved imperative pinned to stay carved, skeleton byte cap 91,600 → 92,300 with the measured value recorded.
- `ship-docsync` registered in `E2E_TOUCHFILES` and `E2E_TIERS` (gate), with a whole-file `describeE2ETier('gate')` self-gate composed with diff selection so the file stays out of the periodic shard census; the tierless `test:evals` invisibility tradeoff is documented in the file header.
- Internal backlog notes corrected to describe the current Step 18 design (the pre-v1.54 "Step 8.5" prose was still documented as current), plus three deferred follow-ups recorded: a machine-checkable dispatch receipt, the same dispatch-pin treatment for land-and-deploy→canary, and the periodic shard-census ceiling arithmetic.

## [1.69.0.0] - 2026-08-22

**The silent-failure wave: tools that reported success while doing nothing —**
Expand Down
100 changes: 99 additions & 1 deletion TODOS.md
Original file line number Diff line number Diff line change
Expand Up @@ -2329,7 +2329,105 @@ Shipped as v0.5.0 on main. Includes `/plan-design-review` (report-only design au

### Auto-invoke /document-release from /ship — SHIPPED

Shipped in v0.8.3. Step 8.5 added to `/ship` — after creating the PR, `/ship` automatically reads `document-release/SKILL.md` and executes the doc update workflow. Zero-friction doc updates.
Shipped in v0.8.4; redesigned twice since. Current design (v0.18.2.0+, carved in
v1.54.0.0): `/ship` Step 18 (`ship/sections/pr-body.md`) dispatches
`/document-release` as a general-purpose subagent AFTER Step 17 (push) and
BEFORE Step 19 (PR creation); the subagent's JSON contract (`files_updated`,
`commit_sha`, `pushed`, `documentation_section`) is baked into the initial PR
body. Subagent failure is non-blocking. The skeleton names "the
/document-release subagent" at three touchpoints (section-index trigger + STOP
pointer, Step 17 handoff, hoisted doc-sync invariant). Pinned by
`test/ship-document-release-dispatch.test.ts` + carve-guards anchors; behavior
proven by the `ship-docsync` gate E2E (`test/skill-e2e-ship-docsync.test.ts`).

### Machine-checkable Step 18 dispatch receipt in /ship's Section self-check

**What:** Make ship's "Section self-check" verify a document-release dispatch
actually occurred (a machine-checkable marker/receipt), instead of relying on
prompt-level invariants alone.

**Why:** Prompt wording deters skipping but can't prove the dispatch happened.
Two residual gaps from the v1.69 review are folded into this scope: (1) an
agent invoking `/document-release` inline via the Skill tool bypasses the
fresh-context subagent + JSON contract and no test can see it; (2) the ship
RE-RUN path names document-release in the re-run list but no test asserts
doc-sync on re-run.

**Context:** The `ship-docsync` E2E asserts the dispatch tool-call on the
primary path; this TODO is the enforcement layer beyond wording. Start from
ship's Section self-check (ship/SKILL.md.tmpl) and the Step 18 parent
processing in ship/sections/pr-body.md.tmpl.

**Effort:** M (human) → S (CC+gstack)
**Priority:** P3
**Depends on:** ship-docsync E2E landed

### Apply the dispatch-pin + E2E pattern to /land-and-deploy → /canary

**What:** Same treatment ship→document-release got: name the handoff at the
skeleton decision points, pin with carve-guards anchors + a free tripwire,
prove with a toolCalls-assert E2E.

**Why:** Identical failure class — a carve or reword can silently strand the
canary handoff out of the always-loaded skeleton, and nothing tests it today.

**Context:** Model files: `test/ship-document-release-dispatch.test.ts` (free
pin) and `test/skill-e2e-ship-docsync.test.ts` (dispatch E2E, gate tier).

**Effort:** M (human) → S (CC+gstack)
**Priority:** P3
**Depends on:** None

### CI gate-lane hollow-coverage burn-down (evals.yml matrix)

**What:** `test/evals-workflow-matrix.test.ts` (added v1.70.1.0) ratchets two
pre-existing CI coverage holes; burn them down. (1) Eight gate-hosting test
files have no `evals.yml` matrix row, so CI never runs them
(`KNOWN_MATRIX_GAPS` in the test enumerates them — notably the plan-mode and
finding-floor smokes and the AUQ format-compliance gate). (2) Four matrix rows
point at whole-file tier-gated files but set no row `tier:` property, so with
`EVALS_TIER` unexported those suites self-skip: `codex-e2e`/`gemini-e2e` run
ZERO tests and report green on every PR (vestigial rows; the periodic cron
lane owns them — consider deleting the rows), and `e2e-pty-plan-smoke` spends
~7 min on setup then skips every describe (hollow-green since the files
adopted `describeE2ETier('gate')` — set `tier: gate` on the row to reactivate,
after confirming the smokes still pass).

**Why:** "Gate tier blocks merge" is silently false for these files. Each fix
is a deliberate cost/flake decision (activating paid suites on every PR), so
they're enumerated instead of drive-by-fixed. The mechanism already exists:
per-row `tier:` property, exported as `EVALS_TIER` by the Run step.

**Context:** Found 2026-08-26 on PR #2700 while adding the `ship-docsync` row.
Fix = add/adjust the matrix row, then DELETE the corresponding burn-down entry
(the tripwire fails on stale entries, so cleanup is enforced).

**Effort:** S per file (mechanical) + one burn-in run each to confirm green
**Priority:** P2
**Depends on:** None

### Periodic paid-test shard census is one ungated file from the detach-timeout floor

**What:** The periodic tier's shard census is 67 files — one ungated slot below
the 68-file (17×4) ceiling. The next paid `skill-e2e-*` file WITHOUT a
whole-file `describeE2ETier` self-gate lands at 68 (still 17 waves, floor
32,130s ≤ 32,400s — passes); the SECOND ungated file trips 18 waves → 34,020s
floor > the 32,400s configured detach timeout, and
`test/eval-detach-timeout-floor.test.ts` fails with a confusing message.

**Why:** Whoever adds the second ungated periodic E2E gets a floor failure
unrelated to their change. Fix options: raise the periodic detach timeout, or
enforce whole-file tier self-gates on all paid files (upgrades them from the
tier-alignment warn-only bucket to the hard invariant, and — bonus — restores
tierless `bun run test:evals` coverage decisions to diff selection alone).

**Context:** `scripts/test-paid-shards.ts` `classifyPaidTestFile` counts
ungated files in both tiers; `ship-docsync` composed `describeE2ETier('gate')`
with diff selection specifically to avoid consuming the last free slot.

**Effort:** S
**Priority:** P3
**Depends on:** None

### `{{DOC_VOICE}}` shared resolver

Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.69.0.0
1.70.1.0
9 changes: 7 additions & 2 deletions browse/test/bun-polyfill.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,13 @@ import * as path from 'path';

// Every test here spawnSync's a `node` child; Windows CI cold-start (AV scan,
// first-touch of node.exe) alone can blow bun's 5s default — observed 5,007ms
// on a 50ms sleep test. Subprocess budget, not assertion looseness.
setDefaultTimeout(20_000);
// on a 50ms sleep test. 20s was still not enough: on 2026-08-26 (PR #2700,
// run 32989821401) the 50ms sleep test blew 20s on BOTH bun retry attempts on
// a degraded windows-latest runner, so cold-start alone doesn't explain it —
// sustained AV/runner pressure does. Subprocess budget, not assertion
// looseness: every assertion still checks exact output, only the slowness
// allowance grows.
setDefaultTimeout(60_000);

// Load the polyfill into a fresh object (don't clobber globalThis.Bun)
const polyfillPath = path.resolve(import.meta.dir, '../src/bun-polyfill.cjs');
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "gstack",
"version": "1.69.0",
"version": "1.70.1",
"description": "Garry's Stack — Claude Code skills + fast headless browser. One repo, one install, entire AI engineering workflow.",
"license": "MIT",
"type": "module",
Expand Down
8 changes: 5 additions & 3 deletions ship/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -937,7 +937,7 @@ sections. Read a section in full before doing its step; do not work from memory.
| addressing Greptile review comments when a PR exists (Step 10) | `sections/greptile.md` |
| the adversarial review and learnings capture (Step 11) | `sections/adversarial.md` |
| writing the CHANGELOG entry (Step 13) | `sections/changelog.md` |
| syncing docs and creating or updating the PR/MR (Steps 18-19) | `sections/pr-body.md` |
| dispatching the /document-release subagent to sync docs (Step 18) and then creating or updating the PR/MR (Step 19) | `sections/pr-body.md` |

---

Expand Down Expand Up @@ -1417,13 +1417,15 @@ If `ALREADY_PUSHED`, skip the push but continue to Step 18. Otherwise push with
git push -u origin <branch-name>
```

**You are NOT done.** The code is pushed but documentation sync and PR creation are mandatory final steps. Continue to Step 18.
**You are NOT done.** The code is pushed but Step 18 (dispatch the /document-release subagent to sync docs) and Step 19 (create the PR/MR) are mandatory final steps. Continue to Step 18.

---

**PR/MR title invariant (always applies — do not skip even if you don't open the section below):** Any PR or MR you create OR update in the next step MUST have a title that starts with `v$NEW_VERSION` (the version bumped in Step 12), in the format `v<NEW_VERSION> <type>: <summary>`. Never create or edit a PR/MR title without this prefix. Compute the correct title with the single source of truth helper: `~/.claude/skills/gstack/bin/gstack-pr-title-rewrite.sh "$NEW_VERSION" "<current title>"`. The full create/update procedure (idempotency, redaction scan, self-check) is in the section below.

> **STOP.** Before syncing docs and creating or updating the PR/MR (Steps 18-19), Read `~/.claude/skills/gstack/ship/sections/pr-body.md` and execute it
**Doc-sync invariant (always applies — do not skip even if you don't open the section below):** Step 18 dispatches the /document-release subagent BEFORE the PR/MR is created or updated in Step 19. Never skip the dispatch itself; only a failed subagent is non-blocking (proceed to Step 19 without a `## Documentation` section).

> **STOP.** Before dispatching the /document-release subagent to sync docs (Step 18) and then creating or updating the PR/MR (Step 19), Read `~/.claude/skills/gstack/ship/sections/pr-body.md` and execute it
> in full. Do not work from memory — that section is the source of truth for this step.

## Step 20: Persist ship metrics
Expand Down
4 changes: 3 additions & 1 deletion ship/SKILL.md.tmpl
Original file line number Diff line number Diff line change
Expand Up @@ -496,12 +496,14 @@ If `ALREADY_PUSHED`, skip the push but continue to Step 18. Otherwise push with
git push -u origin <branch-name>
```

**You are NOT done.** The code is pushed but documentation sync and PR creation are mandatory final steps. Continue to Step 18.
**You are NOT done.** The code is pushed but Step 18 (dispatch the /document-release subagent to sync docs) and Step 19 (create the PR/MR) are mandatory final steps. Continue to Step 18.

---

**PR/MR title invariant (always applies — do not skip even if you don't open the section below):** Any PR or MR you create OR update in the next step MUST have a title that starts with `v$NEW_VERSION` (the version bumped in Step 12), in the format `v<NEW_VERSION> <type>: <summary>`. Never create or edit a PR/MR title without this prefix. Compute the correct title with the single source of truth helper: `~/.claude/skills/gstack/bin/gstack-pr-title-rewrite.sh "$NEW_VERSION" "<current title>"`. The full create/update procedure (idempotency, redaction scan, self-check) is in the section below.

**Doc-sync invariant (always applies — do not skip even if you don't open the section below):** Step 18 dispatches the /document-release subagent BEFORE the PR/MR is created or updated in Step 19. Never skip the dispatch itself; only a failed subagent is non-blocking (proceed to Step 19 without a `## Documentation` section).

{{SECTION:pr-body}}

## Step 20: Persist ship metrics
Expand Down
2 changes: 1 addition & 1 deletion ship/sections/manifest.json
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@
"id": "pr-body",
"file": "pr-body.md",
"title": "Documentation sync + PR/MR creation",
"trigger": "syncing docs and creating or updating the PR/MR (Steps 18-19)"
"trigger": "dispatching the /document-release subagent to sync docs (Step 18) and then creating or updating the PR/MR (Step 19)"
}
]
}
Loading
Loading