Skip to content

fix: reviews whose repository inspection failed still publish a rating and close items - #1025

Draft
vincentkoc wants to merge 1 commit into
mainfrom
fix/recover-pr-988-history-rewrite
Draft

fix: reviews whose repository inspection failed still publish a rating and close items#1025
vincentkoc wants to merge 1 commit into
mainfrom
fix/recover-pr-988-history-rewrite

Conversation

@vincentkoc

Copy link
Copy Markdown
Member

History-rewrite recovery. GitHub auto-closed the original contributor PR #988 on August 4, 2026 when main was rewritten to remove generated-state history. GitHub would not reopen it because the fork no longer shares a commit ancestor. This draft reapplies the original single commit to rewritten main. The original author, authored date, commit message, and co-author trailer are preserved; the predicate change is ported from retired src/clawsweeper.ts to current src/clawsweeper-record-metadata.ts. No unrelated behavior is included.

Recovery Validation

Current range. Base bb69d1fc582d46298df0f26f9192f721f976e9f4; head 96b73e3bc5c058b57ca73ee3337ec351c3c727df.

Equivalence. The original commit message, author identity/date, co-author trailer, and test blob are byte-for-byte preserved. The seven-line predicate change is identical after accounting for the module move on current main.

Commands. ./node_modules/.bin/tsc -p tsconfig.json; node --test test/apply-blocked-local-checkout.test.ts; pre-commit codex review --uncommitted; committed-range codex review --base origin/main.

Observed result. The focused test passed, both Codex reviews reported no actionable findings, and the committed-range review passed 80 relevant tests plus typecheck, lint, and formatting.

Limit. This replacement preserves the original contributor patch and proof package; it does not broaden blocked-checkout phrase detection or change repair candidate selection.

What Problem This Solves

When ClawSweeper's read-only repository inspection fails to start, the review it publishes still
carries a Patch quality tier and a merge-readiness verdict, and the item stays eligible for the
apply lane to close. The review text says the opposite in the same comment — for example:

Inspection infrastructure failure: The only permitted local read command failed before opening
repository files: bwrap: loopback: Failed RTM_NEWADDR: Operation not permitted. Consequently,
current-main source, scoped AGENTS guidance, history, and dependency contracts were not
independently inspected.

while the scores table in the same comment reports:

| Patch quality | 🦐 gold shrimp (3/6) | No actionable review findings were identified. |

Maintainers and contributors read a rating and a "no findings" rationale produced by an inspection
that provably did not run, and the item can be closed on that basis.

This is not rare. Of the ClawSweeper comment bodies on openclaw/openclaw last updated between
2026-07-27 and 07-31T12:00Z, 203 of 8,515 currently match bwrap/RTM_NEWADDR — a steady
2.4% share on each of those days. Among bodies last updated during 2026-07-31T14:00Z and
15:00Z that share was 37.2% and 49.3%. Because reviews land as in-place comment edits these
are lower bounds, and they count comment bodies rather than review runs; the exact collection method
and what it does and does not prove are in Real Behavior Proof below.

Why This Change Was Made

The repository already has the guard for this case. hasBlockedLocalCheckoutAccess detects a review
that reports its inspection never ran, and three apply-lane call sites gate on
hasVerifiedLocalCheckoutAccess: the guard whose skip reason is literally
review lacks verified local checkout access, the comment-sync guard, and the counterpart check in
the same-author PR+issue pair close.

Those guards could not distinguish this case. local_checkout_access: verified is emitted as a
literal by the review-record writer, so for every record that writer produces
hasVerifiedLocalCheckoutAccess was true regardless of what the review reported. The blocked
predicate's only consumer was effectiveReviewStatus, which classifies the record
stale_local_checkout_blocked — that affects re-review scheduling, not publication or close.

The change requires both signals, so the existing guards apply to reviews that already report a
blocked inspection. No new configuration, no new state, no change to how ratings are rendered.

effectiveReviewStatus is unchanged for every input: it tests the blocked predicate first, so
blocked records still return stale_local_checkout_blocked, and non-blocked records fall through to
the same front-matter check as before. That is an invariant by construction in a four-line function,
not something a new test asserts.

Non-goals.

  • bubblewrap, the runner image, and network-namespace permissions — the sandbox failure originates
    in the review runtime, not here.
  • src/repair/workflow-utils.ts reads local_checkout_access from the front matter directly and is
    built as a separate module that does not import this predicate. It is unchanged: blocked
    records remain selectable there. That site produces close candidates for the apply lane rather
    than closing anything itself, so the apply guard above is still what withholds the close — but the
    selection is wasted work and making that module agree is left for a separate change.
  • widening which phrasings count as a blocked inspection (see Limits).
  • the rendered scores table; withholding or marking the patch-quality axis for a blocked review is a
    separate question.

User Impact

On the ordinary apply path, a review whose report matches the existing blocked-inspection predicate
no longer syncs its comment and is not eligible to close. Such a record is already classified
stale_local_checkout_blocked, so it stays due for re-review. This change does not remove or edit a
blocked comment that was already published, and it does not make the scheduled re-review succeed; it
stops the lane from publishing a new one or closing on it.

A matching record also no longer qualifies as the counterpart in a same-author PR+issue pair close,
so that pair is kept open instead. Same fail-safe direction, and no focused test covers that path.

The withholding is bounded in two ways, both deliberate:

  • It applies to reports the existing predicate recognises, not to every failed inspection. The
    promisor/DNS variant in Limits below is a counterexample that this change does not catch.
  • Two pre-existing bypasses are unchanged: shouldProbeClosedState skips the apply guard, and
    staleCanonicalCommentSyncPending skips the comment-sync guard. Both are narrow recovery states,
    and this change does not alter either condition.

For records that do not match the predicate, nothing changes.

Evidence

src/clawsweeper.ts +7/−1, plus one focused test in the narrowest matching file
(test/apply-blocked-local-checkout.test.ts, alongside the existing apply-* policy tests).

Real Behavior Proof

Claim. A review record whose body reports that its read-only inspection never ran is admitted
past the apply lane's local-checkout guard on main, and is withheld after this change with the skip
reason the guard already defines. The claim is scoped to the ordinary apply path — the exception
branches above and src/repair/workflow-utils.ts are not exercised.

Environment. Node v24.18.0, Linux. Base 0ee212e35. Branch head 69926bd38.

1. Non-mocked dry-run against a live GitHub item

Real gh at /usr/bin/gh, no GH_BIN / GH_BIN_ARGS override (env | grep -c '^GH_BIN'0).
--dry-run short-circuits before the comment upsert (src/clawsweeper.ts:29492) and before the close
(src/clawsweeper.ts:29827), so every path below is read-only. Following the apply-repro guidance in
AGENTS.md, one record for the real open issue openclaw/clawsweeper#951 was copied into a temp
items/ dir with a temp --record-root, --closed-dir, and --plans-dir. The record declares
local_checkout_access: verified and labels: [], and its body carries the sandbox-failure sentence.

$ node dist/clawsweeper.js apply-decisions --target-repo openclaw/clawsweeper \
    --record-root $R --items-dir $R/items --closed-dir $R/closed --plans-dir $R/plans \
    --report-path $R/apply-report.json --item-numbers 951 --dry-run \
    --limit 1 --processed-limit 2 --close-delay-ms 0

With this change (head 69926bd38):

[apply] 2026-07-31T23:50:41.883Z starting apply: files=1 dry_run=true apply_kind=issue ... item_numbers=951 ... counts={}
[apply] 2026-07-31T23:50:43.350Z finished apply closed=0/1 processed=1/2 counts={"kept_open":1}

$ cat $R/apply-report.json
[
  {
    "number": 951,
    "action": "kept_open",
    "reason": "review lacks verified local checkout access"
  }
]

Same record, same command, unmodified base 0ee212e35:

[apply] 2026-07-31T23:51:21.636Z starting apply: files=1 dry_run=true ... item_numbers=951 ... counts={}
[apply] 2026-07-31T23:51:27.937Z finished apply closed=0/1 processed=1/2 counts={"skipped_protected_label":1}

$ cat $R/apply-report.json
[
  {
    "number": 951,
    "action": "skipped_protected_label",
    "reason": "protected label: clawsweeper:needs-maintainer-review, clawsweeper:needs-product-decision, clawsweeper:needs-security-review"
  }
]

On base the record is admitted past the local-checkout guard and evaluated further, reaching the live
protected-label check. That check reports labels the record never declared, so the run genuinely
queried GitHub — gh was not mocked. With the change the same record stops at the guard.

This particular item carries protected labels, so on base it stops there rather than at
would close; the admitted-through-to-close path is what the harness test below shows.

2. Focused harness test (before / after)

Exercised surface. The same apply entry point, node dist/clawsweeper.js apply-decisions, driven
end to end through runApplyDecisionsForTest, with gh mocked at the process boundary via
withMockGh so the close proposal can be carried to completion deterministically.

Scenario / fixture. One review record in a temp items/ dir, carrying the front-matter value the
review writer emits (local_checkout_access: verified, asserted in the test) and, in the body, the
sandbox-failure sentence quoted above. Decision close / implemented_on_main, with the durable
review comment already synced.

Command.

./node_modules/.bin/tsc -p tsconfig.json
node --test test/apply-blocked-local-checkout.test.ts

Observed result — on main (fix reverted, test kept):

✖ apply withholds close when the review reports a blocked local checkout
  AssertionError [ERR_ASSERTION]: Expected values to be strictly deep-equal:
  + actual - expected

    [
      {
  +     action: 'review_comment_synced',
  -     action: 'kept_open',
        number: 10,
  +     reason: 'would update durable Codex review comment'
  -     reason: 'review lacks verified local checkout access'
      },
  +   {
  +     action: 'closed',
  +     number: 10,
  +     reason: 'dry-run: would close as already implemented on main'
  +   }
    ]

The item is comment-synced and closed. The guard never fires.

Observed result — with this change:

✔ apply withholds close when the review reports a blocked local checkout (155.679ms)
ℹ tests 1
ℹ pass 1
ℹ fail 0

Reverting only src/clawsweeper.ts to 0ee212e35 and rebuilding reproduces the failure above, so the
test has teeth.

3. Public measurement

Reproducible from the GitHub API. For every clawsweeper[bot] issue comment on openclaw/openclaw
whose body was last updated in the window, the current body was matched against /bwrap|RTM_NEWADDR/
and bucketed by updated_at.

## Bodies last updated 2026-07-27 -> 07-31T12:00Z
2026-07-27  bodies= 973  matching= 18  share=  1.8%
2026-07-28  bodies= 915  matching= 24  share=  2.6%
2026-07-29  bodies=1121  matching= 27  share=  2.4%
2026-07-30  bodies=3148  matching= 74  share=  2.4%
2026-07-31  bodies=2358  matching= 60  share=  2.5%
TOTAL       bodies=8515  matching=203  share=  2.4%

## Bodies last updated 2026-07-31T12:00Z -> 16:51Z (hour buckets, UTC)
2026-07-31T12  bodies= 290  matching=  5  share=  1.7%
2026-07-31T13  bodies= 228  matching=  5  share=  2.2%
2026-07-31T14  bodies= 129  matching= 48  share= 37.2%
2026-07-31T15  bodies= 150  matching= 74  share= 49.3%
2026-07-31T16  bodies= 330  matching= 32  share=  9.7%
TOTAL          bodies=1127  matching=164  share= 14.6%

What this is and is not: matching counts bodies matching /bwrap|RTM_NEWADDR/, which is a
narrower query than hasBlockedLocalCheckoutAccess — the predicate also recognises five other
sandbox phrasings, so the true share is at least this. Buckets are comment bodies last updated in
that bucket
, not publication events and not per-day review incidence. Reviews land as in-place
edits, so a body written during an outage and re-reviewed afterwards now reads clean and is counted
clean. A match shows the published text reports a failed inspection; it is not by itself evidence
that any apply action ran on that item.

4. Suite and static checks

node scripts/run-node-tests.mjs all → 2851 tests, 2845 pass, 6 fail. The same 6 fail on
unmodified 0ee212e35 in this environment (test/repair/target-validation.test.ts — detached-process
reaping and git branch plumbing, both needing a real pnpm install and worktree setup this checkout does
not have). oxfmt --check, oxlint (src / scripts+test), check:active-surface, check:limits,
check:dashboard-queue-boundary all pass.

Limits — what this proof does not cover

  • Coverage is one path. Both runs exercise the ordinary apply path only. The same-author
    pair-close counterpart check, the shouldProbeClosedState and staleCanonicalCommentSyncPending
    bypasses, and the close-promotion selector in src/repair/workflow-utils.ts are untested here.
    The effectiveReviewStatus invariant is argued by construction, not asserted by a new test.
  • The non-mocked run stops earlier on base than the harness run does. openclaw/clawsweeper#951
    carries protected labels, so on base it reaches skipped_protected_label rather than a close. The
    non-mocked run therefore demonstrates that the checkout guard does not fire on base and does fire
    after the change; the close consequence itself is shown only under the mocked harness.
  • No auto-close on a blocked review was observed in production. The close path is reachable —
    clawsweeper[bot] closed 10 openclaw/openclaw items in the last 7 days — but none of the items
    closed during the sampled window carried a matching review. The 39 matching items closed between
    14:00Z and 16:51Z were all closed by humans. The close behaviour above is demonstrated on the real
    apply path, not observed in the wild; the published-rating harm is what is observed.
  • The measurement counts comment bodies, not review runs, and is a lower bound. See the method
    note above it.
  • The claim about the writer is scoped to the writer. hasVerifiedLocalCheckoutAccess was true for
    every record the current review-record writer produces, because that writer emits the literal. It says
    nothing about hand-edited, malformed, or pre-existing records that lack the key; those already
    resolved to unverified and still do.
  • Detection coverage is unchanged and is known to be incomplete. A re-review of one affected PR
    replaced the bwrap failure with promisor objects; Git attempted to fetch them and failed because github.com could not resolve — plausibly the same root, but hasBlockedLocalCheckoutAccess does not
    match that phrasing, so such a review is not even classified stale. Widening the predicate from one
    observed sample seemed worse than leaving it to a decision about how the reviewer should report
    inspection failure structurally; the review prompt currently asks for prose only.
  • False-positive direction. The predicate matches the whole record, so a review of a PR that
    legitimately discusses bwrap: loopback would be treated as blocked. That already causes permanent
    staleness on main; with this change it would also withhold the comment on the ordinary path. The
    direction is fail-safe — the lane declines to act rather than acting on a review it cannot trust — but
    it is a real behaviour change for that case.

`local_checkout_access: verified` is written as a literal in the review-record
template, so `hasVerifiedLocalCheckoutAccess` was true for every record. The
three guards that consult it could not fire, including the apply guard whose
skip reason is "review lacks verified local checkout access".

`hasBlockedLocalCheckoutAccess` already detects a review that reports its
read-only inspection never ran, but its only consumer was
`effectiveReviewStatus`, which affects re-review scheduling and not publication
or close. A review that recorded a sandbox startup failure was therefore still
comment-synced and still eligible to close.

Require both signals so the existing guards apply. `effectiveReviewStatus` is
unchanged: it tests the blocked predicate first, so the
`stale_local_checkout_unverified` branch stays unreachable for blocked records.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. P1 Urgent regression or broken agent/channel workflow affecting real users now. merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. labels Aug 4, 2026
@clawsweeper

clawsweeper Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs changes before merge. Reviewed August 7, 2026, 12:23 PM ET / 16:23 UTC.

ClawSweeper review

What this changes

The branch treats a review record as lacking verified checkout access when its rendered report matches blocked-inspection phrases, with a focused apply-lane regression test.

Merge readiness

⚠️ Ready for maintainer review - 7 items remain

This MEMBER-authored draft addresses a real apply-lane fail-open, but it still makes rendered review prose authoritative for a GitHub-mutation gate. The unchanged prior P2 blocker remains: persist inspection status outside Markdown before merging.

Priority: P1
Reviewed head: 96b73e3bc5c058b57ca73ee3337ec351c3c727df

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) The intended failure mode is demonstrated well, but the implementation needs a structured trust-boundary repair before merge.
Proof confidence 🦞 diamond lobster (5/6) Not applicable: This MEMBER-authored PR is outside the external-contributor proof gate; its body nevertheless includes a production apply dry-run and focused regression evidence for the intended blocked-inspection path.
Patch quality 🦐 gold shrimp (3/6) Security review found an item that needs attention.

Verification

Check Result Evidence
Real behavior Not applicable Not applicable: This MEMBER-authored PR is outside the external-contributor proof gate; its body nevertheless includes a production apply dry-run and focused regression evidence for the intended blocked-inspection path.
Evidence reviewed 3 items Unconditional writer value: Current main always writes local_checkout_access: verified, so the structured field does not currently represent whether repository inspection actually occurred.
Proposed trust boundary: The branch changes verification to reject the whole rendered report when it contains a recognized blocked-checkout phrase; quoted evidence from a successful review can therefore suppress apply actions.
Prior hardening provenance: Recent merged work hardened this same report-metadata surface against model-prose control-data confusion, supporting a structured-state repair rather than another prose-derived gate.
Findings 2 actionable findings [P2] Persist checkout status outside rendered Markdown
[P3] State the OpenClaw Bay impact
Security Needs attention Report prose controls apply eligibility: A phrase match anywhere in model-generated or quoted report text changes whether GitHub actions are permitted, rather than relying on a structured execution result.

How this fits together

ClawSweeper writes durable review records that the apply lane reads before syncing a review comment or closing an item. Those records connect review execution to GitHub mutations and re-review scheduling.

flowchart LR
  A[Repository review] --> B[Durable review record]
  B --> C[Checkout-access gate]
  C --> D[Apply lane]
  D --> E[Sync review comment]
  D --> F[Close eligible item]
  C --> G[Keep open for re-review]
Loading

Before merge

  • Persist checkout status outside rendered Markdown (P2) - The new predicate scans all rendered report text. A successful review can quote one of these sandbox-failure phrases in evidence, causing apply to withhold comment sync and closure despite successful inspection. Persist a runner-owned status and consume that instead; this is the unresolved prior finding on the unchanged head.
  • State the OpenClaw Bay impact (P3) - This changes review and apply status behavior, but the PR body does not say whether OpenClaw Bay is affected or why no Bay change is needed, as required by repository policy.
  • Resolve security concern: Report prose controls apply eligibility - A phrase match anywhere in model-generated or quoted report text changes whether GitHub actions are permitted, rather than relying on a structured execution result.
  • Resolve merge risk (P1) - A successful review that quotes a recognized sandbox-failure phrase can be withheld from comment sync and closure despite verified inspection.
  • Resolve merge risk (P1) - The test proves the blocked path but not the successful-inspection case containing the same phrase.
  • Resolve merge risk (P1) - The PR body does not state whether the review/apply status change affects OpenClaw Bay or why it does not.
  • Complete next step (P2) - A focused mechanical repair can replace prose-derived inspection inference with structured state and cover both false-negative and false-positive cases.

Findings

  • [P2] Persist checkout status outside rendered Markdown — src/clawsweeper-record-metadata.ts:482-484
  • [P3] State the OpenClaw Bay impact — src/clawsweeper-record-metadata.ts:478-484
  • [medium] Report prose controls apply eligibility — src/clawsweeper-record-metadata.ts:484
Agent review details

Security

Needs attention: The patch makes rendered review prose control an apply-lane mutation boundary.

Review metrics

Metric Value Why it matters
Patch scope production +7/-1, tests +102 A small predicate change controls comment publication and closure eligibility across the apply lane.

Root-cause cluster

Relationship: canonical
Canonical: #1025
Summary: This PR is the active recovery of the original blocked-checkout fix after the earlier branch became unreopenable following history rewrite.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge-risk options

Maintainer options:

  1. Persist the inspection result structurally (recommended)
    Write a runner-owned inspection status and have every apply and pair-close gate use it instead of scanning rendered review prose.
  2. Pause the recovery branch
    Keep the PR draft if structured inspection state is not ready, because the current text-derived gate can deny valid apply actions.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Persist a runner-owned local-inspection status, consume it in apply, comment-sync, pair-close, and promotion gates, and add regressions for both blocked inspection and successful inspection that quotes a recognized blocked-checkout phrase.

Technical review

Best possible solution:

Persist a runner-owned inspection outcome in durable metadata and make apply, comment-sync, pair-close, and promotion gates consume that value; retain prose only for diagnostics or legacy fail-closed handling.

Do we have a high-confidence way to reproduce the issue?

Yes—current source proves that the proposed whole-report phrase check changes apply eligibility, including when a successful review quotes the phrase as evidence.

Is this the best way to solve the issue?

No—the fail-safe goal is appropriate, but rendered Markdown is not a safe authority for checkout-execution state; structured runner-owned metadata is the narrower durable fix.

Full review comments:

  • [P2] Persist checkout status outside rendered Markdown — src/clawsweeper-record-metadata.ts:482-484
    The new predicate scans all rendered report text. A successful review can quote one of these sandbox-failure phrases in evidence, causing apply to withhold comment sync and closure despite successful inspection. Persist a runner-owned status and consume that instead; this is the unresolved prior finding on the unchanged head.
    Confidence: 0.98
  • [P3] State the OpenClaw Bay impact — src/clawsweeper-record-metadata.ts:478-484
    This changes review and apply status behavior, but the PR body does not say whether OpenClaw Bay is affected or why no Bay change is needed, as required by repository policy.
    Confidence: 0.96

Overall correctness: patch is incorrect
Overall confidence: 0.98

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against f6f6bfca7d65.

Labels

Label changes:

  • add merge-risk: 🚨 security-boundary: Rendered model-generated text becomes authority for a privileged GitHub-mutation gate.
  • remove proof: sufficient: Current real behavior proof status is not_applicable, not sufficient.

Label justifications:

  • P1: The PR changes a production automation gate that controls review publication and GitHub closure eligibility.
  • merge-risk: 🚨 automation: The changed predicate directly alters apply-lane comment-sync and close decisions.
  • merge-risk: 🚨 security-boundary: Rendered model-generated text becomes authority for a privileged GitHub-mutation gate.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🦞 diamond lobster and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Not applicable: This MEMBER-authored PR is outside the external-contributor proof gate; its body nevertheless includes a production apply dry-run and focused regression evidence for the intended blocked-inspection path.

Evidence

Security concerns:

  • [medium] Report prose controls apply eligibility — src/clawsweeper-record-metadata.ts:484
    A phrase match anywhere in model-generated or quoted report text changes whether GitHub actions are permitted, rather than relying on a structured execution result.
    Confidence: 0.98

Acceptance criteria:

  • [P1] pnpm run check.
  • [P1] node --test test/apply-blocked-local-checkout.test.ts.
  • [P1] fresh committed-range Codex review against main.

What I checked:

  • Unconditional writer value: Current main always writes local_checkout_access: verified, so the structured field does not currently represent whether repository inspection actually occurred. (src/clawsweeper-report-document.ts:584, f6f6bfca7d65)
  • Proposed trust boundary: The branch changes verification to reject the whole rendered report when it contains a recognized blocked-checkout phrase; quoted evidence from a successful review can therefore suppress apply actions. (src/clawsweeper-record-metadata.ts:484, 96b73e3bc5c0)
  • Prior hardening provenance: Recent merged work hardened this same report-metadata surface against model-prose control-data confusion, supporting a structured-state repair rather than another prose-derived gate. (src/clawsweeper-record-metadata.ts:503, 2eb1787e0d18)

Likely related people:

  • steipete: Authored the recent merged report-metadata hardening on the affected record and rendering surfaces. (role: recent area contributor; confidence: high; commits: 3f368a3e394d, 2eb1787e0d18; files: src/clawsweeper-record-metadata.ts, src/clawsweeper-report-document.ts)
  • masatohoshino: Authored the recovered predicate-and-test change that this PR reapplies. (role: original change author; confidence: high; commits: 96b73e3bc5c0; files: src/clawsweeper-record-metadata.ts, test/apply-blocked-local-checkout.test.ts)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Persist runner-owned inspection status.
  • Add the successful-inspection-with-quoted-phrase regression.
  • State the OpenClaw Bay impact in the PR body.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (28 earlier review cycles; latest 8 shown)
  • reviewed 2026-08-07T01:09:27.012Z sha 96b73e3 :: found issues before merge. :: [P2] Record inspection status outside rendered Markdown
  • reviewed 2026-08-07T02:41:15.311Z sha 96b73e3 :: found issues before merge. :: [P2] Record inspection status outside rendered Markdown
  • reviewed 2026-08-07T02:57:08.989Z sha 96b73e3 :: needs changes before merge. :: [P2] Record inspection status outside rendered Markdown
  • reviewed 2026-08-07T08:36:29.234Z sha 96b73e3 :: found issues before merge. :: [P2] Persist inspection status outside review Markdown
  • reviewed 2026-08-07T11:24:23.482Z sha 96b73e3 :: found issues before merge. :: [P2] Do not infer checkout access from report prose
  • reviewed 2026-08-07T13:17:47.589Z sha 96b73e3 :: needs changes before merge. :: [P2] Persist inspection state outside rendered Markdown | [P3] State the OpenClaw Bay impact
  • reviewed 2026-08-07T14:21:16.768Z sha 96b73e3 :: needs changes before merge. :: [P2] Persist runner-owned checkout inspection status | [P3] State the OpenClaw Bay impact
  • reviewed 2026-08-07T15:09:32.925Z sha 96b73e3 :: needs changes before merge. :: [P2] Persist checkout status outside rendered Markdown | [P3] State the OpenClaw Bay impact

@clawsweeper clawsweeper Bot added status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Aug 4, 2026
@clawsweeper clawsweeper Bot added rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. proof: sufficient Contributor real behavior proof is sufficient. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. and removed rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. proof: sufficient Contributor real behavior proof is sufficient. labels Aug 6, 2026
@jesse-merhi jesse-merhi self-assigned this Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P1 Urgent regression or broken agent/channel workflow affecting real users now. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants