fix(triage): honor repo readiness gates and avoid ready-for-dev label loops

[neubig]

Problem

The Extensions issue-triage agent repeatedly marked OpenHands/OpenHands#17510 ready-for-dev even though the repository's deterministic issue-readiness check rejected it: the issue is labeled bug, but its triage-generated Actual Behavior section contains only a screenshot/video placeholder, not the required embedded reproduction media. GitHub Actions removed the label; the triage automation reacted to that unlabeled event and added it back. Between 05:00 and 06:07 UTC on 2026-09-23 this one issue produced about 1,030 GitHub events and 190 triage runs, exhausting the OSS Canvas VM's memory.

The live OSS Canvas trigger now excludes ready-for-dev removals made by github-actions[bot] as an immediate circuit breaker. The Extensions automation still needs to make the right readiness decision itself, including on its initial opened or reopened event. Keep the fix in the existing issue-triage extension and reuse repository guidance/checks; do not introduce a second readiness mechanism.

Acceptance criteria

• Before applying ready-for-dev, the agent checks applicable repository-specific issue readiness rules, including the authoritative .github/scripts/check_issue_readiness.py gate when present. It does not treat placeholder text as required evidence.
• For a bug like OpenHands #17510 with no embedded reproduction media, triage may improve the issue and ask for the missing evidence, but leaves ready-for-dev absent and does not repeatedly rewrite the managed section or comment.
• A deterministic workflow's removal of ready-for-dev does not cause the triage automation to immediately reapply the label on an unchanged issue. A later substantive author edit or supplied evidence can be reconsidered.
• A live or replayed before/after test shows no label loop and at most one triage conversation for an unchanged issue; an issue meeting the repository's criteria still becomes ready.

Observed event actors: all-hands-bot added the label; github-actions[bot] removed it. The current Extensions triage prompt is in skills/github-issue-triage/scripts/worker.py.

---

OpenHands AI triage

The following comments and acceptance criteria were added by the OpenHands AI agent.

Triage

The reported loop comes from a readiness disagreement, not from a missing trigger:
skills/github-issue-triage/scripts/worker.py vets an issue with its own prompt
criteria and then applies ready-for-dev, while a target repository such as
OpenHands/OpenHands evaluates the same label with its own deterministic gate
(.github/scripts/check_issue_readiness.py, invoked by
.github/workflows/issue-readiness-check.yml). That gate infers the issue type
from body sections and requires, for a bug, embedded screenshot/video media in
### Actual Behavior; a triage-written placeholder such as
_no reproduction media attached yet_ fails it. The workflow therefore removes
the label, the removal event re-triggers triage, and each cycle rewrites the
managed section and the readiness feedback comment, changing the next delivery
digest and re-dispatching - the ~1,030 events / 190 runs reported for
OpenHands/OpenHands#17510. This repository's own Ready-for-dev Label Policy
workflow only checks the applying actor's permission, so the fix has to live in
the extension and make the extension defer to the target repository's rules.

Bounded scope: keep the change inside skills/github-issue-triage
(worker.py, its prompt, SKILL.md/README.md, the catalog bundle, and the
contract tests) and make the run consult the target repository's existing
readiness rules before granting ready-for-dev, reuse that repository's
authoritative check instead of restating its rules, and remain idempotent on an
unchanged issue so a deterministic label removal cannot cause an immediate
reapplication.

Non-goals: no second readiness mechanism or re-derived copy of the repository's
criteria; no change to the target repository's readiness workflow or to the
host-side automation trigger / circuit breaker (owned outside this repository);
no new event trigger in the catalog manifest; no edits to unrelated skills,
plugins, or automations; no code implementation or PR review.

Acceptance Criteria

• Before applying ready-for-dev, the triage run consults the target repository's own readiness rules and applies the label only when those rules pass, reusing the repository's authoritative gate when present (for example .github/scripts/check_issue_readiness.py) rather than re-deriving readiness inside the extension.
• Readiness evidence is judged by what the repository gate accepts: descriptive or placeholder text such as _no reproduction media attached yet_ does not satisfy a bug report's required screenshot/video, and a body that would fail the gate is not marked ready.
• When the repository gate fails, the run leaves ready-for-dev absent while it may still record the managed triage section and/or a marked comment requesting the missing evidence; it never applies the label on its own criteria alone.
• A re-run on an unchanged issue that already carries a current managed section or marked comment produces no new conversation, no section rewrite, and no label change, including when the only intervening event was a deterministic ready-for-dev removal.
• After a deterministic workflow removes ready-for-dev from an unchanged issue, the triage automation does not reapply it; a later substantive author edit, newly supplied evidence, or new human discussion re-enables reassessment and can make the issue ready.
• An issue that does satisfy the repository gate (a bug with qualifying reproduction media, or an enhancement with a non-empty Desired Behavior and an Acceptance Criteria checklist) still receives ready-for-dev along with its priority label.
• Automated coverage reproduces the reported scenario: a test that fails before the change and asserts a single delivery for an unchanged issue whose gate fails, and asserts that ready-for-dev is not applied; the existing triage delivery contract tests continue to pass (for example uv run pytest -q tests/test_github_triage_delivery.py).
• A rooted live or replayed before/after demonstration is provided showing no label loop and at most one triage conversation for an unchanged issue, and readiness for an issue that meets the criteria (the issue's fourth acceptance criterion).
• If the managed-section format or the delivery-digest inputs change, TRIAGE_FORMAT_VERSION is bumped so pre-existing marked sections are re-evaluated at most once and then stabilize, and legacy comment digests remain tolerated.
• Because bundle files change, the automation's setup.bundle.version (and the entry version) in automations/catalog/github-issue-triage/manifest.json are bumped and derived artifacts stay in sync (python3 scripts/sync_extensions.py --check clean), and skills/github-issue-triage/SKILL.md and README.md describe the readiness-gate behavior.
• The change stays within the configured repository and issue scope using the existing fine-grained token: no new required permissions, no token value printed, and no mutation of repositories or issues outside scope.

[neubig]

@all-hands-bot Please implement the accepted issue. Keep the change in the existing issue-triage extension, and include a regression for the OpenHands #17510 readiness-loop case.

[all-hands-bot]

Implemented in the existing github-issue-triage extension. Two changes in skills/github-issue-triage/scripts/worker.py, plus a regression for the OpenHands/OpenHands#17510 readiness loop.

What changed

1. Readiness is now the repository's own gate, not a second rule set here.
Before delegating an issue, the scanner reads the target repository's authoritative readiness gate (.github/scripts/check_issue_readiness.py, plus the sibling modules in .github/scripts/ that it imports) from the repository via the contents API, runs it against the issue's current body with the same --body-file / --json interface the repository's workflow uses, and passes the parsed verdict to the agent as the single readiness decision path:

"readiness": {"available": true, "ready": false, "reasons": [...], "issue_type": "bug"}

The prompt now gates ready-for-dev on readiness.ready when available is true, explicitly treats placeholder text (e.g. an ### Actual Behavior holding only Screenshot/video placeholder ... _no reproduction media attached yet_.) as missing evidence, and allows at most one evidence-request comment when the verdict withholds the label. When the repository ships no gate, available is false and the existing scope-and-criteria standard takes over. The extension contains no readiness rules of its own — it runs whatever script the target repository ships, so changing that script changes the verdict for an unchanged body.

2. A deterministic bot removal of ready-for-dev no longer re-opens triage.
run() now ignores an unlabeled event whose label is ready-for-dev and whose sender is a *[bot], logging disposition: "ignored-bot-ready-for-dev-removal" and returning before any delivery. Non-bot removals, author edits, and new evidence are still processed. Combined with the existing body/comment digest, an unchanged issue produces at most one conversation across repeated sweeps.

The gate subprocess runs repository-supplied code with the GitHub token (and GITHUB_TOKEN/GH_TOKEN) stripped from its environment, since it only needs to read a body file and print a verdict.

Supporting changes: prompt version bumped to TRIAGE_FORMAT_VERSION = 3; SKILL.md and README.md document the gate and the bot-removal short-circuit; the catalog bundle version is bumped to 1.4.0 and automations/bundle-index.js regenerated.

Regression coverage (tests/test_github_triage_delivery.py)

A GATE_SOURCE fixture stands in for the repository's gate and is served through the real gh contents path, so readiness() reads and executes it exactly as in production (no mocking of the runner):

• test_triage_runs_repository_readiness_gate_and_withholds_for_placeholder — the #17510 shape (bug label, placeholder-only ### Actual Behavior, populated ### Acceptance Criteria) yields "ready": false.
• test_readiness_gate_verdict_follows_the_served_script — flipping the served script's verdict flips the decision, proving the rules are read from the repository rather than hard-coded.
• test_readiness_falls_back_when_repository_ships_no_gate — {"available": false} when no gate exists.
• test_readiness_gate_does_not_receive_credentials — the gate process sees no GitHub token.
• test_bot_ready_for_dev_removal_does_not_reopen_triage — a github-actions[bot] unlabeled event does not deliver.
• test_author_edit_after_bot_removal_is_still_reconsidered and test_nonbot_ready_for_dev_removal_is_still_processed — legitimate changes still go through.
• test_placeholder_issue_loop_produces_at_most_one_delivery — repeated sweeps for the unchanged #17510-shaped issue deliver once.

I also verified the implementation against the real OpenHands/OpenHands check_issue_readiness.py: the placeholder-only body returns ready: false with the "Actual Behavior section must include a screenshot or video" reason, and a body with embedded media plus a populated Acceptance Criteria returns ready: true.

uv run pytest -q passes (993 passed, 23 skipped), including all triage contract tests and the catalog/skills/automation codegen checks.

Changes are uncommitted in my working copy; no commits pushed and no PR opened.

Generated by OpenHands AI on behalf of the user.