Skip to content

fix: QA agent must always run 3 rounds in improve pipeline#792

Draft
colehurwitz wants to merge 2 commits into
mainfrom
factory/run-7112c066
Draft

fix: QA agent must always run 3 rounds in improve pipeline#792
colehurwitz wants to merge 2 commits into
mainfrom
factory/run-7112c066

Conversation

@colehurwitz

@colehurwitz colehurwitz commented Jun 26, 2026

Copy link
Copy Markdown
Collaborator

Closes #710

Changes

  • Updated Sacred Rule 9 in factory/agents/prompts/ceo.md to mandate exactly 3 QA rounds per experiment, regardless of verdict
  • CLEAN verdict: re-run QA without re-invoking Builder
  • ISSUES_FOUND verdict: re-invoke Builder to fix issues, then re-run QA
  • REVERT verdict: exit immediately — experiment is unsalvageable
  • Clarified closing sentence: "running fewer than 3 rounds without a REVERT exit" is the violation, resolving ambiguity with the legitimate REVERT early-exit path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Jun 26, 2026

Copy link
Copy Markdown

Sentrux Quality Report

Absolute

Scanning ....
[scan] git ls-files: 302 total, 290 kept, 12 dropped (ext:12, meta:0, big:0)
[build_project_map] 290 files, 52 unique dirs, 48 cache misses, 2.3ms
[resolve] 440 resolved, 771 unresolved (of 1211 total specs)
[resolve_imports] project_map 2.3ms, suffix_idx 0.5ms, suffix_resolve 9.5ms, total 12.3ms
[build_graphs] 290 files | maps 1.1ms, imports 12.4ms, calls+inherit 2.8ms, total 16.3ms | 439 import, 4379 call, 0 inherit edges
sentrux check — 2 rules checked

Quality: 4705

✗ [Error] max_cc: 5 function(s) exceed max cyclomatic complexity of 30
    factory/cli.py:cmd_ceo (cc=79)
    factory/study.py:study_project_local (cc=43)
    factory/cli.py:_welcome_wizard (cc=39)
    factory/cli.py:cmd_run (cc=37)
    factory/workflow/validation.py:validate_workflow (cc=31)

✗ 1 violation(s) found

Diff (vs base branch)

Scanning ....
[scan] git ls-files: 302 total, 290 kept, 12 dropped (ext:12, meta:0, big:0)
[build_project_map] 290 files, 52 unique dirs, 48 cache misses, 2.3ms
[resolve] 440 resolved, 771 unresolved (of 1211 total specs)
[resolve_imports] project_map 2.3ms, suffix_idx 0.6ms, suffix_resolve 9.4ms, total 12.4ms
[build_graphs] 290 files | maps 1.0ms, imports 12.4ms, calls+inherit 4.1ms, total 17.6ms | 439 import, 4379 call, 0 inherit edges
sentrux gate — structural regression check

Quality:      4705 -> 4705
Coupling:     0.75 → 0.75
Cycles:       4 → 4
God files:    0 → 0

Distance from Main Sequence: 0.35

✓ No degradation detected

@codecov

codecov Bot commented Jun 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.11%. Comparing base (527199d) to head (edc0346).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #792   +/-   ##
=======================================
  Coverage   87.11%   87.11%           
=======================================
  Files          81       81           
  Lines       12245    12245           
=======================================
  Hits        10667    10667           
  Misses       1578     1578           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix: QA agent must always run 3 rounds in improve pipeline

1 participant