Skip to content

feat(profiles): add aig to the production clawhub profile - #48

Merged
Patrick-Erichsen merged 3 commits into
openclaw:mainfrom
boy-hack:feat/clawhub-profile-add-aig
Aug 20, 2026
Merged

feat(profiles): add aig to the production clawhub profile#48
Patrick-Erichsen merged 3 commits into
openclaw:mainfrom
boy-hack:feat/clawhub-profile-add-aig

Conversation

@boy-hack

Copy link
Copy Markdown
Contributor

Summary

  • What changed: The production clawhub profile now runs aig alongside
    skillspector and clawscan-static (previously aig was only evaluated
    through the unpublished clawhub-aig candidate profile). RenderClawHubPrompt
    now attaches AIG SARIF evidence to the Codex judge whenever aig produced a
    result, instead of keying off artifact.Profile == "clawhub-aig", so
    production runs get the same evidence the candidate profile already
    received.

Security / Trust Impact

  • Security/trust impact explained

aig now runs as part of the production clawhub profile, requiring
LLM_API_KEY (or OPENAI_API_KEY) at scan time — already listed as an
optional/required env for the aig scanner adapter and added to this
profile's sandbox.env. RenderClawHubPrompt's AIG-evidence attachment is
now driven by scanner-result presence (clawHubAIGAnalysis(artifact) != nil)
rather than the profile id string, so the same evidence path that
clawhub-aig already exercised now also fires for clawhub. The
malicious-signal heuristic (clawHubHasMaliciousSignal) is unchanged. The
clawhub-aig candidate profile is left in place for isolated evaluation.

Verification

  • go test -count=1 ./...
  • go vet ./...
  • go run ./cmd/clawscan --help
  • Focused scanner/benchmark/manual proof: go run ./cmd/clawscan profiles -v
    confirms the clawhub profile lists skillspector, clawscan-static, aig;
    added TestRenderClawHubPromptIncludesAIGEvidenceForProductionProfile to
    cover AIG evidence reaching the Codex judge prompt under the production
    clawhub profile label.

The clawhub profile now runs skillspector, clawscan-static, and aig
together instead of evaluating aig only through the clawhub-aig
candidate profile. RenderClawHubPrompt now attaches AIG SARIF evidence
to the Codex judge whenever aig produced a result, rather than keying
off the profile id, so production runs get the same evidence the
clawhub-aig candidate already received. The clawhub-aig candidate
profile and the malicious-signal heuristic are left unchanged.
@clawsweeper

clawsweeper Bot commented Aug 18, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

@clawsweeper clawsweeper Bot added merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P1 Urgent regression or broken agent/channel workflow affecting real users now. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Aug 18, 2026
@clawsweeper

clawsweeper Bot commented Aug 18, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed August 20, 2026, 4:37 PM ET / 20:37 UTC.

ClawSweeper review

What this changes

The PR adds AIG to the production ClawHub profile, forwards its SARIF output to the Codex judge, and maps existing OpenAI/Codex credentials to AIG’s OpenAI runtime defaults.

Merge readiness

⚠️ Ready for maintainer review - 3 items remain

Keep open: current main still has the two-scanner production profile and only attaches AIG evidence for the candidate profile. The branch is a coherent, tested production rollout, but enabling an additional credentialed remote scanner by default remains a maintainer product and operations decision.

Priority: P2
Reviewed head: 207defc74b684c4bc362b149f1413c12ae1b956c
Owner decision: Required. See Decision needed.

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) A focused implementation with targeted coverage and credible after-fix terminal evidence; the remaining gate is intentional production policy, not patch correctness.
Proof confidence 🐚 platinum hermit (4/6) Sufficient (terminal): The collaborator’s PR comment supplies after-fix terminal profile output and reports a fixture-backed production-profile run where AIG SARIF reached a completed judge result.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Verified Sufficient (terminal): The collaborator’s PR comment supplies after-fix terminal profile output and reports a fixture-backed production-profile run where AIG SARIF reached a completed judge result.
Evidence reviewed 7 items Current main does not implement the rollout: Current main lists only SkillSpector and ClawScan Static in the production profile, while AIG remains limited to the separate candidate profile.
Branch implements the production profile change: The PR head adds AIG and the required AIG runtime environment names to the production profile.
Branch passes AIG evidence by result presence: The judge prompt now appends AIG SARIF evidence whenever an AIG result is present, rather than requiring the candidate profile identifier.
Findings None None.
Security None None.

How this fits together

ClawScan’s ClawHub profile chooses scanners and a sandbox environment for a security scan, then supplies scanner evidence to a Codex judge. This change makes AIG part of that production scanner set and includes its SARIF findings in the final judging prompt.

flowchart LR
  A[ClawHub skill or plugin] --> B[Production ClawHub profile]
  B --> C[Sandboxed scanners]
  C --> D[AIG SARIF evidence]
  D --> E[Codex judge prompt]
  E --> F[Security verdict]
Loading

Decision needed

Question Recommendation
Should the production ClawHub profile make AIG’s credentialed remote analysis mandatory for its default scanner suite? Approve production AIG rollout: Merge the branch and explicitly accept AIG’s additional credentialed scan as part of the standard ClawHub security path.

Why: The patch is technically bounded and preserves existing credential compatibility, but selecting the default production scanner set determines ongoing provider, latency, and cost policy.

Before merge

  • Resolve merge risk (P1) - Merging makes AIG a default production scanner and sends the shared Codex/OpenAI credential through its sandboxed runtime, adding provider requests, latency, and cost to every eligible profile run.
  • Complete next step (P2) - A maintainer must choose whether the production profile should adopt the additional credentialed remote scanner as its default.
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Runtime and test delta runtime/profile +42/-2, tests +134/-9, docs +8/-6 The implementation adds a small credential/profile path with substantially more focused coverage than runtime code.

Merge-risk options

Maintainer options:

  1. Approve the documented production expansion (recommended)
    Merge after recording that AIG may receive the existing Codex/OpenAI credential and add a provider request to default ClawHub scans.
  2. Retain the candidate profile
    Pause this PR if AIG should remain an explicit evaluation path rather than a production default.

Technical review

Best possible solution:

Approve the production rollout only if ClawHub intends AIG’s credentialed remote analysis to be part of its default security suite; otherwise retain the evaluated candidate profile as the opt-in path.

Do we have a high-confidence way to reproduce the issue?

Not applicable as a bug report: this is a production-profile feature change, and the PR discussion supplies a terminal profile check plus a reported fixture-backed scan and judge receipt.

Is this the best way to solve the issue?

Yes technically: result-based evidence attachment and AIG-only credential fallback are narrower than profile-name coupling or a shared SkillSpector fallback, subject to the production-suite decision.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against 113c1a8ede9f.

Labels

Label changes:

  • add P2: This is a bounded production-profile improvement with meaningful but non-urgent operational impact.
  • add proof: sufficient: Contributor real behavior proof is sufficient. The collaborator’s PR comment supplies after-fix terminal profile output and reports a fixture-backed production-profile run where AIG SARIF reached a completed judge result.
  • add rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The collaborator’s PR comment supplies after-fix terminal profile output and reports a fixture-backed production-profile run where AIG SARIF reached a completed judge result.
  • remove rating: 🧂 unranked krab: Current PR rating is rating: 🐚 platinum hermit, so this older rating label is no longer current.
  • remove status: 📣 needs proof: Current PR status label is status: 👀 ready for maintainer look.
  • remove P1: Current review triage priority is P2, so this older priority label is no longer current.

Label justifications:

  • P2: This is a bounded production-profile improvement with meaningful but non-urgent operational impact.
  • merge-risk: 🚨 compatibility: Existing production scans gain a third scanner and new runtime defaults, changing execution behavior and provider use.
  • merge-risk: 🚨 security-boundary: The profile now allowlists AIG’s credential and provider settings and can derive an AIG credential from CODEX_API_KEY.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The collaborator’s PR comment supplies after-fix terminal profile output and reports a fixture-backed production-profile run where AIG SARIF reached a completed judge result.
  • proof: sufficient: Contributor real behavior proof is sufficient. The collaborator’s PR comment supplies after-fix terminal profile output and reports a fixture-backed production-profile run where AIG SARIF reached a completed judge result.

Evidence

What I checked:

Likely related people:

  • Patrick-Erichsen: Authored the recent ClawHub judge authentication work and the current PR-head credential-isolation repair. (role: recent profile contributor and repair author; confidence: high; commits: e20fc9f2c0d5, 207defc74b68; files: internal/profiles/clawhub/clawscan.yml, internal/runner/runner.go)
  • Vito Cappello: Introduced the earlier OpenClaw plugin support across the ClawHub profile and associated runner path. (role: original ClawHub profile feature contributor; confidence: medium; commits: a37f49f1b09f; files: internal/profiles/clawhub/clawscan.yml, internal/runner/runner.go)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Record the maintainer decision on making AIG part of the default production scanner suite.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (2 earlier review cycles)
  • reviewed 2026-08-18T06:59:09.028Z sha 7416e78 :: needs real behavior proof before merge. :: [P1] Preserve CODEX_API_KEY-only profile runs | [P3] Update the profile description's scanner count
  • reviewed 2026-08-18T07:34:08.294Z sha 1a0e00b :: needs real behavior proof before merge. :: [P1] Keep the derived AIG key out of SkillSpector

…ents

The clawhub profile's Codex judge command already accepts CODEX_API_KEY
as an OPENAI_API_KEY substitute, but aig's own credential requirement
only recognized LLM_API_KEY/OPENAI_API_KEY, so promoting aig into the
production profile broke existing CODEX_API_KEY-only deployments before
any scanner could run. applyRuntimeEnvDefaults now backfills
OPENAI_API_KEY from CODEX_API_KEY when aig is requested and no other
key is set, mirroring the existing SkillSpector provider default.

Also fixes a stale "runs both scanners" description now that the
clawhub profile lists three scanners.
@clawsweeper clawsweeper Bot added the merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. label Aug 18, 2026
@Patrick-Erichsen
Patrick-Erichsen requested a review from a team as a code owner August 20, 2026 20:33
@Patrick-Erichsen

Copy link
Copy Markdown
Collaborator

Maintainer repair and proof at 207defc:

  • Scoped the CODEX_API_KEY fallback to AIG's LLM_API_KEY, so it cannot enable SkillSpector LLM mode.
  • Verified upstream AIG 0.2.1 defaults to OpenRouter/DeepSeek. When the source credential is OPENAI_API_KEY or CODEX_API_KEY, ClawScan now selects https://api.openai.com/v1 and gpt-5.5; an explicit LLM_API_KEY retains AIG's documented provider defaults.
  • Added DEFAULT_MODEL and DEFAULT_BASE_URL to both ClawHub profiles' sandbox allowlists.

Validation:

$ TMPDIR="$(realpath "$TMPDIR")" go test -count=1 ./...
ok github.com/openclaw/clawscan/cmd/clawscan
ok github.com/openclaw/clawscan/cmd/verify-clawhub-prompt
ok github.com/openclaw/clawscan/internal/clawhubprompt
ok github.com/openclaw/clawscan/internal/installpolicy
ok github.com/openclaw/clawscan/internal/profiles
ok github.com/openclaw/clawscan/internal/runner
ok github.com/openclaw/clawscan/schemas
ok github.com/openclaw/clawscan/scripts/update-benchmark-readme
ok github.com/openclaw/clawscan/scripts/update-skilltrustbench-baseline

$ go vet ./...
# clean

$ go run ./cmd/clawscan profiles -v
clawhub scanners: skillspector, clawscan-static, aig
clawhub sandbox env: OPENAI_API_KEY, CODEX_API_KEY, SKILLSPECTOR_PROVIDER, LLM_API_KEY, DEFAULT_MODEL, DEFAULT_BASE_URL

Fixture-backed production-profile execution also completed all three scanner entries and the judge receipt, with AIG SARIF ruleId: T04 reaching the judge result (status: completed). The final independent autoreview reported no actionable findings (correctness confidence 0.91).

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Aug 20, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. P1 Urgent regression or broken agent/channel workflow affecting real users now. labels Aug 20, 2026
@Patrick-Erichsen

Copy link
Copy Markdown
Collaborator

Owner decision: approve the production AIG rollout.

We accept AIG as a default credentialed scanner in the ClawHub profile, including the additional provider request, latency, and cost. The reviewed provider-safe defaults at 207defc are required for this approval.

@Patrick-Erichsen
Patrick-Erichsen merged commit f3ac276 into openclaw:main Aug 20, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. merge-risk: 🚨 security-boundary 🚨 Merging this PR could weaken sandboxing, authorization, credentials, or sensitive data. P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants