Skip to content

refactor(e2e): consolidate more retained profile jobs - #8966

Merged
cv merged 3 commits into
mainfrom
codex/consolidate-more-retained-e2e
Aug 13, 2026
Merged

refactor(e2e): consolidate more retained profile jobs#8966
cv merged 3 commits into
mainfrom
codex/consolidate-more-retained-e2e

Conversation

@cv

@cv cv commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

Three E2E targets repeated an execution profile in dedicated workflow jobs. This change moves them into the existing target catalogue and reusable profile while preserving their runner, credential, setup, timeout, environment, and evidence contracts.

Related Issue

Refs #7912

Changes

  • Move hermes-slack and sandbox-operations into the NVIDIA inference profile.
  • Move openclaw-inference-switch into the standard profile. Its one Anthropic-compatible mode is now target environment instead of a one-row job matrix.
  • Remove the three dedicated jobs and their duplicate boundary checks. Planner tests preserve each migrated target's execution contract and prove the IDs no longer belong to the retained-job inventory.
  • Make the shared profile validator reject extra job settings, environment, steps, and artifact path overrides.
  • Pin every migrated target field in one complete catalogue contract test.
  • Keep skill-agent dedicated because its explicit artifact allowlist is a distinct evidence boundary.
  • Document one logical target catalogue that the planner partitions into three credential-profile matrices. Retained jobs remain exceptions for distinct setup, evidence, credential, or multi-job boundaries.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification:
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Focused contracts verify profile credential isolation, exact checkout and CLI restoration, credential-free installation, target environment, runner and timeout selection, evidence creation, artifact upload, and Docker credential cleanup. The catalogue retains the sandbox isolation rationale beside its open-policy setting.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: docs-updated
  • Evidence: test/e2e/README.md documents the single target catalogue, its three credential-profile matrices, and the criteria for retained workflow jobs. The reviewer verified the follow-up against .github/workflows/e2e-standard-profile.yaml and tools/e2e/standard-profile-workflow-boundary.mts. No public documentation changes are needed.
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit:
  • Station profile/scenario:
  • Result:
  • Supporting evidence:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: Nine focused E2E-support files passed 174/174 tests. The follow-up shared-profile and workflow-boundary tests passed 37/37 tests. The final workflow-plan target contract passed 59/59 tests. The integration watch-trigger contract passed 5/5 tests. npm run test:projects:check, npm run checks:repository, Markdown lint, Biome lint, and git diff --check passed. The CLI build and type check passed. Normal pre-commit, commit-msg, and pre-push hooks passed.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result: Not run locally by maintainer direction; GitHub CI owns the broad suite.
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Carlos Villela cvillela@nvidia.com

Summary by CodeRabbit

  • Documentation

    • Clarified how end-to-end targets are grouped across credential-profile test matrices.
    • Documented exceptions for targets requiring specialized setup, evidence handling, or credentials.
  • Tests

    • Updated end-to-end coverage for the revised target catalog and workflow structure.
    • Strengthened validation of workflow steps, environment settings, and target-derived artifact paths.
    • Adjusted expectations for token rotation and dispatch compatibility.
  • Chores

    • Consolidated end-to-end workflow execution and removed obsolete standalone validation jobs.
    • Retained covered targets within the appropriate test matrices.

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@cv cv self-assigned this Aug 13, 2026
@github-code-quality

github-code-quality Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 47640b4 in the codex/consolidate-mo... branch remains at 96%, unchanged from commit 7aba3cc in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 47640b4 in the codex/consolidate-mo... branch remains at 82%, unchanged from commit 7aba3cc in the main branch.

Show a code coverage summary of the most impacted files.
File main 7aba3cc codex/consolidate-mo... 47640b4 +/-
src/lib/cua/bounded-file.ts 94% 84% -10%
src/lib/cua/run...ime-manifest.ts 91% 84% -7%
src/lib/cua/contract.ts 87% 80% -7%
src/lib/messagi.../ilink-login.ts 96% 89% -7%
src/lib/onboard...file-builder.ts 95% 91% -4%
src/lib/state/o...d-checkpoint.ts 90% 87% -3%
src/lib/onboard...-diagnostics.ts 94% 96% +2%
src/lib/onboard...press-resume.ts 78% 82% +4%
src/lib/agent/dashboard-ui.ts 91% 98% +7%
src/lib/onboard...der/snapshot.ts 75% 83% +8%

Updated August 13, 2026 09:56 UTC

@coderabbitai

coderabbitai Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: acd63356-8fab-41e6-a574-285d9e2ed771

📥 Commits

Reviewing files that changed from the base of the PR and between d75e902 and 47640b4.

📒 Files selected for processing (1)
  • test/e2e/support/workflow-plan.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • test/e2e/support/workflow-plan.test.ts

📝 Walkthrough

Walkthrough

The PR removes three standalone E2E jobs and routes their targets through catalogue-managed workflow matrices. It narrows related validators and updates boundary tests, documentation, source-shape exceptions, and watch-trigger expectations.

Changes

E2E target catalogue and planning

Layer / File(s) Summary
Catalogue targets and workflow plan
tools/e2e/target-catalogue.mts, test/e2e/support/workflow-plan.test.ts, test/e2e/README.md
Adds catalogue definitions for hermes-slack, openclaw-inference-switch, and sandbox-operations. The target factory accepts explicit test paths. Plan tests and documentation describe matrix placement and retained-job rules.

Workflow execution and validation

Layer / File(s) Summary
Standalone job removal and validator narrowing
.github/workflows/e2e.yaml, tools/e2e/inference-switch-workflow-boundary.mts, tools/e2e/upload-e2e-artifacts-workflow-boundary.mts, tools/e2e/standard-profile-workflow-boundary.mts, tools/e2e/workflow-boundary.mts
Removes the three standalone jobs and their report dependencies. Retains OpenClaw EXDEV lifecycle dependencies. Removes sandbox validation, narrows inference-switch and artifact-upload contracts, and strengthens standard-profile validation.

Contract coverage alignment

Layer / File(s) Summary
Boundary tests and trigger expectations
test/e2e/support/*, test/vitest-watch-triggers.test.ts, ci/source-shape-test-budget.json
Removes obsolete standalone-job and artifact-path cases. Updates remaining tests to use token-rotation and Hermes-only inference-switch validation. Removes the obsolete source-shape exception and sandbox watch-trigger entry.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Mergeability Score: ⚪ Minimal · up to 47640

The PR consolidates retained E2E jobs while preserving their documented execution and evidence contracts; no actionable merge-blocking risk remains after normal checks and review.

Possibly related PRs

Suggested labels: area: e2e, integration: openclaw, integration: hermes

Suggested reviewers: prekshivyas

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: consolidating retained E2E profile jobs and moving targets into reusable workflow profiles.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/consolidate-more-retained-e2e

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 2 blockers · 1 warning · 0 suggestions
  • Model comparison: normalized findings differ; normalized terminology decisions differ; normalized E2E selections match; Nemotron reported 2 more blockers, 1 more warning, the same number of suggestions.
3 terminology differences from the second opinion

Advisory only. These are normalized differences from the primary terminology receipt.

  • logical target set at test/e2e/README.md:219: primary classified it as justified; the second opinion classified it as define.
  • catalogue shape at test/e2e/README.md:239: selected only by the second-opinion lane as define.
  • exception at test/e2e/README.md:239: selected only by the second-opinion lane as define.

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

3 semantic terminology decisions

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

  • established — credential profile at test/e2e/README.md:220: Keep credential profile for the three matrix classifications.
  • established — retained workflow job at test/e2e/README.md:239: Keep retained workflow job for jobs outside the catalogue execution shape.
  • justified — logical target set at test/e2e/README.md:219: Keep logical target set because it distinguishes the catalogue source from generated matrices.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite for the commit under review.

Recommended E2E: inference-routing

Manual-only E2E: cloud-onboard, security-posture, cloud-inference, network-policy
The manual PR workflow does not run these selectors for the commit under review. Run them from reviewed code on main.

3 optional E2E recommendations
  • hermes-slack
  • openclaw-inference-switch
  • sandbox-operations

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

cv added 2 commits August 13, 2026 02:31
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant