Skip to content

test(hermes): align restart provenance expectation - #8680

Merged
apurvvkumaria merged 2 commits into
mainfrom
codex/fix-8679-hermes-restart-test
Aug 10, 2026
Merged

test(hermes): align restart provenance expectation#8680
apurvvkumaria merged 2 commits into
mainfrom
codex/fix-8679-hermes-restart-test

Conversation

@apurvvkumaria

@apurvvkumaria apurvvkumaria commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

Summary

Align the Hermes restart-drift regression test with the explicit restart and health evidence added by #8679. This fixes the exact-result assertion that failed after the production behavior merged.

Related Issue

Follow-up to #8679 and #8671.

Changes

  • Expect restarted: true after the supervisor returns a valid restart marker.
  • Expect healthPassed: true after the recovered Hermes gateway passes its health check.
  • Preserve the assertions that no post-refusal mutations run and secret-like output remains redacted.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: This changes only an exact test expectation for behavior already merged in fix(hermes): preserve MCP rebuild restart evidence #8679; runtime behavior, CLI output, configuration, and recovery guidance are unchanged.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Codex security review passed with no findings; the test continues to require restart and health evidence, forbid post-refusal mutations, and verify redaction.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: The change is assertion-only and does not alter user-facing behavior or existing recovery guidance.
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit:
  • Station profile/scenario:
  • Result:
  • Supporting evidence:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: affected CLI suites passed 33/33 under Python 3.14; the test(rebuild): align stale recovery fixture with OpenShell 0.0.101 #8678 stale-recovery case passed 1/1 on the updated base; CLI type-check passed.
  • Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Apurv Kumaria akumaria@nvidia.com

Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
@apurvvkumaria apurvvkumaria self-assigned this Aug 10, 2026
@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The Hermes reconciliation refusal test now verifies that the gateway restart completes and health checks pass while preserving the existing failure details.

Changes

Hermes drift validation

Layer / File(s) Summary
Restart result assertion
src/lib/actions/sandbox/gateway-restart-hermes-drift.test.ts
The test expects restarted: true and healthPassed: true in the sanitized reconciliation refusal result.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Possibly related PRs

Suggested labels: release-target, v0.0.106

Suggested reviewers: cv

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly identifies the Hermes restart provenance test update and matches the main change.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/fix-8679-hermes-restart-test

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit feb2e02 in the codex/fix-8679-herme... branch remains at 96%, unchanged from commit 6812957 in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit feb2e02 in the codex/fix-8679-herme... branch remains at 81%, unchanged from commit 1a8e6bd in the main branch.

Show a code coverage summary of the most impacted files.
File main 1a8e6bd codex/fix-8679-herme... feb2e02 +/-
src/lib/inference/vllm.ts 88% 87% -1%
src/lib/onboard/command.ts 95% 94% -1%
src/lib/actions...all/run-plan.ts 85% 85% 0%
src/lib/onboard.ts 32% 32% 0%
src/lib/inference/local.ts 80% 81% +1%
src/lib/inferen...al-lifecycle.ts 86% 88% +2%
src/lib/inferen...e-provenance.ts 86% 90% +4%
src/lib/onboard/config-sync.ts 31% 100% +69%
src/lib/onboard...enclaw-setup.ts 17% 100% +83%
src/lib/inferen...ocal-network.ts 0% 95% +95%

Updated August 10, 2026 05:04 UTC

@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Model comparison: normalized findings match; normalized terminology decisions match; normalized E2E selections differ; severity counts match.
4 additional E2E selections from the second opinion

Advisory only. The primary lane did not select these E2E jobs or targets.

  • rebuild-hermes: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • rebuild-hermes-stale-base: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • state-backup-restore: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • managed-image-protected-runtime: The completed second-opinion lane identified E2E coverage that the primary lane omitted.

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite against this exact revision.

Recommended E2E: None

2 optional E2E recommendations
  • gateway-guard-recovery
  • mcp-bridge

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
@apurvvkumaria
apurvvkumaria enabled auto-merge (squash) August 10, 2026 05:08
@apurvvkumaria
apurvvkumaria merged commit 44e8bc4 into main Aug 10, 2026
66 of 67 checks passed
@apurvvkumaria
apurvvkumaria deleted the codex/fix-8679-hermes-restart-test branch August 10, 2026 05:10
@wscurran wscurran added chore Build, CI, dependency, or tooling maintenance area: sandbox OpenShell sandbox lifecycle, runtime, config, or recovery integration: hermes Hermes integration behavior labels Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: sandbox OpenShell sandbox lifecycle, runtime, config, or recovery chore Build, CI, dependency, or tooling maintenance integration: hermes Hermes integration behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants