refactor(sandbox): give rebuild test harnesses one shared core - #8581
Conversation
Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe rebuild-flow test infrastructure now has a shared harness core with separate generic and DCode profiles. Test suites use the profile modules and centralized lifecycle hooks. The obsolete harness is removed from the ChangesRebuild harness refactor
Estimated code review effort: 4 (Complex) | ~45 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Comment |
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage in commit ed3c73c in the TypeScript / code-coverage/cliThe overall coverage in commit ed3c73c in the Show a code coverage summary of the most impacted files.
Updated |
There was a problem hiding this comment.
🧹 Nitpick comments (2)
test/helpers/rebuild-flow-harness.ts (1)
107-109: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAdd mock-history cleanup to the harness.
If Vitest does not enable
clearMocks, callvi.clearAllMocks()beforevi.restoreAllMocks()so each test starts with empty mock history.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/helpers/rebuild-flow-harness.ts` around lines 107 - 109, Update the afterEach cleanup in the rebuild-flow harness to call vi.clearAllMocks() before vi.restoreAllMocks(), ensuring mock history is reset between tests while preserving the existing purgeRebuildModule() cleanup.Source: Coding guidelines
test/helpers/rebuild-flow-dcode-harness.ts (1)
171-720: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚖️ Poor tradeoffCentralize common mock setup in
rebuild-flow-harness.ts.The shared core defines no direct spies. The generic and DCode harnesses independently define the same console, session, registry, Docker, and lifecycle spies. This duplication can cause divergent test behavior.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/helpers/rebuild-flow-dcode-harness.ts` around lines 171 - 720, Centralize the duplicated console, session, registry, Docker, and lifecycle spy setup in the shared rebuild-flow harness used by createRebuildFlowHarness. Update the generic and DCode harnesses to reuse those shared spies and return values instead of defining independent mocks, while preserving existing override behavior and spy handles.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@test/helpers/rebuild-flow-dcode-harness.ts`:
- Around line 171-720: Centralize the duplicated console, session, registry,
Docker, and lifecycle spy setup in the shared rebuild-flow harness used by
createRebuildFlowHarness. Update the generic and DCode harnesses to reuse those
shared spies and return values instead of defining independent mocks, while
preserving existing override behavior and spy handles.
In `@test/helpers/rebuild-flow-harness.ts`:
- Around line 107-109: Update the afterEach cleanup in the rebuild-flow harness
to call vi.clearAllMocks() before vi.restoreAllMocks(), ensuring mock history is
reset between tests while preserving the existing purgeRebuildModule() cleanup.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 89684682-10c1-4ddb-805e-667e52d52a62
📒 Files selected for processing (23)
scripts/checks/test-create-require-budget.mtssrc/lib/actions/sandbox/rebuild-base-image-resolution-flow.test.tssrc/lib/actions/sandbox/rebuild-dcode-artifact-drift.test.tssrc/lib/actions/sandbox/rebuild-dcode-base-image-lease.test.tssrc/lib/actions/sandbox/rebuild-dcode-mutation-edge.test.tssrc/lib/actions/sandbox/rebuild-dcode-pre-delete-drift.test.tssrc/lib/actions/sandbox/rebuild-dcode-preflight.test.tssrc/lib/actions/sandbox/rebuild-dcode-recovered-provider.test.tssrc/lib/actions/sandbox/rebuild-dcode-recovery.test.tssrc/lib/actions/sandbox/rebuild-flow-credential-preflight.test.tssrc/lib/actions/sandbox/rebuild-flow-lifecycle.test.tssrc/lib/actions/sandbox/rebuild-flow-recovery.test.tssrc/lib/actions/sandbox/rebuild-flow-target-credentials.test.tssrc/lib/actions/sandbox/rebuild-flow-target-image.test.tssrc/lib/actions/sandbox/rebuild-flow-target-session.test.tssrc/lib/actions/sandbox/rebuild-hermes-post-restore.test.tssrc/lib/actions/sandbox/rebuild-local-provider-recreate.test.tssrc/lib/actions/sandbox/rebuild-prepared-recovery.test.tstest/helpers/rebuild-dcode-flow-helpers.tstest/helpers/rebuild-flow-dcode-harness.tstest/helpers/rebuild-flow-generic-harness.tstest/helpers/rebuild-flow-harness.tstest/rebuild-stale-recovery.test.ts
💤 Files with no reviewable changes (1)
- scripts/checks/test-create-require-budget.mts
PR Review Advisor — No blocking findings reportedAdvisor assessment: No blocking advisor findings reported Model lanes
3 additional E2E selections from the second opinionAdvisory only. The primary lane did not select these E2E jobs or targets.
Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests. E2E guidanceAdvisory only. A maintainer can dispatch the default E2E suite against this exact revision. Recommended E2E: This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge. |
|
I reviewed the two automated maintainability suggestions against the repository’s current Vitest configuration and the profile-specific fixtures.
Focused verification is green: stale-recovery 5/5, DCode preflight 15/15 with the repository’s slow-environment timeout override, and the generic target-session case 1/1. All required GitHub gates currently pass; the remaining Nemotron failure is advisory-service behavior, not a product-test failure. The branch still needs a current-main refresh and independent approval before merge. |
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/helpers/rebuild-flow-harness.ts`:
- Around line 101-106: Update the beforeEach setup around
acceptThirdPartySoftware to explicitly delete
process.env.NEMOCLAW_ACCEPT_THIRD_PARTY_SOFTWARE when the option is false, while
retaining the existing "1" assignment when it is true. Keep
restoreRebuildFlowEnv responsible for restoring the caller’s original
environment.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: dff43337-bc16-48aa-abc8-209bbfa5a31b
📒 Files selected for processing (23)
scripts/checks/test-create-require-budget.mtssrc/lib/actions/sandbox/rebuild-base-image-resolution-flow.test.tssrc/lib/actions/sandbox/rebuild-dcode-artifact-drift.test.tssrc/lib/actions/sandbox/rebuild-dcode-base-image-lease.test.tssrc/lib/actions/sandbox/rebuild-dcode-mutation-edge.test.tssrc/lib/actions/sandbox/rebuild-dcode-pre-delete-drift.test.tssrc/lib/actions/sandbox/rebuild-dcode-preflight.test.tssrc/lib/actions/sandbox/rebuild-dcode-recovered-provider.test.tssrc/lib/actions/sandbox/rebuild-dcode-recovery.test.tssrc/lib/actions/sandbox/rebuild-flow-credential-preflight.test.tssrc/lib/actions/sandbox/rebuild-flow-lifecycle.test.tssrc/lib/actions/sandbox/rebuild-flow-recovery.test.tssrc/lib/actions/sandbox/rebuild-flow-target-credentials.test.tssrc/lib/actions/sandbox/rebuild-flow-target-image.test.tssrc/lib/actions/sandbox/rebuild-flow-target-session.test.tssrc/lib/actions/sandbox/rebuild-hermes-post-restore.test.tssrc/lib/actions/sandbox/rebuild-local-provider-recreate.test.tssrc/lib/actions/sandbox/rebuild-prepared-recovery.test.tstest/helpers/rebuild-dcode-flow-helpers.tstest/helpers/rebuild-flow-dcode-harness.tstest/helpers/rebuild-flow-generic-harness.tstest/helpers/rebuild-flow-harness.tstest/rebuild-stale-recovery.test.ts
💤 Files with no reviewable changes (1)
- scripts/checks/test-create-require-budget.mts
🚧 Files skipped from review as they are similar to previous changes (21)
- src/lib/actions/sandbox/rebuild-dcode-base-image-lease.test.ts
- src/lib/actions/sandbox/rebuild-flow-recovery.test.ts
- src/lib/actions/sandbox/rebuild-flow-target-session.test.ts
- test/rebuild-stale-recovery.test.ts
- src/lib/actions/sandbox/rebuild-flow-target-image.test.ts
- src/lib/actions/sandbox/rebuild-flow-credential-preflight.test.ts
- src/lib/actions/sandbox/rebuild-flow-target-credentials.test.ts
- src/lib/actions/sandbox/rebuild-dcode-preflight.test.ts
- src/lib/actions/sandbox/rebuild-flow-lifecycle.test.ts
- src/lib/actions/sandbox/rebuild-dcode-pre-delete-drift.test.ts
- src/lib/actions/sandbox/rebuild-dcode-mutation-edge.test.ts
- src/lib/actions/sandbox/rebuild-dcode-artifact-drift.test.ts
- src/lib/actions/sandbox/rebuild-dcode-recovery.test.ts
- src/lib/actions/sandbox/rebuild-prepared-recovery.test.ts
- src/lib/actions/sandbox/rebuild-base-image-resolution-flow.test.ts
- test/helpers/rebuild-dcode-flow-helpers.ts
- src/lib/actions/sandbox/rebuild-local-provider-recreate.test.ts
- src/lib/actions/sandbox/rebuild-dcode-recovered-provider.test.ts
- test/helpers/rebuild-flow-generic-harness.ts
- src/lib/actions/sandbox/rebuild-hermes-post-restore.test.ts
- test/helpers/rebuild-flow-dcode-harness.ts
… profile Signed-off-by: Tinson Lai <tinsonl@nvidia.com>
cv
left a comment
There was a problem hiding this comment.
Accepted issue #8359 has no active blockers. The complete 23-file change preserves the two profile-specific mock sets, centralizes shared loading and cleanup, migrates all consumers, and has a negative net LOC delta. Security review found no weakened credential, policy, sandbox, or system boundary because only test infrastructure changes. No competing PR, unresolved review finding, production compatibility change, or documentation impact remains. The PR is current with main, open, mergeable, and has no maintainer review.
Signed-off-by: Apurv Kumaria <akumaria@nvidia.com>
Security review: passI reviewed the 24 changed files and the final two compatibility repairs. The change is confined to test suites, test-helper modules, and one repository check. It changes no shipped command, runtime path, credential flow, network request, policy schema, sandbox privilege, dependency, or package artifact.
Validation passed for CLI type-checking, repository structural checks, 62 rebuild-focused files with 706 passing tests, and both affected integration files with 9 passing tests. Changed-file hooks also passed. No security blocker remains. This review does not waive any required repository check. |
<!-- markdownlint-disable MD041 --> ## Summary This PR updates the August 10, 2026 v0.0.106 release entry with gateway readiness fixes that merged after PR #8756. PRs #8765, #8767, and #8768 remain outside this entry because they are open and do not carry the `v0.0.106` release label. ## Changes - Document acceptance of OpenShell v0.0.101 `Server:` endpoint output and target-bound process tags when trusted listener evidence matches the configured gateway. - Document preservation of selected-gateway stale state so onboarding can reconcile a registered gateway when a gateway-scoped OpenShell status check cannot connect. - Record evidence-backed exclusions for internal image, startup, qualification, proxy-environment, CI, and test-harness changes in PRs #8754, #8609, #8762, #8432, #8766, and #8581. - Exclude PRs #8765, #8767, and #8768 because their changes are absent from `main` and the PRs do not carry the `v0.0.106` release label. The release entry must be updated after any of those PRs merges for v0.0.106. ## Type of Change - [ ] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [x] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [ ] Tests added or updated for changed behavior - [x] Existing tests cover changed behavior — justification: `test/changelog-docs.test.ts` validates dated changelog SPDX placement, version headings, forbidden terms, and link form. - [ ] Tests not applicable — justification: - [x] Docs updated for user-facing behavior changes - [ ] Docs not applicable — justification: - [ ] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [ ] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Documentation Writer Review - [x] Documentation writer subagent reviewed the completed changes - Result: `docs-updated` - Evidence: `docs/changelog/2026-08-10.mdx`; an independent Codex Desktop subagent reviewed the writing rules and documentation style, terminology, structure, voice, code-sample presentation, links, source and test accuracy, release meaning, product scope, and evidence-backed exclusions at commit `190bf882c`. - Agent: Codex Desktop <!-- docs-review-head-sha: 190bf88 --> <!-- docs-review-agents-blob-sha: c4923a3 --> ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: Not applicable; `scripts/prepare-dgx-station-host.sh` is unchanged. - Station profile/scenario: Not applicable. - Result: Not applicable. - Supporting evidence: Not applicable. ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run validate:pr` passed after refreshing `origin/main` when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result or justification: `npx vitest run test/changelog-docs.test.ts` passed 6 tests. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — command/result: Not applicable to a documentation-only release-entry update. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) — result: passed with 0 errors and 2 existing warnings. - [x] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) — no page was added. --- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved gateway readiness detection for OpenShell v0.0.101 endpoint output. * Process tags are now accepted only when they match trusted listener information for the configured gateway. * Preserved stale gateway status during connection failures to support accurate onboarding reconciliation. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Co-authored-by: Carlos Villela <cvillela@nvidia.com>
Summary
This refactor replaces two drifting rebuild test harnesses with one shared core and named generic and Deep Agents Code profiles. The shared core owns module loading, source handles, session setup, temporary-state cleanup, and the mock lifecycle. The profiles keep only their distinct behavior.
The final compatibility repair also moves the remaining consumer to the generic profile and updates the Deep Agents Code test doubles for the current gateway-readiness contract. No production or user-visible behavior changes.
Related Issue
Fixes #8359
Parent epic: #8345
Changes
test/helpers/rebuild-flow-harness.tsto the common loader, module, session, and cleanup surface.rebuild-flow-generic-harness.tsandrebuild-flow-dcode-harness.ts.The current diff changes 24 files with 930 additions and 1,007 deletions, a net reduction of 77 lines.
Type of Change
Quality Gates
Documentation Writer Review
no-docs-neededdocs/requires an update.Verification
Signed-off-by:line and every commit appears asVerifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run validate:prpassed after refreshingorigin/mainwhen hooks were skipped or unavailablenpm testfor broad runtime/test-harness changes;npm run checkfor repo-wide validation/coverage changes — command/result: the local aggregate run reached test execution but produced widespread process, network, and temporary-state contention across unrelated files on the shared host. The run was stopped after the failure class was established. The changed rebuild set passes in isolation, and required CI will run the aggregate projects in their normal environment.npm run docsbuilds without warnings (doc changes only) — command/result: not applicable; no documentation changed.Signed-off-by: Tinson Lai tinsonl@nvidia.com