fix(images): normalize managed runtime bundle metadata - #8702
Conversation
Signed-off-by: San Dang <sdang@nvidia.com>
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe Dockerfiles now copy managed runtime artifacts as root-owned, read-only files. Final-image checks report individual failures for discovery, artifact metadata, permissions, and runtime-directory creation. Workflow and contract tests validate build preparation and Dockerfile instructions. ChangesManaged runtime image contract
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant Workflow
participant DockerBuild
participant DiscoveryRuntime
participant RuntimeContract
participant AssertionReporter
Workflow->>Workflow: Validate and prepare reviewed files
Workflow->>DockerBuild: Start managed-image build
DockerBuild->>DiscoveryRuntime: Execute discovery contract
DiscoveryRuntime->>RuntimeContract: Return contract output
DockerBuild->>RuntimeContract: Validate artifact and directory metadata
RuntimeContract->>AssertionReporter: Report failed sub-check
AssertionReporter-->>DockerBuild: Fail image build
Possibly related PRs
Suggested labels: Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@Dockerfile`:
- Around line 1535-1536: Reject symlinked /run/nemoclaw paths by updating the
runtime-directory setup in Dockerfile, agents/hermes/Dockerfile, and
agents/langchain-deepagents-code/Dockerfile: after install -d, require it to be
a directory, not a symlink, and have exact 0:0:755 ownership and permissions,
invoking managed_runtime_assertion_failed on any mismatch. Update the
corresponding contract/layout tests in
test/support/managed-bootstrap-image-contract.ts,
test/langchain-deepagents-code-image.test.ts,
test/openclaw-final-image-layout.test.ts, and
test/hermes-final-image-layout.test.ts to assert these checks and add a symlink
negative-path case.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: d14b2d4c-a378-43ef-93b8-b3c58ead57bc
📒 Files selected for processing (8)
Dockerfileagents/hermes/Dockerfileagents/langchain-deepagents-code/Dockerfiletest/hermes-final-image-layout.test.tstest/langchain-deepagents-code-image.test.tstest/mcp-tool-discovery-image-contract.test.tstest/openclaw-final-image-layout.test.tstest/support/managed-bootstrap-image-contract.ts
PR Review Advisor — No blocking findings reportedAdvisor assessment: No blocking advisor findings reported Model lanes
6 terminology differences from the second opinionAdvisory only. These are normalized differences from the primary terminology receipt.
1 additional E2E selection from the second opinionAdvisory only. The primary lane did not select these E2E jobs or targets.
Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests. 2 semantic terminology decisionsTerminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.
E2E guidanceAdvisory only. A maintainer can dispatch the default E2E suite against this exact revision. Recommended E2E: This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge. |
Signed-off-by: San Dang <sdang@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/managed-images.yaml:
- Around line 97-105: Update the “Make reviewed runtime bundle files
group-writable” step to validate every listed artifact path with `[[ -f "$path"
&& ! -L "$path" ]]` before invoking `chmod 0664`. Reject the workflow when any
path is missing, non-regular, or symlinked, while preserving the existing mode
change for valid runtime bundle files.
In `@test/mcp-tool-discovery-image-contract.test.ts`:
- Around line 123-126: Strengthen the assertions in the test around the
workflow’s intended job and run block so they inspect the complete chmod command
rather than independent text occurrences. Bind each reviewedRuntimeBundleFiles
path to the chmod 0664 invocation, and assert the exact command operands so
comments, echo statements, or unrelated commands cannot satisfy the test.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: cdc05730-cc01-485a-8eea-93ad94789ddb
📒 Files selected for processing (6)
.github/workflows/managed-images.yamlDockerfileagents/hermes/Dockerfileagents/langchain-deepagents-code/Dockerfileci/source-shape-test-budget.jsontest/mcp-tool-discovery-image-contract.test.ts
🚧 Files skipped from review as they are similar to previous changes (3)
- agents/hermes/Dockerfile
- Dockerfile
- agents/langchain-deepagents-code/Dockerfile
Signed-off-by: San Dang <sdang@nvidia.com>
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/support/managed-bootstrap-image-contract.ts`:
- Around line 70-73: Add an image-contract test alongside the existing
runtime-directory checks that builds the image with invalid /run/nemoclaw state,
such as a symlink or incorrect ownership/mode, and asserts the build fails with
the corresponding managed_runtime_assertion_failed diagnostic. Keep source-text
checks only as supplemental coverage, and do not rely solely on the direct
runDiagnostic() formatter test.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 032db256-89f4-4003-82e0-bd64ce0cf45c
📒 Files selected for processing (9)
.github/workflows/managed-images.yamlDockerfileagents/hermes/Dockerfileagents/langchain-deepagents-code/Dockerfiletest/hermes-final-image-layout.test.tstest/langchain-deepagents-code-image.test.tstest/mcp-tool-discovery-image-contract.test.tstest/openclaw-final-image-layout.test.tstest/support/managed-bootstrap-image-contract.ts
🚧 Files skipped from review as they are similar to previous changes (7)
- .github/workflows/managed-images.yaml
- test/hermes-final-image-layout.test.ts
- test/openclaw-final-image-layout.test.ts
- Dockerfile
- agents/hermes/Dockerfile
- agents/langchain-deepagents-code/Dockerfile
- test/langchain-deepagents-code-image.test.ts
Signed-off-by: San Dang <sdang@nvidia.com>
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
test/mcp-tool-discovery-image-contract.test.ts (1)
183-207: 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick winExercise each reviewed bundle file through the workflow.
Line 184 locks the full shell text to
expectedRun. Lines 204-207 only replacereviewedRuntimeBundleFiles[3].Iterate every
reviewedRuntimeBundleFilesentry for each invalid preparer. Assert that the executed workflow step fails and reports the affected path. Keep the valid execution assertion for all files. This removes the source-text dependency and detects a regression that skips validation for any earlier bundle file.As per path instructions, “Prefer observable outcomes through the public boundary over source-text ... assertions.”
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@test/mcp-tool-discovery-image-contract.test.ts` around lines 183 - 207, Update the test around the metadataStep workflow to stop depending on the exact shell source in expectedRun and instead validate observable execution outcomes. For each invalidReviewedBundlePreparers entry, iterate every reviewedRuntimeBundleFiles path, remove and corrupt that file as needed, execute metadataStepRun, and assert failure with stderr identifying the affected path; retain successful execution and permission assertions for all files in the valid fixture.Source: Path instructions
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@test/mcp-tool-discovery-image-contract.test.ts`:
- Around line 183-207: Update the test around the metadataStep workflow to stop
depending on the exact shell source in expectedRun and instead validate
observable execution outcomes. For each invalidReviewedBundlePreparers entry,
iterate every reviewedRuntimeBundleFiles path, remove and corrupt that file as
needed, execute metadataStepRun, and assert failure with stderr identifying the
affected path; retain successful execution and permission assertions for all
files in the valid fixture.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 12cc012f-24ac-4566-8f5a-93f1206dc21e
📒 Files selected for processing (1)
test/mcp-tool-discovery-image-contract.test.ts
Signed-off-by: San Dang <sdang@nvidia.com>
## Summary [Brev image run 31352614414](https://github.com/brevdev/nemoclaw-image/actions/runs/31352614414) failed the shared managed-image `RUN` without emitting a concrete assertion name. This change adds named, fail-closed diagnostics for MCP discovery bundle execution, JSON-contract validation, discovery-tree traversal, and the first unsafe path while preserving every existing predicate, assertion, and successful behavior. Contract mismatches report bounded, credential-redacted actual fields beside the expected contract. ## Related Issue Related to #8665. PR #8702 addresses the image-metadata root cause and overlaps the same shared block; this PR does not close the issue. ## Changes - Report the bundle or `find` exit status before another command can overwrite it. - Keep the existing three-field discovery contract and acceptance of extra fields, but report its sanitized actual response and exact expected response on failure. - Mirror NemoClaw's current credential-pattern families in the bounded response diagnostic, including malformed JSON and private-key material. - Report the first unsafe path with dereferenced UID, GID, file type, and mode plus the path's symlink status. Dangling links retain link metadata. - Apply the same block to the OpenClaw, Hermes, and Deep Code Dockerfiles without changing the five managed-runtime assertions added by #8673. - Execute every new failure message and the unchanged success path through the shared contract helper for all three Dockerfiles. - Preserve the evidence boundary: the run log proves that no guarded assertion failed, while the previously silent discovery prefix did not identify which check stopped the build. - Do not dispatch another Brev image build; downstream image validation remains after merge. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: Internal image-build failure diagnostics and contract coverage only; no public command, configuration, default, successful runtime behavior, or documented workflow changes. - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Codex Desktop independently reviewed head `f67df5af4` against the security rubric. The review found and verified fixes for canonical credential redaction and `find -L` referent metadata, then reported PASS with no remaining publication blocker. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Documentation Writer Review - [x] Documentation writer subagent reviewed the completed changes - Result: `no-docs-needed` - Evidence: No documentation paths changed. The final diff contains internal managed-image build diagnostics and their contract tests only. - Agent: Codex Desktop <!-- docs-review-head-sha: f67df5a --> <!-- docs-review-agents-blob-sha: c4923a3 --> ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: - Station profile/scenario: - Result: - Supporting evidence: ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run validate:pr` passed after refreshing `origin/main` when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — `npm exec -- vitest run --project integration test/mcp-tool-discovery-image-contract.test.ts test/openclaw-final-image-layout.test.ts test/hermes-final-image-layout.test.ts test/langchain-deepagents-code-image.test.ts` — 4 files and 49 tests passed. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — not applicable because the change is limited to one managed-image contract helper and its three focused consumers. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved managed image validation for symlink targets and filesystem safety. * MCP tool discovery now distinguishes execution and scanning failures and reports explicit command statuses. * Invalid or unexpected discovery data is handled safely with clearer contract validation. * Diagnostic output now redacts credential- and private-key-like values. * **Tests** * Expanded coverage for discovery failures, malformed data, unsafe permissions, symlinks, and successful validation. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- markdownlint-disable MD041 --> ## Summary [Exact staging run 31396519688](https://github.com/brevdev/nemoclaw-image/actions/runs/31396519688) showed that Docker preserved mode `0664` from the build context when it copied the reviewed MCP discovery runtime. Replay root ownership and read-only modes in each final image before bundle execution, while keeping the existing fail-closed tree assertion unchanged. Add a PR-only Deep Agents Code regression that rebuilds the exact staging source and effective Docker configuration without claiming that the unavailable historical image was reproduced byte-for-byte. ## Related Issue Fixes #8665. Supersedes #8745. GitHub accepted that PR's final fast-forward branch update but did not synchronize its pull ref, commit list, or PR checks. This replacement carries that GitHub-Verified history plus the review repairs at commit `cef961c6ba411682a60819b02a9096b161374e8d` on a fresh branch. Independently supersedes the permission-fix scope of #8702. Its latest PR commit lost the permission-normalizing `COPY` options during conflict resolution and all three managed-image builds reproduce the `0664` failure. This PR uses a distinct legacy-builder-compatible final-image replay and does not transfer that PR's code, tests, managed-startup copy changes, or runtime-directory diagnostics. ## Changes - Replay `root:root` ownership, directory mode `0555`, and regular-file mode `0444` under `/usr/local/lib/nemoclaw/mcp-tool-discovery-runtime` in the OpenClaw, Hermes, and Deep Agents Code final images. - Use non-dereferencing traversal so the replay cannot change an out-of-tree link target, then run the unchanged `find -L` tree assertion after bundle validation. - Report a named permission-replay failure without changing the existing bundle, JSON, tree, or managed-startup diagnostics. - Execute the shared Dockerfile replay against the real reviewed bundle at mode `0664`, run the bundle afterward, and verify ownership, modes, bytes, failure behavior, and link scope for all three Dockerfiles. - Make the PR image workflow reproduce mode `0664` before each real managed-image build, then inspect the immutable final image for root ownership, `0555`/`0444` modes, and the exact bundle contract. - Rebuild the staging base from exact NemoClaw source `d097a22145859102c0495b0310de264b7a27624f`, apply the staging run's effective final-image configuration through the declared build arguments, prove layer ancestry, and run the final-image contract. The recorded `ceaa94...` index was local-only and unavailable, so the job treats it as provenance rather than an executable image identity. - Keep classic Docker builder compatibility by applying permissions in the final `RUN` layer instead of using `COPY --chmod` or `COPY --chown`. - Do not dispatch a Brev image build; exact staging validation remains a post-merge step. ## Type of Change - [x] Code change (feature, bug fix, or refactor) - [ ] Code change with doc updates - [ ] Doc only (prose changes, no code sample modifications) - [ ] Doc only (includes code sample changes) ## Quality Gates - [x] Tests added or updated for changed behavior - [ ] Existing tests cover changed behavior — justification: - [ ] Tests not applicable — justification: - [ ] Docs updated for user-facing behavior changes - [x] Docs not applicable — justification: This restores an internal managed-image file-permission invariant. It changes no public command, configuration, default, API, operator workflow, or successful user behavior. - [x] Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging) - [x] Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Codex Desktop independently reviewed commit `cef961c6b` across all nine security categories, including shell status handling, symlink scope, ownership and mode enforcement, output privacy, workflow authorization, and the historical-image limitation. It found no actionable security issue. - [ ] Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue: ## Documentation Writer Review - [x] Documentation writer subagent reviewed the completed changes - Result: `no-docs-needed` - Evidence: The change repairs internal managed-image MCP tool-discovery filesystem metadata and adds CI-only regression coverage. It does not change public commands, configuration, defaults, APIs, or operator workflows; existing MCP tool-discovery documentation remains accurate. - Agent: Codex Desktop <!-- docs-review-head-sha: cef961c --> <!-- docs-review-agents-blob-sha: c4923a3 --> ## DGX Station Hardware Evidence - [ ] Tested on DGX Station - Tested commit: - Station profile/scenario: - Result: - Supporting evidence: ## Verification - [x] PR description includes a `Signed-off-by:` line and every commit appears as `Verified` in GitHub - [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or `npm run validate:pr` passed after refreshing `origin/main` when hooks were skipped or unavailable - [x] Targeted behavior tests pass for the current change set, or tests are marked not applicable above — 68 tests passed across the managed-image workflow, MCP discovery image contract, and all three final-image layout contracts; the 19-test workflow contract was rerun after the final ordering assertion. - [ ] Applicable broad gate passed — `npm test` for broad runtime/test-harness changes; `npm run check` for repo-wide validation/coverage changes — not applicable to this scoped image-permission repair. `npm run validate:pr` passed for the review repair, including repository checks, source-shape budgets, Biome, hadolint, secret scanning, and CLI typecheck; the final assertion also passed normal commit and push hooks. - [x] Quality Gates section completed with required justifications or waivers - [x] No secrets, API keys, or credentials committed - [ ] `npm run docs` builds without warnings (doc changes only) - [ ] Doc pages follow the [style guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md) (doc changes only) - [ ] New doc pages include SPDX header and frontmatter (new pages only) --- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Security** - Managed images now enforce secure ownership and read-only permissions for the tool-discovery runtime. - Directories remain executable while files remain readable without being writable, and symlinks are handled safely. - **Bug Fixes** - Improved consistency and reliability when building managed images, including protection against permission drift. - **Quality** - Added automated validation for image security properties, runtime execution, and discovery response format. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Signed-off-by: Julie Yaunches <jyaunches@nvidia.com> Co-authored-by: Carlos Villela <cvillela@nvidia.com>
Summary
Normalize the owner and mode of reviewed runtime bundles while Docker copies them into scratch stages, so source file modes cannot make final-image runtime files writable. Add bounded assertion diagnostics that identify the failed contract and path when image assembly rejects runtime metadata.
Related Issue
Fixes #8665
Changes
root:rootwith mode0444in the OpenClaw, Hermes, and Deep Agents Code image stages.0664reviewed bundle files before final contract and managed-startup validation.Type of Change
Quality Gates
Documentation Writer Review
no-docs-neededDGX Station Hardware Evidence
Verification
Signed-off-by:line and every commit appears asVerifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run validate:prpassed after refreshingorigin/mainwhen hooks were skipped or unavailablenpm testfor broad runtime/test-harness changes;npm run checkfor repo-wide validation/coverage changes — command/result:npm run docsbuilds without warnings (doc changes only)Additional validation:
npm run source-shape:checknpm run checks:repositoryhadolint Dockerfile agents/hermes/Dockerfile agents/langchain-deepagents-code/Dockerfile0664input failed the existing discovery-tree contract before this change and passed with output mode0444after it.Signed-off-by: San Dang sdang@nvidia.com
Summary by CodeRabbit
Security & Reliability
Tests