fix(onboard): select IPv4 sandbox fallback DNS and probe the same resolver

[jason-ma-nv]

Summary

On a systemd-resolved host whose upstream resolver list is IPv6-first, sandbox recreation could select an unreachable IPv6 fallback resolver and inject it into the recreated Docker sandbox. This change selects a usable unicast IPv4 fallback and validates that exact resolver immediately before destructive recreation, but only when the clone will actually inject the fallback. Explicit DNS and host-network sandboxes retain their existing behavior.

Related Issue

Closes #7172

Changes

• Restrict the compatibility fallback selector to usable unicast IPv4 resolvers. IPv6-only upstream lists preserve Docker defaults instead of injecting an unreachable resolver.
• Share fallback selection with the clone construction path so the preflight and resulting --dns argument cannot drift.
• Keep the global bridge-DNS preflight on Docker defaults because it does not yet have clone network context.
• Before any stop, rename, or replacement, run the exact fallback probe only when recreation will inject that resolver. Fatal results abort before mutation; inconclusive results preserve the existing warning-and-continue behavior.
• Add focused coverage for IPv4 selection, explicit DNS and host-network parity, exact fallback probing, and fatal-before-mutation behavior.
• Update docs/reference/troubleshooting.mdx with the exact fallback selection, pre-mutation probe, IPv6-only default-preservation, and explicit-DNS/host-network behavior.

Deliberate behavior change: IPv6 is no longer selected as a fallback candidate for this compatibility override. A future IPv6 path should be capability-probed separately.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Quality Gates

• Tests added or updated for changed behavior
• Existing tests cover changed behavior — justification:
• Tests not applicable — justification:
• Docs updated for user-facing behavior changes
• Docs not applicable — justification:
• Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
• Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Maintainer review required clone-aware validation so explicit DNS and host-network sandboxes are not false-blocked; the fallback value remains IP-validated before command construction.
• Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

• Documentation writer subagent reviewed the completed changes
• Result: docs-updated
• Evidence: docs/reference/troubleshooting.mdx documents IPv4 fallback DNS selection, probing the exact --dns path before recreation, preserving the original container on confirmed failure, IPv6-only behavior, and explicit-DNS/host-network exclusions. npm run docs passed with 0 errors and the 2 existing Fern warnings.
• Agent: Codex Desktop

Verification

• PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
• Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
• Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result: 44 focused tests passed across fallback selection, bridge DNS, exact clone-context probing, and recreation mutation ordering; CLI build and typecheck passed.
• Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
• Quality Gates section completed with required justifications or waivers
• No secrets, API keys, or credentials committed
• npm run docs completed successfully (doc changes only) — 0 errors; 2 existing Fern warnings
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only) — not applicable; no new pages were added

Verification detail

• Proven: IPv4 is preferred over an earlier IPv6 upstream; IPv6-only lists preserve Docker defaults; explicit DNS and host-network sandboxes skip the fallback probe; recreation probes the exact selected fallback before mutation; fatal results leave the existing sandbox untouched.
• Documentation: troubleshooting describes the exact resolver and mutation boundary operators see when this path fails.
• Remaining environment proof: a full recreate on an IPv6-first systemd-resolved host still requires that host topology.

---

Signed-off-by: Jason Ma jama@nvidia.com

Summary by CodeRabbit

• Bug Fixes
  • Improved DNS reliability during GPU sandbox recreation by adding a preflight DNS resolver probe with an optional DNS override.
  • Resolver selection is now IPv4-focused, ignoring IPv6/unreachable upstreams and handling IPv6-only upstream lists safely.
  • Preserves existing DNS behavior for containers that already specify DNS or use host networking.
  • Fatal probe errors stop recreation; inconclusive results warn and continue.
• Documentation
  • Updated troubleshooting guidance for GPU routing/compatibility failures tied to DNS recreation.
• Tests
  • Expanded DNS probe and fallback-resolver coverage across bridge and host networking scenarios.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

The change narrows sandbox DNS fallback selection to usable IPv4 resolvers, adds resolver overrides to container DNS probes, and validates the exact fallback during sandbox recreation before stopping the original container.

Changes

DNS fallback alignment

Layer / File(s)	Summary
IPv4 fallback resolver selection src/lib/onboard/docker-gpu-dns-fallback.ts, src/lib/onboard/docker-gpu-dns-fallback.test.ts	Fallback selection ignores IPv6 upstreams and returns null when no usable IPv4 resolver exists.
Clone fallback-DNS planning src/lib/onboard/docker-gpu-patch-clone.ts	Clone construction suppresses fallback DNS for host networking or explicit DNS and otherwise conditionally adds the selected resolver.
Container DNS probe override src/lib/onboard/preflight.ts, src/lib/onboard/preflight-dns-resolver-override.test.ts, src/lib/onboard/bridge-dns-preflight.ts	DNS probes validate and inject an optional resolver into Docker execution; tests cover override, default, and invalid-input behavior.
Recreation fallback probing src/lib/onboard/docker-gpu-patch-recreate.ts, src/lib/onboard/docker-gpu-patch-types.ts, src/lib/onboard/docker-gpu-patch-recreate-dns.test.ts, docs/reference/troubleshooting.mdx	Recreation probes the resolver used by clone construction and blocks fatal failures before stopping the original container.
Recreation test isolation src/lib/onboard/docker-startup-command-patch.test.ts	Startup-command tests disable fallback detection through a helper and update replacement-container timing mocks.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Recreation
  participant getDockerGpuCloneFallbackDns
  participant probeContainerDns
  participant Docker
  Recreation->>getDockerGpuCloneFallbackDns: derive fallback resolver
  getDockerGpuCloneFallbackDns-->>Recreation: IPv4 resolver or null
  Recreation->>probeContainerDns: probe exact resolver
  probeContainerDns->>Docker: docker run with optional --dns
  Docker-->>probeContainerDns: DNS probe result
  probeContainerDns-->>Recreation: success, warning, or fatal result
  Recreation->>Docker: stop original container after non-fatal probe

Loading

Suggested labels: bug-fix, area: onboarding, area: networking

Suggested reviewers: cv

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	The changes match #7172 by selecting IPv4-only fallbacks, preserving null for IPv6-only lists, probing the injected resolver, and adding parity tests.
Out of Scope Changes check	✅ Passed	The code changes stay focused on DNS fallback selection, probing, tests, and docs, with no clear unrelated additions.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: IPv4 fallback DNS selection and probing the same resolver.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/7172-dns-fallback-ipv4-preflight-parity

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-code-quality]

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 7bcbef2 in the fix/7172-dns-fallbac... branch remains at 96%, unchanged from commit e84f5ee in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 7bcbef2 in the fix/7172-dns-fallbac... branch remains at 80%, unchanged from commit e84f5ee in the main branch.

Show a code coverage summary of the most impacted files.

File	main e84f5ee	fix/7172-dns-fallbac... 7bcbef2	+/-
src/lib/onboard/remediation.ts	94%	76%	-18%
src/lib/advisor.../host/docker.ts	100%	89%	-11%
src/lib/onboard/preflight.ts	81%	77%	-4%
src/lib/onboard/docker-cdi.ts	83%	80%	-3%
src/lib/state/m...-acquisition.ts	87%	84%	-3%
src/lib/onboard...-desktop-gpu.ts	89%	86%	-3%
src/lib/onboard...-patch-clone.ts	94%	93%	-1%
src/lib/state/g...way-registry.ts	95%	94%	-1%
src/lib/onboard...tch-recreate.ts	95%	97%	+2%
src/lib/advisories/runner.ts	97%	100%	+3%

---

_{Updated July 23, 2026 23:54 UTC}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard/bridge-dns-preflight.ts`:
- Around line 200-203: Update the fatal DNS failure output in the bridge DNS
preflight flow to append the existing testedResolverSuffix to its headline.
Reuse the suffix created alongside dnsIsFatal, while preserving the current
success and inconclusive message behavior.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 74c6c8e7-7314-47b2-802d-983a56bef3b8

📥 Commits

Reviewing files that changed from the base of the PR and between 4deebcf and 2c5b6a2.

📒 Files selected for processing (5)

• src/lib/onboard/bridge-dns-preflight.ts
• src/lib/onboard/docker-gpu-dns-fallback.test.ts
• src/lib/onboard/docker-gpu-dns-fallback.ts
• src/lib/onboard/preflight.test.ts
• src/lib/onboard/preflight.ts

[github-actions]

PR Review Advisor — Informational

Advisor assessment: Informational / medium confidence
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions
Status: No actionable findings remain in the canonical review ledger.

Model lanes

• GPT-5.6 Terra (primary): Completed · medium confidence · 0 blockers · 0 warnings · 0 suggestions
• Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 6 suggestions
• Model comparison: normalized findings differ; normalized E2E selections differ; Nemotron reported the same number of blockers, the same number of warnings, 6 more suggestions.

Nemotron output stays in workflow artifacts and does not change the assessment above.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: onboard-repair, onboard-resume, cloud-onboard

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard/preflight-dns-resolver-override.test.ts`:
- Around line 37-40: Strengthen the test for probeContainerDns by making
runCaptureImpl record whether it was invoked, then assert it was not called
after the invalid dnsServer input throws. Keep the existing IP-validation error
assertion and ensure the malicious resolver is rejected before reaching the
execution seam.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 1814ccb8-98e4-4495-b196-3fe452b78366

📥 Commits

Reviewing files that changed from the base of the PR and between 2c5b6a2 and ef38a05.

📒 Files selected for processing (6)

• src/lib/onboard/bridge-dns-preflight.ts
• src/lib/onboard/docker-gpu-patch-clone.ts
• src/lib/onboard/docker-gpu-patch-recreate-dns.test.ts
• src/lib/onboard/docker-gpu-patch-recreate.ts
• src/lib/onboard/docker-gpu-patch-types.ts
• src/lib/onboard/preflight-dns-resolver-override.test.ts

[github-actions]

🌿 Preview your docs: https://nvidia-preview-pr-7185.docs.buildwithfern.com/nemoclaw

[apurvvkumaria]

CI diagnosis for exact head 209331aec61655d8f666a71dce96a115c8d93797: CLI shard 1 completed and reported all 198 expected files as passed, then one Vitest/Node process exhausted its ~4 GiB V8 heap while finalizing/collecting results. The blob contains no failed test—only one unhandled worker error. This reproduced on the prior exact code head as well, so I am not masking it with repeated retries. The infrastructure family is tracked by #6237 (coverage-shard source-loader contention) and #7146 (resource attribution/retry policy). Product-focused tests, build, typecheck, docs, and hooks remain green.

[laitingsheng]

Review summary

Verified the full change set at head 6f4b48c in a detached worktree. The fix narrows the compatibility DNS override to usable unicast IPv4, shares one selection function between preflight and clone construction, and probes the exact resolver before any destructive recreation.

Verified

• Solves #7172 — isUsableUpstreamResolver rejects non-IPv4; IPv6-only upstream lists return null and preserve Docker defaults; link-local 169.254.169.253 retention intact. Covers the stated acceptance criteria.
• Selection parity — getDockerGpuCloneFallbackDns() is the single source of truth; clone construction and the recreation probe call it with the same options, so the probed resolver equals the injected --dns. Explicit HostConfig.Dns and host networking return null in both paths, with no duplicate or conflicting --dns.
• Fatal-before-mutation — the recreation probe throws before dockerStop; regression coverage asserts the original container is never stopped on a fatal result.
• Injection guard — dnsServer is validated with net.isIP before shell-command construction and before the execution seam; the hardened test proves a malicious value never reaches runCapture.
• Behavioural tests — selection, parity, exact-command, non-IP rejection, and fatal ordering all assert on outputs, args, throws, and call state. The startup-command suite is isolated via a fallback-disabling helper and a deterministic clock.
• Docs — troubleshooting.mdx documents IPv4 selection, the pre-mutation probe, IPv6-only default preservation, and explicit-DNS/host-network behaviour.
• Discussion — both CodeRabbit inline findings addressed (fatal-suffix reworked to the clone-aware recreation point; resolver-rejection test hardened). No still-valid bot findings outstanding.
• CI — no red required checks; CLI shards, CodeQL, ShellCheck, sandbox image builds, commit-lint, and DCO all pass.

Design note: the global bridge preflight intentionally keeps probing Docker defaults, while the fallback-specific probe lives at the clone-aware recreation point where explicit-DNS and host-network suppression apply — documented in-line and consistent with the direction on the thread.

Approving.

[apurvvkumaria]

Refreshed this PR append-only onto current main in signed, GitHub-Verified merge commit 9f8b44cf5.

• Exact parents: prior PR head 6f4b48c81 and current main 3c4ea5a1e.
• Merge completed without conflicts or manual resolution; the overlapping preflight file preserves both current-main advisory planning and this PR's validated DNS override path.
• Focused DNS/preflight validation: 217/217 passed.
• CLI type-check, source-shape/title/size gates, exact-diff checks, commit/pre-push hooks, and docs build passed.
• Jason Ma's original authorship is preserved in the merge commit.

Fresh exact-head/current-base CI, advisors, and E2E are running.

[apurvvkumaria]

Addressed the exact-head advisor coverage gap in Verified commit 387602738ddd8d2b42303a15a43f2d8cf8c99796.

• Added recreate-level coverage for a nonfatal image_pull_failed DNS probe.
• The test proves the inconclusive warning is emitted, recreation continues, and the selected --dns 9.9.9.9 is retained.
• No production behavior or release label changed.
• Validation passed: focused file 11/11, Docker GPU patch suite 100/100, CLI typecheck, scoped hooks, and independent audit.
• Documentation audit found no user-facing update needed.

Jason Ma remains credited as co-author. Fresh exact-head CI and automated review are running.