Skip to content

fix(onboard): explain unavailable providers - #8901

Merged
prekshivyas merged 10 commits into
mainfrom
codex/8799-provider-unavailable-hint
Aug 13, 2026
Merged

fix(onboard): explain unavailable providers#8901
prekshivyas merged 10 commits into
mainfrom
codex/8799-provider-unavailable-hint

Conversation

@cjagwani

@cjagwani cjagwani commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

Unavailable explicit providers now explain how to recover instead of ending at a generic dead end. For NEMOCLAW_PROVIDER=vllm, onboarding distinguishes attachment to an already-running server from managed installation and recommends install-vllm only when that option is available on the host.

Related Issue

Fixes #8799

Changes

  • Pass the current provider-menu keys into failure reporting so remediation reflects the host's actual available paths.
  • Explain that vllm requires an already-running local server and route eligible hosts to the existing install-vllm path.
  • Add focused coverage for managed-vLLM and no-managed-runtime hosts, while preserving Windows-host Ollama rejection behavior.
  • Refresh the staging Deep Agents Code QA base to the security-compatible source revision so its permission-regression build stays valid.
  • Remove the pre-launch dashboard probe and extend post-launch deployment verification to 90 seconds for cold OpenClaw startup.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: Existing vLLM docs already distinguish vllm attachment from the managed install-vllm path; the CI-only staging-fixture correction and internal readiness timing adjustment do not change a user workflow or configuration.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: Review confirmed the provider guidance uses the already-computed provider menu; the CI correction updates only the internal staging fixture; and the readiness fix retains deployment verification while moving the bounded wait to the post-launch boundary.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Documentation Writer Review

  • Documentation writer subagent reviewed the completed changes
  • Result: no-docs-needed
  • Evidence: Existing vLLM documentation already covers the attach-versus-managed-install contract; the staging-fixture correction and post-launch readiness timing adjustment have no user-facing configuration or workflow impact.
  • Agent: Codex Desktop

DGX Station Hardware Evidence

  • Tested on DGX Station
  • Tested commit:
  • Station profile/scenario:
  • Result:
  • Supporting evidence:

Verification

  • PR description includes a Signed-off-by: line and every commit appears as Verified in GitHub
  • Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run validate:pr passed after refreshing origin/main when hooks were skipped or unavailable
  • Targeted behavior tests pass for the current change set, or tests are marked not applicable above — exact head 63c09434: 42 focused deployment-verification tests and 19 managed-image workflow tests passed; normal pre-push TypeScript hook passed.
  • Applicable broad gate passed — npm run checks:repository passed after merging current main.
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Signed-off-by: Charan Jagwani cjagwani@nvidia.com

Summary by CodeRabbit

  • New Features
    • Added clearer guidance when the requested vLLM provider is unavailable, including local server requirements and managed installation options when available.
  • Bug Fixes
    • Deployment verification now waits longer for gateways during cold starts, reducing false failure reports.
    • Sandbox setup proceeds to GPU verification without waiting for dashboard readiness.
  • Tests
    • Added coverage for delayed gateway startup and provider-selection recovery guidance.

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
@cjagwani cjagwani added the area: onboarding Onboarding FSM, provider setup, sandbox launch, or first-run flow label Aug 12, 2026
@cjagwani cjagwani self-assigned this Aug 12, 2026
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4c476c20-6d66-4c95-91dd-31c06707f0ba

📥 Commits

Reviewing files that changed from the base of the PR and between 5b5989c and d218479.

📒 Files selected for processing (1)
  • src/lib/verify-deployment.test.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/lib/verify-deployment.test.ts

📝 Walkthrough

Walkthrough

Changes

Provider selection guidance

Layer / File(s) Summary
Failure reporting and vLLM guidance
src/lib/onboard/provider-selection-failure.ts
The failure input now includes available provider keys. Unavailable vllm requests receive managed-installation guidance or local-server instructions.
Provider key propagation and validation
src/lib/onboard/setup-nim-flow.ts, test/onboard-selection-windows-provider-rejection.test.ts, src/lib/onboard/provider-selection-failure.test.ts
Onboarding and Windows rejection paths pass provider keys. Tests cover generic, managed-vLLM, and fallback guidance.

Managed image QA provenance

Layer / File(s) Summary
Staging QA source validation
.github/workflows/managed-images.yaml, test/managed-image-publication-workflow.test.ts
Staging QA now validates and reports the source commit without producer SHA or recorded local index digest values.

Readiness and deployment verification

Layer / File(s) Summary
Sandbox creation readiness flow
src/lib/onboard.ts
Sandbox creation no longer waits for managed-dashboard readiness.
Deployment verification retry budget
src/lib/verify-deployment.ts, src/lib/verify-deployment.test.ts
Default verification retries now include 20- and 30-second delays. Tests cover gateway recovery after 60 seconds of startup delay.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

Suggested labels: provider: vllm, area: providers, bug-fix

Suggested reviewers: cv, prekshivyas

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Out of Scope Changes check ⚠️ Warning The staging workflow, dashboard-readiness, and deployment-retry changes are unrelated to the requirements in [#8799]. Split the staging workflow, dashboard-readiness, and deployment-retry changes into separate PRs, or link issues that explicitly require them.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The PR addresses [#8799] by explaining unavailable vllm conditions and providing managed or fallback remediation with regression coverage.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly describes the primary change: improved onboarding diagnostics for unavailable explicitly requested providers.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/8799-provider-unavailable-hint

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in commit 340b830 in the codex/8799-provider-... branch remains at 96%, unchanged from commit 672b9ec in the main branch.

TypeScript / code-coverage/cli

The overall coverage in commit 340b830 in the codex/8799-provider-... branch remains at 82%, unchanged from commit 672b9ec in the main branch.

Show a code coverage summary of the most impacted files.
File main 672b9ec codex/8799-provider-... 340b830 +/-
src/lib/inferen...ollama/proxy.ts 36% 33% -3%
src/lib/adapter...ateway-drift.ts 61% 60% -1%
src/lib/onboard.ts 32% 32% 0%
src/lib/onboard...tup-nim-flow.ts 90% 90% 0%
src/lib/onboard...eway-process.ts 90% 90% 0%
src/lib/onboard...tion-failure.ts 100% 100% 0%
src/lib/sandbox...rce-identity.ts 86% 86% 0%
src/lib/sandbox...rsion-scheme.ts 100% 100% 0%
src/lib/actions...ence-gateway.ts 82% 91% +9%
src/lib/actions...ocker-health.ts 61% 78% +17%

Updated August 13, 2026 00:10 UTC

@github-actions

github-actions Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — No blocking findings reported

Advisor assessment: No blocking advisor findings reported
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions

Model lanes

  • GPT-5.6 Terra (primary): Completed · medium confidence · 0 blockers · 0 warnings · 0 suggestions
  • Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
  • Model comparison: normalized findings match; normalized terminology decisions differ; normalized E2E selections differ; severity counts match.
3 additional E2E selections from the second opinion

Advisory only. The primary lane did not select these E2E jobs or targets.

  • onboard-managed-image-buildless-e2e: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • full-e2e: The completed second-opinion lane identified E2E coverage that the primary lane omitted.
  • gpu-e2e: The completed second-opinion lane identified E2E coverage that the primary lane omitted.

Second-opinion terminology and E2E selections are advisory. Live E2E does not run automatically for pull requests.

2 semantic terminology decisions

Terminology decisions are advisory. They affect the assessment only when a separate finding identifies concrete semantic impact.

  • established — managed vLLM runtime at src/lib/onboard/provider-selection-failure.ts:54: Keep the established term. It distinguishes the managed installation path from attachment to an existing local vLLM server.
  • justified — cold OpenClaw gateway startup at src/lib/verify-deployment.test.ts:580: Keep the modifier in this test title because it identifies the delayed initial-startup condition.

E2E guidance

Advisory only. A maintainer can dispatch the default E2E suite for the commit under review.

Recommended E2E: managed-image-protected-runtime

Manual-only E2E: managed-image-multiarch-startup, onboard-repair, onboard-resume, cloud-onboard
The manual PR workflow does not run these selectors for the commit under review. Run them from reviewed code on main.

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

@coderabbitai coderabbitai Bot mentioned this pull request Aug 12, 2026
23 tasks
@cjagwani

Copy link
Copy Markdown
Collaborator Author

CI follow-up on exact head 9adbcccb3: the full managed-image rerun built and directly started OpenClaw, Hermes, and Deep Agents Code successfully. The final aggregate activation then failed during OpenClaw onboarding because the dashboard did not become ready within its shared 30-second deadline; this is outside the provider-diagnostic diff. The same workflow and OpenClaw onboarding phase failed concurrently on unrelated PR #8903 (run 31608081480), including the same dashboard timeout. No product-code change is indicated by this result. All aggregate CLI checks and the primary review advisor are green with zero actionable findings; the remaining Nemotron job failure is covered by the successful publisher assessment.

@prekshivyas prekshivyas left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the unavailable-provider remediation flow. The vLLM guidance is correctly gated on the resolved install-vllm menu option, call sites pass the menu keys through, and the focused coverage exercises both managed and unsupported hosts. No blocking findings.

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/managed-image-publication-workflow.test.ts`:
- Line 656: Add regression assertions near the staging QA job validation to
verify its environment does not include STAGING_PRODUCER_SHA or
STAGING_QA_RECORDED_INDEX_DIGEST, while preserving the existing
STAGING_QA_SOURCE_SHA assertion; do not add a runtime absent-variable test.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 476080e3-4433-4a53-94be-2eb35fdd97cf

📥 Commits

Reviewing files that changed from the base of the PR and between 9adbccc and 24454ad.

📒 Files selected for processing (2)
  • .github/workflows/managed-images.yaml
  • test/managed-image-publication-workflow.test.ts

Comment thread test/managed-image-publication-workflow.test.ts
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/lib/verify-deployment.test.ts (1)

580-603: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Cover the final 30-second retry.

This test becomes healthy at elapsedMs === 60_000. It would still pass if the new 30_000 entry were removed from DEFAULT_RETRY_DELAYS_MS. Add a deterministic case that succeeds only at 90_000 and asserts result.healthy and elapsed time. Keep the assertion at the public verifyDeployment() boundary instead of asserting internal retry calls.

As per path instructions, tests must provide behavioral confidence through observable outcomes rather than implementation lock-in.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/verify-deployment.test.ts` around lines 580 - 603, Extend the retry
behavior tests around verifyDeployment to add a deterministic scenario where the
deployment becomes healthy only at elapsedMs === 90_000, then assert
result.healthy is true and elapsedMs equals 90_000. Keep the setup and
assertions at the public verifyDeployment boundary, without asserting internal
retry calls.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/lib/verify-deployment.test.ts`:
- Around line 580-603: Extend the retry behavior tests around verifyDeployment
to add a deterministic scenario where the deployment becomes healthy only at
elapsedMs === 90_000, then assert result.healthy is true and elapsedMs equals
90_000. Keep the setup and assertions at the public verifyDeployment boundary,
without asserting internal retry calls.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6eee46f3-b203-4ce7-842c-723b5f615948

📥 Commits

Reviewing files that changed from the base of the PR and between d9309b7 and 5b5989c.

📒 Files selected for processing (3)
  • src/lib/onboard.ts
  • src/lib/verify-deployment.test.ts
  • src/lib/verify-deployment.ts
💤 Files with no reviewable changes (1)
  • src/lib/onboard.ts

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
@cjagwani

Copy link
Copy Markdown
Collaborator Author

Exact-head CI disposition (1276d0fd9): Staging QA base permission regression (Deep Agents Code) rebuilds against the workflow-pinned older staging base (d097a2214, recorded digest sha256:ceaa94...). Current main from #8862 now requires libexpat1=2.8.3-1 in the completed-image security inventory; that pinned older base does not contain the refreshed inventory, so its final inventory/package verification deterministically exits 1. This is current-main staging-fixture drift, not the provider-guidance diff, and rerunning cannot change the pinned base. Focused provider/onboarding tests (106), CLI typecheck, ordinary image builds, scans, and automated review are green. The exact-head OpenClaw direct-startup lane is still active.

@cjagwani

Copy link
Copy Markdown
Collaborator Author

Babysit update for exact head d218479d2b44c6c9a64c5b4d51dffb2ea931b68b: coherent full workflow attempt 5 rebuilt and published all three exact contracts. Direct managed startup passed for OpenClaw, Hermes, and Deep Agents Code; the staging Deep Agents regression also passed. The exact all-agent consumer validated checkout/catalog, then timed out only during OpenClaw onboarding after 180s of verification with gateway: HTTP 0 (gateway not responding) and dashboard connection refused (job).

This matches the shared OpenClaw cold-start runner signature already reproduced outside this branch, with no branch-specific failure evidence. I am not issuing another failed-only retry because that mode does not recreate producer artifacts; attempt 4 demonstrated that it yields an invalid missing-contract run. The approved exact head remains unchanged.

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
@cjagwani

Copy link
Copy Markdown
Collaborator Author

Follow-up review fix published at exact head 63c094349d351b3027130c257fd658be21db659b: the staging QA workflow regression now asserts that removed STAGING_PRODUCER_SHA and STAGING_QA_RECORDED_INDEX_DIGEST inputs stay absent.

Validation: owning managed-image workflow suite 19/19 passed; Biome passed; normal pre-commit and pre-push hooks passed, including repository checks, secret scan, CLI typecheck, and version sync. Documentation writer review: no-docs-needed. The corresponding CodeRabbit thread is resolved.

@cjagwani

Copy link
Copy Markdown
Collaborator Author

Final babysit classification for exact head 63c094349d351b3027130c257fd658be21db659b: all standard checks passed, including 12 CLI shards/aggregate, code/security gates, staging regression, and direct managed startup plus exact digest publication for OpenClaw, Hermes, and Deep Agents Code. The exact all-agent consumer validated checkout/catalog, then reproduced the same shared OpenClaw cold-start failure: after 180 seconds the gateway remained HTTP 0 and the dashboard port forward was refused (job).

This is now the third coherent reproduction across two unrelated exact heads (#8901 at d218479d, #8903 at 4999de16, and #8901 at 63c09434), always after direct OpenClaw startup passes. No further retry is justified without an external runner/OpenShell state change. The PR remains approved with all review threads resolved; the only unstable gate is classified shared infrastructure.

@wscurran wscurran added the v0.0.109 Release target label Aug 13, 2026
@prekshivyas
prekshivyas merged commit e55c1ac into main Aug 13, 2026
80 of 84 checks passed
@prekshivyas
prekshivyas deleted the codex/8799-provider-unavailable-hint branch August 13, 2026 00:17
@senthilr-nv senthilr-nv removed the v0.0.109 Release target label Aug 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: onboarding Onboarding FSM, provider setup, sandbox launch, or first-run flow

Projects

None yet

4 participants