nano v3 disable piecewise

[tcherckez-nvidia]

…on Nano V3

Under enable_chunked_prefill=true, Nemotron Nano V3 (hybrid Mamba2+MoE) produces device-side NaNs during piecewise CUDA-graph replay for small, heavily-padded token buckets, tripping the FlashInfer sampler's NaN assert on the first generation step. Disable piecewise CUDA-graph capture to avoid the faulty replay path while the underlying issue is investigated.

Un-waive the TestNemotronNanoV3 accuracy variants (bf16/fp8 verified passing; nvfp4 to be confirmed in Blackwell CI).

Summary by CodeRabbit

• Bug Fixes

  • Disabled a model compilation option to avoid a known issue that could cause invalid numerical results in certain deployments.

• Chores

  • Reduced test exclusions for several deployment accuracy checks, allowing more validation to run automatically.

Description

Test Coverage

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR disables piecewise_enabled in the nano_v3.yaml compile_model config, adding a comment referencing an NVIDIA bug about NaNs from piecewise + chunked-prefill. It also removes five SKIP waiver entries for TestNemotronNanoV3 accuracy test variants in the integration test waiver list.

Changes

Config and Test Waiver Update

Layer / File(s)	Summary
Disable piecewise compilation examples/auto_deploy/model_registry/configs/nano_v3.yaml	piecewise_enabled changed from true to false with an added comment referencing nvbugs/6367792 (piecewise + chunked-prefill producing NaNs).
Remove NanoV3 accuracy test waivers tests/integration/test_lists/waives.txt	Removed five SKIP entries for TestNemotronNanoV3::test_accuracy variants (bf16, fp8, nvfp4 with attn_dp_off-trtllm, and one GB300 nvfp4 variant).

Estimated code review effort: 1 (Trivial) | ~3 minutes

Possibly related PRs

• NVIDIA/TensorRT-LLM#16081: Both PRs modify the same waives.txt file, adjusting SKIP entries for autodeploy accuracy tests.
• NVIDIA/TensorRT-LLM#16105: Both PRs modify waives.txt entries tied to nvbugs-referenced test waivers.
• NVIDIA/TensorRT-LLM#16109: Both PRs change accuracy/test_llm_api_autodeploy.py SKIP entries for Nemotron-family autodeploy variants.

Suggested reviewers: jieli-matrix, pcastonguay, yuxianq

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The description explains the issue and fix, but it omits the required sectioned template and leaves Test Coverage and checklist content incomplete.	Add the requested Description, Test Coverage, and PR Checklist sections, and list the relevant tests or waiver updates validating the change.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title is concise and accurately reflects the main change: disabling piecewise for Nano V3.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[tcherckez-nvidia]

/bot run

[tensorrt-cicd]

PR_Github #58473 [ run ] triggered by Bot. Commit: a699789 Link to invocation