[[TRTLLM-13948][feat] Clean up DeepSeek tests using CPP Transceiver v1 - #17090
[[TRTLLM-13948][feat] Clean up DeepSeek tests using CPP Transceiver v1#17090asfiyab-nvidia wants to merge 14 commits into
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
💤 Files with no reviewable changes (1)
WalkthroughDeepSeek disaggregated-serving tests now use NIXL with the Python transceiver runtime across accuracy scenarios and YAML configurations. UCX coverage is split by tensor parallel size. Test lists and duration records remove obsolete entries. ChangesDisaggregated-serving test coverage
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/integration/defs/accuracy/test_disaggregated_serving.py`:
- Line 1056: Update the test_gen_only_sync method with the required return type
annotation, using the appropriate annotation for its test method behavior.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 69392dbb-874d-41f6-9c44-e37bd47fcebe
📒 Files selected for processing (8)
tests/integration/defs/accuracy/test_disaggregated_serving.pytests/integration/test_lists/qa/llm_function_core.txttests/integration/test_lists/qa/llm_function_rtx6k.txttests/integration/test_lists/test-db/l0_dgx_b200.ymltests/integration/test_lists/test-db/l0_dgx_b300.ymltests/integration/test_lists/test-db/l0_dgx_h100.ymltests/integration/test_lists/test-db/l0_dgx_h200.ymltests/integration/test_lists/waives.txt
💤 Files with no reviewable changes (7)
- tests/integration/test_lists/test-db/l0_dgx_b300.yml
- tests/integration/test_lists/qa/llm_function_rtx6k.txt
- tests/integration/test_lists/test-db/l0_dgx_h100.yml
- tests/integration/test_lists/test-db/l0_dgx_h200.yml
- tests/integration/test_lists/test-db/l0_dgx_b200.yml
- tests/integration/test_lists/waives.txt
- tests/integration/test_lists/qa/llm_function_core.txt
| ids=["python", "cpp"]) | ||
| def test_gen_only_sync(self, transceiver_runtime): | ||
| """Test gen-only synchronous KV transfer with each NIXL runtime. | ||
| def test_gen_only_sync(self): |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Add the required return annotation.
-def test_gen_only_sync(self):
+def test_gen_only_sync(self) -> None:As per coding guidelines, every Python function must be annotated.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| def test_gen_only_sync(self): | |
| def test_gen_only_sync(self) -> None: |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/integration/defs/accuracy/test_disaggregated_serving.py` at line 1056,
Update the test_gen_only_sync method with the required return type annotation,
using the appropriate annotation for its test method behavior.
Source: Coding guidelines
chienchunhung
left a comment
chienchunhung
left a comment
There was a problem hiding this comment.
test_gen_only_sync is no longer parametrized, so pytest no longer generates the [python] node ID. Please update this test-list entry to ...::test_gen_only_sync; otherwise the QA selection references a nonexistent test and the retained synchronous-transfer coverage will not run.
|
Thanks for the effort! Please note, |
Thanks for the catch! Updated the test lists |
Thanks for the input! I have removed |
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/integration/defs/accuracy/test_disaggregated_serving.py (1)
1644-1650: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick winRemove the duplicate
cuda_graph_configkey.Ruff F601 rejects this dictionary because line 1650 duplicates line 1644. The second value overrides the first value.
Proposed fix
"max_batch_size": 128, "max_num_tokens": 1024, - "cuda_graph_config": None, "tensor_parallel_size": 4,🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/integration/defs/accuracy/test_disaggregated_serving.py` around lines 1644 - 1650, Remove the second duplicate “cuda_graph_config” entry from the configuration dictionary in the disaggregated serving test, keeping the existing first entry and all other settings unchanged.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@tests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp1_gentp1_deepseek_v3_lite_ucx.yaml`:
- Line 1: Add the repository-standard NVIDIA copyright header at the beginning
of the new YAML file, using the year of the latest meaningful modification, and
keep the existing hostname configuration unchanged after the header.
---
Outside diff comments:
In `@tests/integration/defs/accuracy/test_disaggregated_serving.py`:
- Around line 1644-1650: Remove the second duplicate “cuda_graph_config” entry
from the configuration dictionary in the disaggregated serving test, keeping the
existing first entry and all other settings unchanged.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 127777fa-40fc-4659-9814-cacd71df5f78
📒 Files selected for processing (31)
tests/integration/defs/.test_durationstests/integration/defs/accuracy/test_disaggregated_serving.pytests/integration/defs/disaggregated/test_configs/disagg_config_cache_aware_balance_deepseek_v3.yamltests/integration/defs/disaggregated/test_configs/disagg_config_cache_reuse_deepseek_v3.yamltests/integration/defs/disaggregated/test_configs/disagg_config_conditional_deepseek_v3.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp1_gentp1_deepseek_v3_lite.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp1_gentp1_deepseek_v3_lite_one_mtp.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp1_gentp1_deepseek_v3_lite_one_mtp_attention_dp_overlap.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp1_gentp1_deepseek_v3_lite_one_mtp_ctxpp2_gentp2.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp1_gentp1_deepseek_v3_lite_two_mtp.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp1_gentp1_deepseek_v3_lite_ucx.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp2_gentp2_deepseek_v3_lite.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp2_gentp2_deepseek_v3_lite_attention_dp.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp2_gentp2_deepseek_v3_lite_attention_dp_one.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp2_gentp2_deepseek_v3_lite_attention_dp_one_mtp.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp2_gentp2_deepseek_v3_lite_attention_dp_overlap.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp2_gentp2_deepseek_v3_lite_attention_dp_overlap_cuda_graph.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp2_gentp2_deepseek_v3_lite_nixl.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp2_gentp2_deepseek_v3_lite_overlap_cuda_graph.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp2_gentp2_deepseek_v3_lite_transceiver_runtime_python.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp2ep2pp2_gentp4_deepseek_v3_lite_one_mtp_block_reuse.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp2ep2pp2_gentp4_deepseek_v3_lite_one_mtp_block_reuse_chunked.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp4_gentp4_deepseek_r1_v2_fp4_tllm.yamltests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp4_gentp4_deepseek_r1_v2_fp4_tllm_mtp.yamltests/integration/defs/disaggregated/test_configs/disagg_config_deepseek_v3_lite_empty_batch.yamltests/integration/defs/disaggregated/test_configs/disagg_config_gentp2_deepseek_v3_lite_attention_dp_gen_only.yamltests/integration/defs/disaggregated/test_disaggregated.pytests/integration/test_lists/qa/llm_function_core.txttests/integration/test_lists/test-db/l0_dgx_b200.ymltests/integration/test_lists/test-db/l0_dgx_b300.ymltests/integration/test_lists/test-db/l0_dgx_h100.yml
💤 Files with no reviewable changes (3)
- tests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp2_gentp2_deepseek_v3_lite_transceiver_runtime_python.yaml
- tests/integration/test_lists/test-db/l0_dgx_b300.yml
- tests/integration/defs/.test_durations
🚧 Files skipped from review as they are similar to previous changes (1)
- tests/integration/test_lists/qa/llm_function_core.txt
| @@ -0,0 +1,17 @@ | |||
| hostname: localhost | |||
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win
Add the NVIDIA copyright header.
This new file starts with hostname at Line 1 and has no NVIDIA copyright header. Add the repository-standard header with the year of the latest meaningful modification before the YAML content.
As per coding guidelines, all new files must contain the NVIDIA copyright header with the year of the latest meaningful modification. The PR objective identifies this as a new TP1 UCX configuration.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@tests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp1_gentp1_deepseek_v3_lite_ucx.yaml`
at line 1, Add the repository-standard NVIDIA copyright header at the beginning
of the new YAML file, using the year of the latest meaningful modification, and
keep the existing hostname configuration unchanged after the header.
Source: Coding guidelines
|
/bot run --disable-fail-fast --extra-stage "DGX_H100-4_GPUs-PyTorch-DeepSeek-1" |
1 similar comment
|
/bot run --disable-fail-fast --extra-stage "DGX_H100-4_GPUs-PyTorch-DeepSeek-1" |
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
… coverage of DSV3.2 Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
…run with Python transceiver Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
…2_nixl_python Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
…defs/disaggregated/test_configs Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
…me_python config from test Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
…thon and associated test list entries Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
… associated test list entries Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
…_gpu and associated test list entries" This reverts commit d1eb996.
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
8b95114 to
5536469
Compare
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
Signed-off-by: Asfiya Baig <asfiyab@nvidia.com>
|
/bot run --disable-fail-fast --extra-stage "DGX_H100-4_GPUs-PyTorch-DeepSeek-1" |
|
|
|
/bot run --disable-fail-fast --extra-stage "DGX_H100-4_GPUs-PyTorch-DeepSeek-1" |
|
PR_Github #64126 [ run ] triggered by Bot. Commit: |
|
PR_Github #64126 [ run ] completed with state
|
7 participants
Description
This PR removes the DeepSeek tests that use the CPP Transceiver V1. Summary of changes:
Test Changes in
test_disaggregated_serving.pyAll tests that are not removed have been updated to use the NIXL backend with the Python Transceiver runtime if they weren't already.
TestDeepSeekV32Exptest_kv_cache_v2_nixl_python- redundant withtest_auto_dtype(besides TP settings)TestDeepSeekV3LiteRetained tests:
test_gen_only_sync- Updated test and corresponding test-lists to be non-parametrized over the Transceiver runtimetest_gen_only_spec_dectest_auto_dtypetest_guided_decodingtest_gen_firsttest_auto_dtype_with_helix- Uses UCX backend with CPP runtime and does not support Python transceiver execution path. This test has been left unchanedRemoved tests:
test_nixl_backend- redundant withtest_auto_dtypetest_kv_cache_v2_nixl_python- shares redundancies withtest_auto_dtypeandtest_guided_decodingThe tests that were removed were also deleted from corresponding test lists
Test Changes in
tests/integration/defs/disaggregated/test_configs/disagg_config_ctxtp2_gentp2_deepseek_v3_lite_transceiver_runtime_python.yamlas it doesn't test anything uniquely anymore post the transceiver migrationdisagg_config_ctxtp2_gentp1cp2_deepseek_v3_lite_bf16_tllm_gen.yaml,disagg_config_ctxtp2_gentp2_deepseek_v3_lite_mpi.yaml,disagg_config_ctxtp2_gentp2_deepseek_v3_lite_ucx.yamlunchangedTest Coverage
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.Dev Engineer Review
test_gen_only_syncentry to use the non-parameterized node ID.QA Engineer Review
Modified or removed test functions:
TestDeepSeekV3Lite.test_gen_only_sync; covered by QA and CI test lists.TestDeepSeekV3Lite.test_nixl_backend; no test-list entries remain.TestDeepSeekV3Lite.test_auto_dtype; no test-list entries remain.TestDeepSeekV32Exp.test_guided_decoding; covered by existing test-list coverage.TestDeepSeekV32Exp.test_kv_cache_v2_nixl_python; no test-list entries remain.test_disaggregated_deepseek_v3_lite_fp8_transceiver_runtime_python; no test-list entries remain.Retained DeepSeek R1 KV-cache v2 NIXL Python and DeepSeek V3.2 auto-dtype coverage in
qa/llm_function_core.txt.Verdict: sufficient.