[None][doc] Show trtllm-bench KV host cache config

[zcxGGmu]

Summary

• Add a trtllm-bench PyTorch workflow section for KV cache host offloading.
• Show that host cache capacity is configured through --config using kv_cache_config.host_cache_size.
• Clarify that --kv_host_cache_bytes is not a top-level trtllm-bench option.

Test Plan

• Static doc smoke check for the new section, YAML example, and required option names
• git diff --check
• Not run: full Sphinx build (python -m sphinx --version fails locally with No module named sphinx)

Refs #6912

Dev Engineer Review

• Added trtllm-bench documentation for PyTorch KV cache host offloading.
• Documents nested TorchLlmArgs settings through --config, including kv_cache_config.host_cache_size.
• Documents that the smaller limit applies when both max_tokens and host_cache_size are set.
• Removes the invalid --kv_host_cache_bytes reference.
• Static documentation checks and git diff --check passed.
• Full Sphinx validation was not run because Sphinx was unavailable locally.

QA Engineer Review

No test changes.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 5d43a937-1178-4e07-897e-733cb6c24d2e

📥 Commits

Reviewing files that changed from the base of the PR and between e47bac8 and edc1fdb.

📒 Files selected for processing (1)

• docs/source/developer-guide/perf-benchmarking.md

🚧 Files skipped from review as they are similar to previous changes (1)

• docs/source/developer-guide/perf-benchmarking.md

---

Walkthrough

The benchmarking guide documents PyTorch benchmark KV cache host offloading. It adds YAML configuration for kv_cache_config.host_cache_size, a trtllm-bench throughput example using --config, and the effective capacity limit when max_tokens is also configured.

Changes

KV cache host offloading

Layer / File(s)	Summary
Benchmarking guide configuration docs/source/developer-guide/perf-benchmarking.md	Adds the host offloading section to the table of contents. Documents nested YAML options, --config usage, host-memory reuse for evicted or paused KV cache blocks, and the smaller-capacity limit when max_tokens is set.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Suggested reviewers: chang-l, nv-guomingz

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the documentation change for trtllm-bench KV host cache configuration.
Description check	✅ Passed	The description explains the documentation change, configuration details, validation steps, and the unavailable Sphinx test.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/source/developer-guide/perf-benchmarking.md`:
- Around line 276-279: After the YAML example, document that when both
max_tokens and host_cache_size are configured, the runtime allocates host-cache
memory based on the smaller effective limit; clarify that host_cache_size:
1073741824 does not always result in a 1 GiB host cache.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 2c891c87-75ca-4876-a9c9-1019789959af

📥 Commits

Reviewing files that changed from the base of the PR and between 67579d8 and e47bac8.

📒 Files selected for processing (1)

• docs/source/developer-guide/perf-benchmarking.md

[brnguyen2]

Docs-only, accurate: kv_cache_config.host_cache_size matches the KvCacheConfig field, and the kvcache.md anchor resolves. Fine to merge; two optional wording touch-ups inline.

[brnguyen2]

"It is not a top-level trtllm-bench option such as --kv_host_cache_bytes" names a flag that has never existed, which reads oddly in reference docs and invites someone to search for it. The first paragraph already makes the point ("does not expose every LLM API option as a top-level CLI flag"). Suggest dropping this sentence.

[brnguyen2]

Worth stating the general rule once instead of the KV-specific one: the --config YAML accepts any TorchLlmArgs field, so this same mechanism covers cuda_graph_config, moe_config, etc. That saves the next doc section for the next un-flagged option.

[zcxGGmu]

Thanks for the review — I pushed edc1fdbda addressing the feedback.

Changes made:

• Clarified that --config can pass any TorchLlmArgs field, including nested settings such as kv_cache_config, cuda_graph_config, and moe_config.
• Removed the reference to the nonexistent --kv_host_cache_bytes option.
• Documented that when both kv_cache_config.max_tokens and kv_cache_config.host_cache_size are set, the runtime uses the smaller token-capacity limit.

Validation:

• python static documentation guard checks — passed for required wording, removed flag reference, link target, and balanced code fences.
• git diff --check — passed.
• Added-line secret scan — passed.

Note: Full Sphinx docs build was not run locally because sphinx is not installed in this environment.