[None][feat] Support Inkling-NVFP4 model

[KleinBlueC]

Dev Engineer Review

• Adds PyTorch support for InklingForConditionalGeneration with hybrid attention, relative-position bias, short-convolution state, sigmoid-gated MoE routing, NVFP4 decoding, and BF16 vision/audio towers.
• Adds Triton attention kernels, multimodal preprocessing, checkpoint mapping, KVCacheManagerV2, CUDA-graph handling, reasoning parsing, and evaluation integration.
• Adds configuration registration, EOS fallback handling, quantization-value normalization, and public model exports.
• Adds documentation and Blackwell accuracy references for GSM8K, MMLU, and MMMU.
• Review focus:
  • Validate Triton kernel correctness, cache page handling, relative-bias shapes, sliding-window masking, and decode output reuse.
  • Validate request-scoped convolution state cleanup and CUDA-graph preparation across warmup, capture, replay, and steady-state execution.
  • Validate NVFP4 expert weight mapping, tensor slicing, padded LM-head trimming, and complete checkpoint accounting.
  • Check multimodal placeholder expansion, image/audio/video geometry, dtype handling, and unsupported hash-prefix caching behavior.
  • Check API consistency, error handling, performance, and configuration values against CODING_GUIDELINES.md.
  • Investigate reported failures in L0_MergeRequest_PR pipelines 51386, 51419, 51449, 51605, 51668, and 51832 before another CI run.

QA Engineer Review

• Added model tests:
  • TestInkling_NVFP4.test_nvfp4 in text and multimodal accuracy suites.
  • Inkling registration, geometry, layer classification, checkpoint patterns, weight accounting, and tensor-shape tests.
  • Vision, audio, video, processor, placeholder, token-ID, and cache-prefix tests.
• Added parser tests for typed content, reasoning, tool invocation, control-token boundaries, streaming, registration, and special-token handling.
• Added sampling-parameter tests for EOS precedence, EOS lists, explicit stops, and missing EOS values.
• Added LM evaluation tests for post-processing resolution, token-limit preservation, special-token retention, and Inkling/MMMU extraction.
• Added CI and QA entries in:
  • tests/integration/test_lists/qa/llm_function_core.txt
  • tests/integration/test_lists/test-db/l0_b200.yml
• The added accuracy tests are covered in both QA and post-merge test lists.
• Verdict: needs follow-up because the reported L0_MergeRequest_PR pipelines failed and CBTS coverage data is unavailable.

Description

Adds PyTorch-backend support for Inkling (thinkingmachines/Inkling-NVFP4), a
RoPE-free hybrid-attention MoE reasoning model with vision and audio towers.
Registered as InklingForConditionalGeneration.

Architecture

• No RoPE — position comes from a learned relative-position bias added
  pre-softmax, per (query token, head, relative distance).
• Hybrid attention — 55 local sliding-window layers (window 512, 16 KV heads)
  interleaved with 11 global full-causal layers (8 KV heads).
• Four causal short convolutions per layer — two inside attention (K, V) and two
  on the residual stream, carrying per-request state across decode steps.
• Sigmoid-gated MoE — top-6 of 256 routed experts with an additive selection
  bias, then a log-sigmoid renorm spanning the routed and two shared-expert logits.
• muP + unpadded vocab — hidden states divided by logits_mup_width_multiplier
  before the head; logits sliced from 201024 down to 200058.
• Multimodal — hMLP vision tower and dMel audio codebook tower, both BF16, each
  fusing one row per patch / per frame into the text embedding stream. Video is
  multi-frame images; there is no separate video encoder.

NVFP4 covers the text decoder and routed experts (layers 3–65); layer-2 experts,
attention, shared experts, and both towers stay BF16 per the checkpoint's
hf_quant_config.json.

Implementation

Area	Notes
modeling_inkling.py	Decoder stack, MoE, short convolutions, multimodal wrapper
attention_backend/inkling_triton.py	Paged prefill + decode Triton kernels applying the relative bias as a score_mod. No existing fused backend exposes such a hook (context FMHA is disabled for kRELATIVE, trtllm-gen rejects a relative bias, FlashInfer has no additive per-token bias). rel_logits is static-shape, so the decode kernel is CUDA-graph capturable
modeling_inkling_multimodal.py	Vision tower, audio tower, video frame sampling, and the shared input processor that expands one <image> placeholder per patch and one <audio> per dMel frame. Pure numpy + torch preprocessing
configs/inkling.py	Config classes rebuilt from the checkpoint's nested dicts (transformers does not ship Inkling), plus the per-layer classification helpers used by both the model and KV-cache sizing
checkpoints/hf/inkling_weight_mapper.py	Checkpoint accounting (every key is consumed-text / deferred / unaccounted) and load-path remapping, including de-interleaving the gate/up projections
pyexecutor/*	Registers the short-conv state pool under a new CONV_STATE_MANAGER. Pool rows and attention decode metadata are published into stable CUDA buffers eagerly from input prep, so the captured decode forward does no host→device copy and each replay reads the current batch (same stable-pointer pattern as Mamba2Metadata)

KV cache: the per-layer KV-head split (local 16 / global 8) structurally requires
KVCacheManagerV2 — V1's unified pool would coerce it to one value and mis-size the
per-layer KV bytes, a correctness bug. The model defaults to V2 and the incompatible
path raises rather than silently downgrading.

Serving / eval plumbing (small, each needed end to end):

• --reasoning_parser inkling for Inkling's typed-content blocks, with streaming.
• needs_raw_special_tokens so a delimiter-based reasoning parser actually sees its
  markers (previously only the tool-parser path preserved them).
• end_id falls back to the config's eos_token_id — checkpoints whose terminator
  lives only in config.json otherwise never stop and run to max_tokens.
• hf_quant_config.json may spell "no quantization" as the string "none", which
  previously reached QuantAlgo("none") and raised.
• Warmup dummy ids are bounded to the head vocab; a vocab padded above the output
  vocab otherwise fails KV-cache estimation with "Token ID out of range".
• --post_process_fn inkling / inkling_mmmu for trtllm-eval.

Accuracy

Measured on the complete datasets, TRT-LLM and SGLang side by side at TP=4 with
CUDA graph and the overlap scheduler on, batch 8, driven by the same client so
prompt rendering and scoring are shared code:

Task	TRT-LLM	SGLang	Δ
GSM8K (1319)	96.13 (1268/1319)	96.59 (1274/1319)	−0.45 pt
MMLU (14042, 5-shot)	85.35 (11985/14042)	85.48 (12003/14042)	−0.13 pt
MMMU (858)	79.72 (684/858)	78.32 (672/858)	+1.40 pt

Not supported in this release

MTP / speculative decoding (the checkpoint ships next-N draft weights; nothing
builds or loads them), LoRA, function calling, constrained/guided decoding, EPD
disaggregated serving, and multimodal-hash prefix caching (refused loudly per
modality, so multimodal requests still run — just uncached).

One workaround worth flagging for review: every Inkling all-reduce is rebuilt with
ONESHOT after construction. Under CUDA-graph capture a symmetric all-reduce
corrupts the run when its send buffer is unregistered while its recv buffer is a
registered NCCL window at a 12288 B message — which Inkling hits exactly (hidden
6144, bf16, one decode token), sending the first global-attention layer non-finite.
The all-reduces involved are built by generic modules (attention o_proj, MoE
down_proj), so pinning the strategy afterwards keeps the mitigation model-local.

Test Coverage

Accuracy (integration) — added to l0_b200 and llm_function_core:

• accuracy/test_llm_api_pytorch.py::TestInkling_NVFP4::test_nvfp4 — GSM8K + MMLU
  on the text decoder.
• accuracy/test_llm_api_pytorch_multimodal.py::TestInkling_NVFP4::test_nvfp4 —
  MMMU on the vision path.

Unit — CPU-only, no checkpoint / GPU / network needed, all well under a second:

• unittest/_torch/modeling/test_modeling_inkling.py — config parsing,
  registration, per-layer classification; plus checkpoint-gated weight accounting
  and tensor-shape checks that read only the safetensors index and skip cleanly
  when the checkpoint is absent.
• unittest/_torch/modeling/test_modeling_inkling_multimodal.py — the three media
  paths and the input processor on synthetic configs: hMLP scale plan and module
  tree, the fold's value preservation, dMel preprocessing, the audio codebook
  forward against a reference, frame sampling, and the fail-loud placeholder
  contract.
• unittest/llmapi/test_reasoning_parser.py — full-parse and streaming equivalence
  for the Inkling parser, including control tokens split across delta boundaries.
• unittest/llmapi/test_sampling_params.py — the end_id config fallback.
• unittest/others/test_lm_eval.py — the offline post-processing hook.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[tensorrt-cicd]

PR_Github #63648 [ run ] completed with state FAILURE. Commit: 3d079c0
/LLM/main/L0_MergeRequest_PR pipeline #51605 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[KleinBlueC]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #63713 [ run ] triggered by Bot. Commit: 3d079c0 Link to invocation

[tensorrt-cicd]

PR_Github #63713 [ run ] completed with state FAILURE. Commit: 3d079c0
/LLM/main/L0_MergeRequest_PR pipeline #51668 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[KleinBlueC]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #63889 [ run ] triggered by Bot. Commit: 3d079c0 Link to invocation

[tensorrt-cicd]

PR_Github #63889 [ run ] completed with state FAILURE. Commit: 3d079c0
/LLM/main/L0_MergeRequest_PR pipeline #51832 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[coderabbitai]

Actionable comments posted: 13

🧹 Nitpick comments (8)

tensorrt_llm/_torch/models/checkpoints/hf/inkling_weight_mapper.py (1)

329-336: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Document the interleaved w13_weight layout. The checkpoint stores [g0, u0, ...]; _split_interleaved_gate_up correctly maps even rows to w1 and odd rows to w3. Update lines 331–332 and add a sentinel-row test.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/_torch/models/checkpoints/hf/inkling_weight_mapper.py` around
lines 329 - 336, Update the docstring for the per-expert mapping method to state
that w13_weight uses the interleaved [g0, u0, …] layout, with even rows mapped
to w1 and odd rows to w3, rather than describing gate and up rows as contiguous
halves. Add a sentinel-row test covering _split_interleaved_gate_up to verify
the even/odd mapping.

tensorrt_llm/_torch/configs/inkling.py (2)

140-142: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Return a precise element type from _local_ids.

The annotation is the bare set. Use set[int] so callers get the element type.

♻️ Proposed change

     `@property`
-    def _local_ids(self) -> set:
+    def _local_ids(self) -> set[int]:
         return set(self.local_layer_ids)

As per coding guidelines: "Annotate every function, use None for procedures, avoid unnecessary Any and type: ignore, prefer built-in generic types".

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/_torch/configs/inkling.py` around lines 140 - 142, Update the
_local_ids property return annotation from bare set to set[int], preserving its
existing set(self.local_layer_ids) implementation.

Source: Coding guidelines

---

241-250: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Annotate _as_config and the two __init__ methods.

_as_config has no parameter or return annotation. Both __init__ methods have no -> None. The guidelines require an annotation on every function.

♻️ Proposed change

     `@staticmethod`
-    def _as_config(value):
+    def _as_config(value: PretrainedConfig | dict | None) -> PretrainedConfig | None:
         if value is None or isinstance(value, PretrainedConfig):
             return value

Also add -> None to InklingTextConfig.__init__ (line 89) and InklingConfig.__init__ (line 220).

As per coding guidelines: "Annotate every function, use None for procedures".

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/_torch/configs/inkling.py` around lines 241 - 250, Annotate the
static method _as_config with appropriate parameter and return types, and add ->
None to both InklingTextConfig.__init__ and InklingConfig.__init__. Preserve the
existing conversion and initialization behavior.

Source: Coding guidelines

tests/unittest/llmapi/test_reasoning_parser.py (1)

919-941: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Create a fresh parser for each parse call.

The test reuses one parser instance for three parse calls. That relies on parse being stateless. If the parser later keeps buffer state between calls, this test fails for a reason unrelated to the assertions it makes.

♻️ Proposed change

-    parser = ReasoningParserFactory.create_reasoning_parser("inkling")
-    r = parser.parse(
+    def _parse(text):
+        return ReasoningParserFactory.create_reasoning_parser("inkling").parse(text)
+
+    r = _parse(
         f'{INK_CH}need tool{INK_EM}{INK_MM}<|content_invoke_tool_json|>{{"n":1}}{INK_EM}{INK_END}'
     )
     assert r.content == '{"n":1}'
     assert r.reasoning_content == "need tool"
-    r = parser.parse(
+    r = _parse(
         f'{INK_CH}need tool{INK_EM}{INK_MM}<|content_invoke_tool_text|>lookup{INK_EM}{INK_END}'
     )
     assert r.content == "lookup"
     assert r.reasoning_content == "need tool"
     rep = f"{INK_CH}reason{INK_EM}" + f"{INK_MM}{INK_CT}Answer: A{INK_EM}{INK_END}" * 6
-    r = parser.parse(rep)
+    r = _parse(rep)

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unittest/llmapi/test_reasoning_parser.py` around lines 919 - 941,
Update test_inkling_reasoning_parser_tool_and_repetition_segmentation to create
a fresh parser via ReasoningParserFactory.create_reasoning_parser("inkling")
before each of its three parse calls, rather than reusing one parser instance
across calls.

tensorrt_llm/_torch/models/modeling_inkling_multimodal.py (1)

436-444: 🎯 Functional Correctness | 🔵 Trivial | 💤 Low value

Replace the assert with an explicit check.

Python removes assert under -O. If n_layers < 1, idxs has one element, and idxs[0] = 0 then idxs[-1] = len(scales) - 1 write the same slot. The returned plan then holds a single scale, and InklingVisionModel.__init__ builds zero layers. Validate n_layers at the top of the function instead.

🐛 Proposed fix

     if patch_size <= 1:
         raise ValueError("patch_size must be greater than 1")
+    if n_layers < 1:
+        raise ValueError("n_layers must be at least 1")
@@
-    assert len(idxs) >= 2
     idxs[0] = 0
     idxs[-1] = len(scales) - 1

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/_torch/models/modeling_inkling_multimodal.py` around lines 436 -
444, Replace the `assert len(idxs) >= 2` in the scale-planning function with an
explicit validation of `n_layers` at the function entry, rejecting values below
2 before computing or mutating `idxs`. Preserve the existing index assignment
and returned scale plan for valid layer counts.

tensorrt_llm/_torch/models/modeling_inkling.py (2)

883-885: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Remove the truncated comment at the end of __init__.

The comment stops mid-sentence and describes buffers that this class no longer owns. The decode metadata buffers now live in InklingAttentionMetadata (ink_seq_lens / ink_page_table). A reader looks for a statement that does not exist.

♻️ Proposed cleanup

         self.local_num_heads = num_heads // tp_size
-        # Stable GPU buffers for the CUDA-graph-safe runtime decode metadata,
-        # refreshed eagerly (before capture/replay) by the model engine via

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/_torch/models/modeling_inkling.py` around lines 883 - 885,
Remove the truncated stable GPU buffers comment at the end of __init__,
including its continuation, since the buffers are owned by
InklingAttentionMetadata rather than this class.

---

429-435: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Reduce host-side work on the decode path in InklingConvRuntime.build.

build runs once per forward. attn_metadata.seq_lens.tolist() forces a device-to-host transfer when seq_lens is a CUDA tensor, and write_state_indices allocates a fresh torch.tensor(slots, ...) on line 276 for every step. Both land on the decode critical path.

Two changes remove them:

• Read the already-materialized host sequence lengths (attn_metadata.seq_lens is built from a pinned CPU tensor in _prepare_tp_inputs) instead of calling .tolist() on a device tensor.
• Fill state_indices_cpu[:n] in place from the Python list rather than building a temporary tensor.

♻️ Proposed change for the staging write

         n = len(slots)
-        self.state_indices_cpu[:n].copy_(torch.tensor(slots, dtype=torch.int32))
+        sl_np = self.state_indices_cpu[:n].numpy()
+        for i, slot in enumerate(slots):
+            sl_np[i] = slot
         self.state_indices[:n].copy_(self.state_indices_cpu[:n], non_blocking=True)

Measure the decode step time before and after.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/_torch/models/modeling_inkling.py` around lines 429 - 435,
Reduce decode-path host overhead in InklingConvRuntime.build by reusing the host
sequence lengths prepared by _prepare_tp_inputs instead of calling .tolist() on
attn_metadata.seq_lens. Update write_state_indices to populate
state_indices_cpu[:n] in place from the Python slots list, avoiding a temporary
torch.tensor allocation. Measure decode step time before and after the changes.

tensorrt_llm/_torch/attention_backend/inkling_triton.py (1)

482-494: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Build the eager page table with a single host-to-device copy.

build_page_table allocates one CUDA tensor per request row inside the loop. Each torch.tensor(valid, device=device) is its own H2D transfer. This path is the eager decode fallback, so it runs whenever the metadata was not published.

♻️ Proposed refactor

     batch = len(block_ids_per_seq)
-    pt = torch.zeros((batch, max_pages), dtype=torch.int32, device=device)
-    for i, blocks in enumerate(block_ids_per_seq):
-        valid = [int(b) for b in blocks if int(b) >= 0]
-        if valid:
-            pt[i, : len(valid)] = torch.tensor(valid, dtype=torch.int32, device=device)
-    return pt
+    host = torch.zeros((batch, max_pages), dtype=torch.int32)
+    host_np = host.numpy()
+    for i, blocks in enumerate(block_ids_per_seq):
+        valid = [b for b in map(int, blocks) if b >= 0][:max_pages]
+        if valid:
+            host_np[i, : len(valid)] = valid
+    return host.to(device=device, non_blocking=True)

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/_torch/attention_backend/inkling_triton.py` around lines 482 -
494, Update build_page_table to assemble all filtered valid block IDs into a
host-side dense representation first, then create or copy the complete [batch,
max_pages] int32 page table to device once outside the row loop. Preserve zero
padding and each row’s existing ordering and placement.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tensorrt_llm/_torch/models/checkpoints/hf/inkling_weight_mapper.py`:
- Around line 260-274: Complete type annotations in the mapper: annotate the
_text_config property with InklingTextConfig, change preprocess_weights and
related interfaces from unparameterized Dict to dict[str, torch.Tensor],
annotate regex match parameters as re.Match[str], and use None for methods
without a return value. Add equivalent parameter and return annotations to the
nested _assign function, covering the additional methods around the later
referenced section without changing behavior.

In `@tensorrt_llm/_torch/models/modeling_inkling_multimodal.py`:
- Around line 900-921: Replace the assert in sample_video_frames with an
explicit ValueError when total_frames is not positive, preventing empty videos
from reaching np.linspace. Also update plan_out_scales at
tensorrt_llm/_torch/models/modeling_inkling_multimodal.py:436-444 to perform a
top-of-function n_layers < 1 check that raises explicitly instead of relying on
assert len(idxs) >= 2.
- Around line 189-197: Update the media-loading boundary around
BaseMediaIO.async_load/load_file so untrusted requests cannot read arbitrary
bare paths or file:// URLs; reject them or enforce an approved media-root
allowlist before opening files. Ensure InklingImagePreprocessor receives only
already-decoded media objects or validated paths, while preserving supported
URL/data handling.

In `@tensorrt_llm/_torch/models/modeling_inkling.py`:
- Around line 1-39: Run pre-commit formatting across all files and apply the
resulting changes: in tensorrt_llm/_torch/models/modeling_inkling.py lines 1-39,
apply ruff-format; in tensorrt_llm/_torch/attention_backend/inkling_triton.py
lines 560-686, apply all seven Ruff auto-fixes and ruff-format, including the
InklingAttentionMetadata body; in
tensorrt_llm/_torch/pyexecutor/conv_state_manager.py lines 55-86, apply
ruff-format to wrap the write_conv_state_indices signature.

In `@tensorrt_llm/_torch/pyexecutor/model_engine.py`:
- Around line 3811-3815: Correct the `_maybe_prepare_inkling_runtime` docstring
to state the actual callers, replacing `_prepare_tp_inputs_no_cache` with
`_prepare_star_attention_inputs` if that is the intended eager path. Do not
claim publication occurs in `_prepare_tp_inputs_no_cache` unless adding the
helper call there is explicitly required.

In `@tensorrt_llm/evaluate/post_processing.py`:
- Around line 223-224: Update the no-marker passthrough condition in the
post-processing logic to use _INK_CONTROL_RE.search(text) is None, so all
Inkling control tokens—including framing-only headers—enter parser handling. Add
coverage for framing-only input and output truncated immediately after the
header, preserving passthrough only when no control token is present.

In `@tensorrt_llm/llmapi/llm_args.py`:
- Around line 5067-5069: Regenerate the LLM args golden manifest using
scripts/generate_llm_args_golden_manifest.py so the attn_backend entry includes
the newly added INKLING telemetry value, then commit the updated generated
manifest.

In `@tensorrt_llm/llmapi/reasoning_parser.py`:
- Around line 719-729: Replace bare list annotations with list[str] for the
content and reasoning accumulator parameters of _emit() and _consume(), and for
the batch-parser accumulators at tensorrt_llm/llmapi/reasoning_parser.py lines
751-752, streaming-parser accumulators at lines 781-782, and finish-parser
accumulators at lines 793-794; preserve the existing string-appending behavior.

In `@tensorrt_llm/sampling_params.py`:
- Around line 554-559: Update the EOS fallback handling around self.end_id and
self.stop_token_ids to skip any stop_token equal to self.end_id, matching the
generation-config path while retaining other unique stop tokens. Add a
regression test covering repeated primary EOS IDs such as [7, 7, 8], asserting 7
is excluded from stop_token_ids.

In `@tests/integration/defs/accuracy/test_llm_api_pytorch_multimodal.py`:
- Around line 721-726: Update TestInkling_NVFP4.test_nvfp4 and its LLM
configuration to use tensor_parallel_size=4, matching MPI and device-memory
gating, and document the MMMU reference configuration. Ensure the same runtime
requirements are reflected for the existing test-list entries in l0_b200.yml and
llm_function_core.txt.

In `@tests/integration/defs/accuracy/test_llm_api_pytorch.py`:
- Around line 8140-8172: Update the test_nvfp4 task loop to pass a fresh
SamplingParams instance to each task.evaluate call instead of reusing the
class-level self.sampling_params object. Preserve the existing MAX_NUM_TOKENS
configuration while allowing each task to independently set
truncate_prompt_tokens without mutating shared state.

In `@tests/unittest/_torch/modeling/test_modeling_inkling.py`:
- Line 1: Run the configured ruff formatter on test_modeling_inkling.py and
commit the resulting formatting changes, without altering its behavior or
unrelated files.
- Around line 90-107: Add a focused test alongside
test_model_defaults_pin_v2_and_disable_block_reuse that constructs an Inkling
configuration with transceiver_runtime set to "CPP" and invokes
KvCacheCreator._fallback_if_unsupported_kv_cache_manager_v2(). Assert it raises
the Inkling-specific NotImplementedError and that the message recommends
transceiver_runtime='PYTHON', preserving the V2-only KV-head geometry
requirement.

---

Nitpick comments:
In `@tensorrt_llm/_torch/attention_backend/inkling_triton.py`:
- Around line 482-494: Update build_page_table to assemble all filtered valid
block IDs into a host-side dense representation first, then create or copy the
complete [batch, max_pages] int32 page table to device once outside the row
loop. Preserve zero padding and each row’s existing ordering and placement.

In `@tensorrt_llm/_torch/configs/inkling.py`:
- Around line 140-142: Update the _local_ids property return annotation from
bare set to set[int], preserving its existing set(self.local_layer_ids)
implementation.
- Around line 241-250: Annotate the static method _as_config with appropriate
parameter and return types, and add -> None to both InklingTextConfig.__init__
and InklingConfig.__init__. Preserve the existing conversion and initialization
behavior.

In `@tensorrt_llm/_torch/models/checkpoints/hf/inkling_weight_mapper.py`:
- Around line 329-336: Update the docstring for the per-expert mapping method to
state that w13_weight uses the interleaved [g0, u0, …] layout, with even rows
mapped to w1 and odd rows to w3, rather than describing gate and up rows as
contiguous halves. Add a sentinel-row test covering _split_interleaved_gate_up
to verify the even/odd mapping.

In `@tensorrt_llm/_torch/models/modeling_inkling_multimodal.py`:
- Around line 436-444: Replace the `assert len(idxs) >= 2` in the scale-planning
function with an explicit validation of `n_layers` at the function entry,
rejecting values below 2 before computing or mutating `idxs`. Preserve the
existing index assignment and returned scale plan for valid layer counts.

In `@tensorrt_llm/_torch/models/modeling_inkling.py`:
- Around line 883-885: Remove the truncated stable GPU buffers comment at the
end of __init__, including its continuation, since the buffers are owned by
InklingAttentionMetadata rather than this class.
- Around line 429-435: Reduce decode-path host overhead in
InklingConvRuntime.build by reusing the host sequence lengths prepared by
_prepare_tp_inputs instead of calling .tolist() on attn_metadata.seq_lens.
Update write_state_indices to populate state_indices_cpu[:n] in place from the
Python slots list, avoiding a temporary torch.tensor allocation. Measure decode
step time before and after the changes.

In `@tests/unittest/llmapi/test_reasoning_parser.py`:
- Around line 919-941: Update
test_inkling_reasoning_parser_tool_and_repetition_segmentation to create a fresh
parser via ReasoningParserFactory.create_reasoning_parser("inkling") before each
of its three parse calls, rather than reusing one parser instance across calls.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 54bc7c1e-088b-4aec-9c43-3ed5e6c6cded

📥 Commits

Reviewing files that changed from the base of the PR and between 7e2fca0 and 56796ab.

📒 Files selected for processing (37)

• docs/source/models/supported-models.md
• tensorrt_llm/_torch/attention_backend/inkling_triton.py
• tensorrt_llm/_torch/attention_backend/utils.py
• tensorrt_llm/_torch/configs/__init__.py
• tensorrt_llm/_torch/configs/inkling.py
• tensorrt_llm/_torch/model_config.py
• tensorrt_llm/_torch/models/__init__.py
• tensorrt_llm/_torch/models/checkpoints/__init__.py
• tensorrt_llm/_torch/models/checkpoints/hf/inkling_weight_mapper.py
• tensorrt_llm/_torch/models/modeling_inkling.py
• tensorrt_llm/_torch/models/modeling_inkling_multimodal.py
• tensorrt_llm/_torch/pyexecutor/_util.py
• tensorrt_llm/_torch/pyexecutor/config_utils.py
• tensorrt_llm/_torch/pyexecutor/conv_state_manager.py
• tensorrt_llm/_torch/pyexecutor/model_engine.py
• tensorrt_llm/_torch/pyexecutor/py_executor_creator.py
• tensorrt_llm/_torch/pyexecutor/resource_manager.py
• tensorrt_llm/evaluate/lm_eval.py
• tensorrt_llm/evaluate/post_processing.py
• tensorrt_llm/llmapi/llm_args.py
• tensorrt_llm/llmapi/llm_utils.py
• tensorrt_llm/llmapi/reasoning_parser.py
• tensorrt_llm/sampling_params.py
• tensorrt_llm/serve/openai_server.py
• tests/integration/defs/accuracy/references/gsm8k.yaml
• tests/integration/defs/accuracy/references/mmlu.yaml
• tests/integration/defs/accuracy/references/mmmu.yaml
• tests/integration/defs/accuracy/test_llm_api_pytorch.py
• tests/integration/defs/accuracy/test_llm_api_pytorch_multimodal.py
• tests/integration/test_lists/qa/llm_function_core.txt
• tests/integration/test_lists/test-db/l0_b200.yml
• tests/unittest/_torch/executor/test_mamba_cache_manager.py
• tests/unittest/_torch/modeling/test_modeling_inkling.py
• tests/unittest/_torch/modeling/test_modeling_inkling_multimodal.py
• tests/unittest/llmapi/test_reasoning_parser.py
• tests/unittest/llmapi/test_sampling_params.py
• tests/unittest/others/test_lm_eval.py

🚧 Files skipped from review as they are similar to previous changes (18)

• tensorrt_llm/_torch/models/checkpoints/init.py
• tensorrt_llm/_torch/models/init.py
• tests/integration/defs/accuracy/references/mmlu.yaml
• tensorrt_llm/_torch/model_config.py
• tests/integration/defs/accuracy/references/mmmu.yaml
• tests/integration/test_lists/qa/llm_function_core.txt
• docs/source/models/supported-models.md
• tests/integration/test_lists/test-db/l0_b200.yml
• tensorrt_llm/_torch/pyexecutor/resource_manager.py
• tensorrt_llm/_torch/pyexecutor/py_executor_creator.py
• tensorrt_llm/llmapi/llm_utils.py
• tensorrt_llm/serve/openai_server.py
• tests/unittest/llmapi/test_sampling_params.py
• tensorrt_llm/_torch/configs/init.py
• tensorrt_llm/_torch/pyexecutor/config_utils.py
• tensorrt_llm/_torch/pyexecutor/_util.py
• tests/unittest/others/test_lm_eval.py
• tensorrt_llm/evaluate/lm_eval.py

[coderabbitai]

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Trace how multi_modal_data reaches input processors: decoded objects vs raw strings.
rg -nP --type=py -C 5 'multi_modal_data' tensorrt_llm/serve tensorrt_llm/inputs | head -120
rg -nP --type=py -C 5 'def (load_image|load_audio|async_load_image|default_multimodal_input_loader)' tensorrt_llm/inputs | head -80

Repository: NVIDIA/TensorRT-LLM

Length of output: 11861

---

🏁 Script executed:

#!/bin/bash
set -eu

echo '--- multimodal processor symbols ---'
ast-grep outline tensorrt_llm/_torch/models/modeling_inkling_multimodal.py
echo '--- relevant processor code ---'
sed -n '150,230p' tensorrt_llm/_torch/models/modeling_inkling_multimodal.py
sed -n '640,715p' tensorrt_llm/_torch/models/modeling_inkling_multimodal.py
echo '--- loader implementations ---'
sed -n '55,235p' tensorrt_llm/inputs/utils.py
sed -n '760,850p' tensorrt_llm/inputs/utils.py
echo '--- serving call sites and request conversion ---'
sed -n '1460,1565p' tensorrt_llm/serve/openai_server.py
sed -n '1680,1740p' tensorrt_llm/serve/openai_server.py
rg -n -P --type=py -C 8 'default_multimodal_input_loader|async_load_image|load_image\(|load_audio\(|mm_data_tracker|retrieve_all_sync' tensorrt_llm/inputs tensorrt_llm/serve

Repository: NVIDIA/TensorRT-LLM

Length of output: 50375

---

🏁 Script executed:

#!/bin/bash
set -eu

echo '--- chat media parsing and decoding ---'
sed -n '120,230p' tensorrt_llm/serve/chat_utils.py
rg -n -P -C 8 'def _make_media_io|class .*Media|async_load|read\(\)|retrieve_all|add_data' tensorrt_llm/serve/chat_utils.py tensorrt_llm/inputs/utils.py

echo '--- tracker storage and retrieval ---'
sed -n '300,450p' tensorrt_llm/inputs/utils.py
sed -n '930,975p' tensorrt_llm/inputs/utils.py

echo '--- Inkling input processor dispatch ---'
sed -n '930,1135p' tensorrt_llm/_torch/models/modeling_inkling_multimodal.py
rg -n -P -C 8 'multi_modal_data|data\["|data.get|_to_pil_rgb|_to_waveform|image_preprocessor|audio_preprocessor' tensorrt_llm/_torch/models/modeling_inkling_multimodal.py

echo '--- external request input construction ---'
rg -n -P -C 6 'generate_async\(inputs=|inputs\s*=\s*prompt|multi_modal_data' tensorrt_llm/serve tensorrt_llm/llmapi tests/unittest/_torch/modeling/test_modeling_inkling_multimodal.py

Repository: NVIDIA/TensorRT-LLM

Length of output: 50375

---

🏁 Script executed:

#!/bin/bash
set -eu

echo '--- media IO selection and async_load implementations ---'
sed -n '90,125p' tensorrt_llm/serve/chat_utils.py
fd -i 'media_io.py' tensorrt_llm
sed -n '1,280p' tensorrt_llm/inputs/media_io.py

echo '--- Inkling text and tokenized call paths ---'
sed -n '1080,1285p' tensorrt_llm/_torch/models/modeling_inkling_multimodal.py

Repository: NVIDIA/TensorRT-LLM

Length of output: 18462

---

🏁 Script executed:

#!/bin/bash
set -eu

rg -n -P '^class (BaseMediaIO|ImageMediaIO|AudioMediaIO|VideoMediaIO)|def (async_load|load)\(' tensorrt_llm/inputs/media_io.py
sed -n '520,930p' tensorrt_llm/inputs/media_io.py

Repository: NVIDIA/TensorRT-LLM

Length of output: 13322

---

🏁 Script executed:

#!/bin/bash
set -eu

echo '--- serving entry points that accept multimodal input directly ---'
rg -n -P -C 8 'multi_modal_data|generate_async\(inputs=|generate\(inputs=|prompt_token_ids' \
  tensorrt_llm/serve tensorrt_llm/llmapi \
  -g '*.py' | head -400

echo '--- direct LLM input normalization ---'
rg -n -P -C 8 'class (LLM|Request)|def generate|TextPrompt|TokenPrompt|multi_modal_data' \
  tensorrt_llm/llmapi tensorrt_llm/executor tensorrt_llm/inputs \
  -g '*.py' | head -400

Repository: NVIDIA/TensorRT-LLM

Length of output: 50375

---

🏁 Script executed:

#!/bin/bash
set -eu

python3 - <<'PY'
import ast
from pathlib import Path

files = {
    "inkling": Path("tensorrt_llm/_torch/models/modeling_inkling_multimodal.py"),
    "media_io": Path("tensorrt_llm/inputs/media_io.py"),
    "llm": Path("tensorrt_llm/llmapi/llm.py"),
}

trees = {name: ast.parse(path.read_text()) for name, path in files.items()}

def find_function(tree, name):
    for node in ast.walk(tree):
        if isinstance(node, (ast.FunctionDef, ast.AsyncFunctionDef)) and node.name == name:
            return node
    raise AssertionError(name)

inkling_call = find_function(trees["inkling"], "call_with_text_prompt")
media_async = find_function(trees["media_io"], "async_load")
media_image_file = find_function(trees["media_io"], "load_file")
llm_preprocess = find_function(trees["llm"], "_preprocess")

def source(node, tree):
    return ast.get_source_segment(tree, node) or ""

assert "_to_pil_rgb" not in source(inkling_call, trees["inkling"])
assert "self._text_processor" in source(inkling_call, trees["inkling"])
assert 'parsed.scheme in ("", "file")' in source(media_async, trees["media_io"])
assert "load_file" in source(media_async, trees["media_io"])
assert "Path(_normalize_file_uri(url))" in source(media_image_file, trees["media_io"])
assert '"multi_modal_data" in inputs' in source(llm_preprocess, trees["llm"])

print("Inkling receives media after the serving MediaIO path.")
print("BaseMediaIO.async_load dispatches bare paths and file:// URLs to load_file.")
print("The direct LLM dictionary API forwards multi_modal_data to the input processor.")
PY

Repository: NVIDIA/TensorRT-LLM

Length of output: 522

---

Path Traversal (CWE-22): Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal')

Reachability: External · Exploitability: Moderate

Reachability path

● Entry
  tensorrt_llm/evaluate/lm_eval.py:530
  MultimodalLmEvalWrapper: Multimodal wrapper for lm-evaluation-harness that handles vision-language models. This wrapper extends the base LmEvalWrapper to support …
│
▼
● Sink
  tensorrt_llm/_torch/models/modeling_inkling_multimodal.py

Reject untrusted local media paths at the media boundary.

OpenAI requests decode media before Inkling receives it, but BaseMediaIO.async_load accepts bare paths and file:// URLs and calls load_file without an allowlist. An attacker can therefore make the server read a local media file. Reject local paths for untrusted requests, or restrict them to an approved media root and pass only decoded objects to Inkling.

🧰 Tools

🪛 ast-grep (0.45.0)

[warning] 195-195: File path is request-/variable-derived; validate and normalize to prevent path traversal.
Context: open(path, "rb")
Note: [CWE-22] Improper Limitation of a Pathname to a Restricted Directory ('Path Traversal').

(open-filename-from-request)

---

[warning] 189-189: Do not make http calls without encryption
Context: "http://"
Note: [CWE-319] Cleartext Transmission of Sensitive Information.

(requests-http)

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/_torch/models/modeling_inkling_multimodal.py` around lines 189 -
197, Update the media-loading boundary around BaseMediaIO.async_load/load_file
so untrusted requests cannot read arbitrary bare paths or file:// URLs; reject
them or enforce an approved media-root allowlist before opening files. Ensure
InklingImagePreprocessor receives only already-decoded media objects or
validated paths, while preserving supported URL/data handling.

Source: Linters/SAST tools

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[coderabbitai]

Actionable comments posted: 1

🧹 Nitpick comments (5)

tensorrt_llm/_torch/attention_backend/inkling_triton.py (2)

479-491: 🚀 Performance & Scalability | 🔵 Trivial | ⚡ Quick win

Build the page table with one host-to-device copy.

Line 490 runs inside the per-request loop, so this helper issues one H2D transfer per request. The eager decode fallback in tensorrt_llm/_torch/models/modeling_inkling.py calls it on every decode step, so the cost scales with batch size on the latency-critical path. Assemble a single padded host list and transfer it once.

Add annotations for block_ids_per_seq and device as well.

♻️ Proposed fix

-def build_page_table(block_ids_per_seq, max_pages: int, device) -> torch.Tensor:
+def build_page_table(
+    block_ids_per_seq: Sequence[Sequence[int]],
+    max_pages: int,
+    device: torch.device,
+) -> torch.Tensor:
     """Pack a ragged ``block_ids_per_seq`` (from
     ``KVCacheManagerV2.get_batch_cache_indices``) into a dense
     ``[batch, max_pages]`` int32 page table, padding short rows with 0 (never
     read: the decode kernel bounds every access by the per-request ``seq_len``).
     """
     batch = len(block_ids_per_seq)
-    pt = torch.zeros((batch, max_pages), dtype=torch.int32, device=device)
-    for i, blocks in enumerate(block_ids_per_seq):
-        valid = [int(b) for b in blocks if int(b) >= 0]
-        if valid:
-            pt[i, : len(valid)] = torch.tensor(valid, dtype=torch.int32, device=device)
-    return pt
+    rows = [[0] * max_pages for _ in range(batch)]
+    for row, blocks in zip(rows, block_ids_per_seq):
+        valid = [int(b) for b in blocks if int(b) >= 0]
+        row[: len(valid)] = valid[:max_pages]
+    return torch.tensor(rows, dtype=torch.int32, device=device)

Add the import at the top of the file:

from collections.abc import Sequence

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/_torch/attention_backend/inkling_triton.py` around lines 479 -
491, Update build_page_table to annotate block_ids_per_seq and device, and
replace the per-row tensor assignments with a single host-side padded
representation of all valid block IDs, then perform one host-to-device tensor
construction/copy for the complete [batch, max_pages] int32 page table. Preserve
zero padding and filtering of negative IDs.

Source: Coding guidelines

---

341-347: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Both Triton wrappers validate shapes with assert. Python invoked with -O removes every assert, so the head-divisibility and rel_logits checks vanish in production. The kernels then mis-map cur_head // kv_group_num or read the bias tensor out of bounds, with no error. Raise ValueError at both sites.

• tensorrt_llm/_torch/attention_backend/inkling_triton.py#L341-L347: raise ValueError for a non-divisible num_heads/num_kv_heads pair, and for a non-contiguous or wrongly shaped rel_logits; include the [total_tokens, num_heads, rel_extent] shape check.
• tensorrt_llm/_torch/attention_backend/inkling_triton.py#L426-L432: apply the same two ValueError checks against k_cache.shape[1] and the [batch, num_heads, rel_extent] bias shape.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/_torch/attention_backend/inkling_triton.py` around lines 341 -
347, Replace the assert-based validation in both wrapper
sites—tensorrt_llm/_torch/attention_backend/inkling_triton.py:341-347 and
:426-432—with unconditional ValueError checks. Validate head divisibility using
num_kv_heads at the first site and k_cache.shape[1] at the second, and validate
rel_logits contiguity plus the required [total_tokens, num_heads, rel_extent]
and [batch, num_heads, rel_extent] shapes respectively.

tensorrt_llm/_torch/configs/inkling.py (2)

140-142: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Parameterize the _local_ids return type.

set is unparameterized. Use set[int] so the element type is explicit.

♻️ Proposed change

     `@property`
-    def _local_ids(self) -> set:
+    def _local_ids(self) -> set[int]:
         return set(self.local_layer_ids)

As per coding guidelines, "prefer built-in generic types".

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/_torch/configs/inkling.py` around lines 140 - 142, Update the
_local_ids property return annotation from unparameterized set to set[int],
preserving its existing return value and behavior.

Source: Coding guidelines

---

241-250: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Annotate _as_config and the InklingConfig.__init__ sub-config parameters.

_as_config has no parameter or return annotation. The text_config, audio_config, vision_config, and mtp_config parameters are also unannotated. Add PretrainedConfig | dict | None style annotations.

♻️ Proposed change

     `@staticmethod`
-    def _as_config(value):
+    def _as_config(
+        value: PretrainedConfig | dict | None,
+    ) -> PretrainedConfig | None:
         if value is None or isinstance(value, PretrainedConfig):
             return value

As per coding guidelines, "Annotate every function".

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tensorrt_llm/_torch/configs/inkling.py` around lines 241 - 250, Annotate the
static method _as_config with PretrainedConfig | dict | None for its value
parameter and return type, and add the same style of annotations to the
text_config, audio_config, vision_config, and mtp_config parameters in
InklingConfig.__init__. Preserve the existing conversion and initialization
behavior.

Source: Coding guidelines

tests/unittest/_torch/modeling/test_modeling_inkling_multimodal.py (1)

316-466: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Test coverage summary (unit tests) and one gap.

1. Added test functions: test_vision_scale_plan_and_module_tree, test_scale_plan_is_a_strictly_growing_progression, test_fold_timespace_to_depth_is_value_preserving, test_vision_tower_emits_one_row_per_patch, test_vision_tower_load_weights_is_strict, test_dmel_preprocess_shape_and_range, test_dmel_multi_clip_concat_and_empty_clip, test_audio_tower_matches_codebook_reference, test_audio_tower_optional_norm_and_bin_count_guard, test_sample_video_frames, test_sample_video_as_images_preserves_count_and_order, test_video_frames_expand_to_one_image_span_each, test_registration, test_serving_and_profiler_interface_contract, test_text_only_passthrough, test_image_placeholder_expands_to_num_patches, test_audio_placeholder_expands_to_num_frames, test_image_and_audio_expand_independently, test_call_with_token_ids_expands_and_passes_text_through, test_mm_token_ids_cover_both_modalities, test_fail_loud_on_placeholder_media_mismatch, test_fail_loud_on_audio_placeholder_mismatch, test_multimodal_hash_prefix_cache_is_refused_per_modality. No test is modified or removed.
2. Test-list registration: this file lives under tests/unittest/, so CI runs it through pytest tests/unittest/. No entry in tests/integration/test_lists/ is required.
3. Coverage verdict: sufficient for the vision, audio, video, and placeholder paths, with one gap. InklingInputProcessor.__init__ now rejects a shared image_token_id / audio_token_id (modeling_inkling_multimodal.py lines 983-990). No test exercises that branch. Add one case that builds the processor with both ids equal and expects ValueError.

💚 Proposed test

+def test_shared_image_audio_token_id_is_rejected():
+    with pytest.raises(ValueError, match="must "):
+        _processor(image_token_id=200054, audio_token_id=200054)

As per path instructions, "Always produce a test coverage summary, even if no issues are found."

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/unittest/_torch/modeling/test_modeling_inkling_multimodal.py` around
lines 316 - 466, Add a unit test covering the validation branch in
InklingInputProcessor.__init__: construct the processor with identical
image_token_id and audio_token_id values and assert that ValueError is raised.
Keep the existing processor tests unchanged and use the established _processor
setup or constructor arguments to exercise the shared-token rejection.

Source: Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/unittest/_torch/modeling/test_modeling_inkling.py`:
- Around line 52-68: Update the checkpoint helper annotations: change _index()
to return dict[str, str], _exclude_modules() to return set[str], and
_safetensors_shape() to return tuple[list[int], str]. Keep their existing
implementations and behavior unchanged.

---

Nitpick comments:
In `@tensorrt_llm/_torch/attention_backend/inkling_triton.py`:
- Around line 479-491: Update build_page_table to annotate block_ids_per_seq and
device, and replace the per-row tensor assignments with a single host-side
padded representation of all valid block IDs, then perform one host-to-device
tensor construction/copy for the complete [batch, max_pages] int32 page table.
Preserve zero padding and filtering of negative IDs.
- Around line 341-347: Replace the assert-based validation in both wrapper
sites—tensorrt_llm/_torch/attention_backend/inkling_triton.py:341-347 and
:426-432—with unconditional ValueError checks. Validate head divisibility using
num_kv_heads at the first site and k_cache.shape[1] at the second, and validate
rel_logits contiguity plus the required [total_tokens, num_heads, rel_extent]
and [batch, num_heads, rel_extent] shapes respectively.

In `@tensorrt_llm/_torch/configs/inkling.py`:
- Around line 140-142: Update the _local_ids property return annotation from
unparameterized set to set[int], preserving its existing return value and
behavior.
- Around line 241-250: Annotate the static method _as_config with
PretrainedConfig | dict | None for its value parameter and return type, and add
the same style of annotations to the text_config, audio_config, vision_config,
and mtp_config parameters in InklingConfig.__init__. Preserve the existing
conversion and initialization behavior.

In `@tests/unittest/_torch/modeling/test_modeling_inkling_multimodal.py`:
- Around line 316-466: Add a unit test covering the validation branch in
InklingInputProcessor.__init__: construct the processor with identical
image_token_id and audio_token_id values and assert that ValueError is raised.
Keep the existing processor tests unchanged and use the established _processor
setup or constructor arguments to exercise the shared-token rejection.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6e31d991-e70c-46cf-afb1-0706305d8372

📥 Commits

Reviewing files that changed from the base of the PR and between 7608520 and 3d079c0.

📒 Files selected for processing (33)

• docs/source/models/supported-models.md
• tensorrt_llm/_torch/attention_backend/inkling_triton.py
• tensorrt_llm/_torch/configs/__init__.py
• tensorrt_llm/_torch/configs/inkling.py
• tensorrt_llm/_torch/model_config.py
• tensorrt_llm/_torch/models/__init__.py
• tensorrt_llm/_torch/models/checkpoints/__init__.py
• tensorrt_llm/_torch/models/checkpoints/hf/inkling_weight_mapper.py
• tensorrt_llm/_torch/models/modeling_inkling.py
• tensorrt_llm/_torch/models/modeling_inkling_multimodal.py
• tensorrt_llm/_torch/pyexecutor/_util.py
• tensorrt_llm/_torch/pyexecutor/config_utils.py
• tensorrt_llm/_torch/pyexecutor/model_engine.py
• tensorrt_llm/_torch/pyexecutor/py_executor_creator.py
• tensorrt_llm/_torch/pyexecutor/resource_manager.py
• tensorrt_llm/evaluate/lm_eval.py
• tensorrt_llm/evaluate/post_processing.py
• tensorrt_llm/llmapi/llm_utils.py
• tensorrt_llm/llmapi/reasoning_parser.py
• tensorrt_llm/sampling_params.py
• tensorrt_llm/serve/openai_server.py
• tests/integration/defs/accuracy/references/gsm8k.yaml
• tests/integration/defs/accuracy/references/mmlu.yaml
• tests/integration/defs/accuracy/references/mmmu.yaml
• tests/integration/defs/accuracy/test_llm_api_pytorch.py
• tests/integration/defs/accuracy/test_llm_api_pytorch_multimodal.py
• tests/integration/test_lists/qa/llm_function_core.txt
• tests/integration/test_lists/test-db/l0_b200.yml
• tests/unittest/_torch/modeling/test_modeling_inkling.py
• tests/unittest/_torch/modeling/test_modeling_inkling_multimodal.py
• tests/unittest/llmapi/test_reasoning_parser.py
• tests/unittest/llmapi/test_sampling_params.py
• tests/unittest/others/test_lm_eval.py

🚧 Files skipped from review as they are similar to previous changes (22)

• tests/integration/defs/accuracy/references/gsm8k.yaml
• tensorrt_llm/_torch/model_config.py
• tests/integration/defs/accuracy/references/mmmu.yaml
• tensorrt_llm/llmapi/llm_utils.py
• tensorrt_llm/_torch/models/checkpoints/init.py
• tests/integration/test_lists/test-db/l0_b200.yml
• tensorrt_llm/_torch/models/init.py
• tensorrt_llm/_torch/pyexecutor/resource_manager.py
• docs/source/models/supported-models.md
• tensorrt_llm/_torch/pyexecutor/_util.py
• tensorrt_llm/serve/openai_server.py
• tests/unittest/llmapi/test_sampling_params.py
• tests/integration/defs/accuracy/references/mmlu.yaml
• tests/integration/test_lists/qa/llm_function_core.txt
• tensorrt_llm/_torch/pyexecutor/config_utils.py
• tensorrt_llm/_torch/configs/init.py
• tensorrt_llm/_torch/pyexecutor/py_executor_creator.py
• tests/unittest/others/test_lm_eval.py
• tensorrt_llm/sampling_params.py
• tensorrt_llm/_torch/pyexecutor/model_engine.py
• tensorrt_llm/evaluate/lm_eval.py
• tensorrt_llm/_torch/models/modeling_inkling.py