Skip to content

[None][fix] Honor MSA block-index strides for q_len <= 32 - #17285

Open
peihu-nv wants to merge 1 commit into
NVIDIA:feat/m3_with_msafrom
peihu-nv:peihengh/m3-msa-selector-stride-feature
Open

[None][fix] Honor MSA block-index strides for q_len <= 32#17285
peihu-nv wants to merge 1 commit into
NVIDIA:feat/m3_with_msafrom
peihu-nv:peihengh/m3-msa-selector-stride-feature

Conversation

@peihu-nv

@peihu-nv peihu-nv commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator

@coderabbitai summary

Description

MiniMax-M3's legacy MSA sparse-attention path for q_len <= 32 can receive
kv_block_indexes as a non-contiguous, head-major tensor view. The SM100
kernel treated the underlying pointer as a contiguous
[query, KV head, top-k] allocation, which could select incorrect KV pages
and produce incorrect or non-finite attention output.

This change passes the tensor's query, KV-head, and top-k strides through the
existing MSA native interface and uses them for every sparse block-index load.
It also updates the bounds-check extent for the strided view. The attention
algorithm is unchanged, and no materialization, new kernel, or additional
buffer is introduced. The feature branch's existing downstream MSA patch and
TensorRT-LLM-collected test carry the change until it lands in MSA directly.

Test Coverage

  • Focused SM100 regression on GB300: contiguous and non-contiguous head-major
    selectors are finite and bit-identical for q_len 1 and 5 with one and two
    KV splits (4/4 combinations passed).
  • Five matched full AgentX pairs on GB300 using the MiniMax-M3 feature stack:
    all ten 1,800-second, default-sampling profiles completed validly with zero
    serving errors and no non-finite marker.
  • Focused CUDA timing measured 23.284 us for stride-aware attention and
    23.361 us for the compact reference; avoiding materialization saved
    11.161 us in the selector-plus-consumer path.
  • The combined downstream patch applies cleanly to the pinned MSA commit, and
    the six patched native-source hashes match the E2E-tested implementation.

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

  • Test cases are provided for new code paths (see test instructions)

  • If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

  • Any new dependencies have been scanned for license and vulnerabilities

  • CODEOWNERS updated if ownership changes

  • Documentation updated as needed

  • Update tava architecture diagram if there is a significant design change in PR.

  • The reviewers assigned automatically/manually are appropriate for the PR.

  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Signed-off-by: peihengh <259410613+peihu-nv@users.noreply.github.com>
@peihu-nv
peihu-nv requested review from a team as code owners August 5, 2026 06:00
@peihu-nv

peihu-nv commented Aug 5, 2026

Copy link
Copy Markdown
Collaborator Author

/bot run

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #63953 [ run ] triggered by Bot. Commit: 02db4d9 Link to invocation

@tensorrt-cicd

Copy link
Copy Markdown
Collaborator

PR_Github #63953 [ run ] completed with state SUCCESS. Commit: 02db4d9
/LLM/main/L0_MergeRequest_PR pipeline #51887 completed with status: 'SUCCESS'

CI Report

Link to invocation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants