Skip to content

[None][fix] Honor MSA block-index strides for q_len <= 32 #17284

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
peihu-nv wants to merge 8 commits into
base: main
Choose a base branch
from
Draft

[None][fix] Honor MSA block-index strides for q_len <= 32 #17284

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
334db27
[None][perf] Avoid paged MSA K/V materialization during prefill
peihu-nv Jul 17, 2026
411708f
Merge branch 'main' into peihengh/m3-strided-paged-kv-main
peihu-nv Jul 27, 2026
5a38f2c
use fetch_content for patching
tburt-nv Jul 28, 2026
f494551
Merge branch 'main' into peihengh/m3-strided-paged-kv-main
peihu-nv Jul 28, 2026
235f552
[None][fix] Consolidate MSA wheel packaging
peihu-nv Jul 28, 2026
722bfd6
[None][fix] Address MSA packaging review feedback
peihu-nv Aug 5, 2026
ddeee3a
Merge remote-tracking branch 'upstream/main' into peihengh/m3-strided…
peihu-nv Aug 5, 2026
a92acdf
[None][fix] Honor MSA block-index strides for q_len <= 32
peihu-nv Aug 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ tensorrt_llm/pg_utils_bindings.*.so
tensorrt_llm/flash_mla
tensorrt_llm/flash_mla_cpp_tllm.*.so
tensorrt_llm/flash_mla_cpp_tllm.pyi
/3rdparty/fmha_sm100/
tensorrt_llm/runtime/kv_cache_manager_v2/**/*.so
**/*__mypyc*.so
tensorrt_llm/scripts
Expand Down
3 changes: 0 additions & 3 deletions .gitmodules
View file
Open in desktop

This file was deleted.

2 changes: 1 addition & 1 deletion 3rdparty/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,7 +135,7 @@ foreach(DEP_IDX RANGE ${DEP_COUNT_MINUS_ONE})
PATCH_COMMAND
bash
-c
"patch -p1 --forward --batch --dry-run -i '${_patch_file}' && patch -p1 --forward --batch -i '${_patch_file}' || echo 'Patch already applied, skipping.'"
"patch -p1 --forward --batch --dry-run -i '${_patch_file}' && patch -p1 --forward --batch -i '${_patch_file}' || patch -p1 --reverse --batch --dry-run -i '${_patch_file}' || (echo 'Patch state for ${DEP_NAME} is inconsistent. Remove ${CMAKE_BINARY_DIR}/_deps/${DEP_NAME}-src and reconfigure.' >&2 && false)"
)
endif()

Expand Down
1 change: 0 additions & 1 deletion 3rdparty/MSA
Open in desktop
Submodule MSA deleted from e2ebe7
7 changes: 7 additions & 0 deletions 3rdparty/fetch_content.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,13 @@
"git_shallow": true,
"source_subdir": "dont-add-this-project-with-add-subdirectory"
},
{
"name": "msa",
"git_repository": "https://gitlab.com/nvidia/tensorrt-llm/oss-components/msa.git",
"git_tag": "e2ebe7656649f619af0ad1d457b534283034655e",
"source_subdir": "dont-add-this-project-with-add-subdirectory",
"patch_file": "patches/msa_strided_paged_kv.patch"
},
{
"name": "nanobind",
"git_repository": "https://github.com/wjakob/nanobind",
Expand Down
Loading
Loading