Skip to content

Updated for PyTorch 2.13 RC #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
menicosia wants to merge 4 commits into
base: main
Choose a base branch
from
+7 −0
Open

Updated for PyTorch 2.13 RC #1

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b6b5ba2
Updated for PyTorch 2.13 RC
menicosia Jun 29, 2026
31395a2
Merge remote-tracking branch 'upstream'
menicosia Jul 7, 2026
bf84abf
catch up with Nick's report versioning approach, add last few changes…
menicosia Jul 7, 2026
41c9c87
I have no idea what I did to alter 2026-03 report; restoring to nick'…
menicosia Jul 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions pytorch/feature-gap-reports/2026-06-pytorch-feature-gap-report.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,7 @@ Python helpers such as `_should_use_multimem_all_gather_matmul` and `_multimem_a

**Status / next steps:** Track rocSHMEM feature growth; avoid labeling multicast-dependent APIs as cross-vendor without runtime guards.

> Nick's notes: confer with Prachi, we've integrated rocSHMEM for all the features it can support; but rocSHMEM itself is limited by the hardware - expect changes beyond MI455.
---

### 5. Tensor Memory Accelerator (TMA) and CUDA-Specific Codegen
Expand Down Expand Up @@ -200,6 +201,8 @@ This means ROCm may take TMA-named code paths in templates but without true TMA

**Status / next steps:** Classify TMA-dependent optimizations as CUDA-only in docs; pursue AMD-specific memory hierarchy optimizations (buffer loads, LDS tiling) as the ROCm counterpart.

> Nick's notes: Nick and Glen are working on this. Tensor Memory Accelerator (Nvidia), ours is Tensor Data Mover (TDM). First version will be in MI450. This falls under MI450 umbrella [TODO] - Marco to follow up on these specific stories so that we can announce TMA support with MI450 launch.

---

### 6. FP4 / NVFP4 / MXFP4 and Advanced Low-Precision GEMM
Expand Down Expand Up @@ -230,6 +233,8 @@ NVFP4/MXFP4 dtype support (`float4_e2m1fn_x2`) exists at the type level, but per

**Status / next steps:** Needs verification on whether AITER/MSLK ROCm paths will cover MXFP4 grouped GEMM; track `gfx950` MX GEMM maturity separately from NVFP4.

> Nick's notes: Not sure if FP8 sparse really exists? Follow up with Jagadish. MSLK is the new name for FBGEMM (aka FaceBook GEMM, a meta library that support some low precision data types). This is not just about FP8. Story behind NVFP4; AMD chose emulation. FB and MS expressed that they wanted HW support, for NVFP4. Will this be supported in MI450?

---

### 7. Grouped GEMM and Scaled Grouped GEMM Fast Paths
Expand Down Expand Up @@ -300,6 +305,8 @@ FlexAttention is not globally disabled on ROCm — platform detection uses `torc

**Status / next steps:** Triage ROCm combo-kernel issues; confirm which skips indicate missing functionality vs. test infra gaps.

> Nick's notes: Bring this up with Jack, find where falls on his list of priorities

---

### 10. CUDA-Only Runtime, Allocator, and Tooling Features
Expand Down