Skip to content

Computation communication overlap #249

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Draft
wants to merge 236 commits into
base: main
Choose a base branch
from
Draft

Computation communication overlap #249

wants to merge 236 commits into from

Conversation

fzyzcjy
Copy link
Contributor

@fzyzcjy fzyzcjy commented Jun 24, 2025 •
edited
Loading

WIP

Some quick dump:

  • Can use few SMs to run communication while most SMs to run computation. DeepEP with few SMs in low latency mode will almost still run in full speed.
    • The change is pretty simple to support few-SM, just change num_warp_groups etc
    • I can provide more info and experiment results if needed
  • I am waiting for fp4 kernel before doing more computation communication overlap, that's why this draft is pending and no progress at all
    • but anyway the code here does show overlap and speedup b/t combine send and (the fp8 DeepGEMM) down gemm

@fzyzcjy fzyzcjy changed the title Computation communication overlap Computation communication overlap, use few SMs for low-latency mode, etc Jul 3, 2025
@fzyzcjy fzyzcjy changed the title Computation communication overlap, use few SMs for low-latency mode, etc Computation communication overlap Jul 3, 2025
@fzyzcjy fzyzcjy mentioned this pull request Jul 3, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@fzyzcjy