New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix bugs for multi_latent_attention #1203

Open

xqiangx1991 wants to merge 1 commit into NVIDIA:main from xqiangx1991:main

xqiangx1991 commented Oct 9, 2024

The initialization parameters for DotProductAttention and TEDotProductAttention are different. Using DotProductAttention to construct MultiLatentAttention will result in an error.
In the MLASelfAttention module, the dimensions of k_pos_emb and k_no_pe differ in terms of num_attention_heads.torch.cat will result in an error.


          fix bugs for multi_latent_attention

afa43bb

github-actions bot commented Dec 8, 2024

Marking as stale. No activity in 60 days.

github-actions bot added the stale label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale