add qwen2/3 old fused qkv/ffn #3150
Conversation
|
Thanks for your contribution! |
| ) | ||
|
|
||
| return actions | ||
| if config.tie_word_embeddings: |
There was a problem hiding this comment.
这里没有tie_word_embeddings的概念了吧？formers默认是 base_actions["lm_head.weight"] = partial(fn, is_column=False) 这个吧
| final_actions = {} | ||
|
|
||
| base_actions = { | ||
| "lm_head.weight": partial(fn, is_column=True), |
|
|
||
| mappings = make_base_actions() | ||
|
|
||
| if config.tie_word_embeddings: |
|
|
||
| LAYER_ROWWISE = ["self_attn.o_proj.weight"] | ||
| base_actions = { | ||
| "lm_head.weight": partial(fn, is_column=True), |
|
完善PR描述内容，新增测试等。 |
| { | ||
| f"{cls.base_model_prefix}.layers.{layer_idx}.mlp.experts.{e}.{k}": partial( | ||
| fn, is_column=True | ||
| ) |
There was a problem hiding this comment.
fuse_attention_ffn的时候需要输入 is_naive_2fuse=True
| for k in LAYER_COLWISE | ||
| } | ||
| ) | ||
| # colwise |
There was a problem hiding this comment.
缺少fuse ffn, 然后注意 is_naive_2fuse 的添加
| "mlp.gate_proj.weight", | ||
| ] | ||
|
|
||
| FUSE_LAYER_COLWISE = [ |
Codecov Report❌ Patch coverage is ❌ Your patch status has failed because the patch coverage (55.20%) is below the target coverage (75.00%). You can increase the patch coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## develop #3150 +/- ##
==========================================
Coverage ? 36.74%
==========================================
Files ? 416
Lines ? 71324
Branches ? 0
==========================================
Hits ? 26207
Misses ? 45117
Partials ? 0 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
4 participants
No description provided.