[ New features ] : add aiXcoder model implementation with tokenizer and validation #2902
Conversation
|
Thanks for your contribution! |
|
|
YoctoHan
force-pushed
the
feat/aixcoder
branch
from
310b58c to
45fe78e
Compare
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #2902 +/- ##
==========================================
Coverage ? 31.73%
==========================================
Files ? 425
Lines ? 68827
Branches ? 0
==========================================
Hits ? 21839
Misses ? 46988
Partials ? 0 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
YoctoHan
force-pushed
the
feat/aixcoder
branch
2 times, most recently
from
8db79fe to
68c92d8
Compare
YoctoHan
force-pushed
the
feat/aixcoder
branch
2 times, most recently
from
49ec0fd to
1b73a8c
Compare
YoctoHan
force-pushed
the
feat/aixcoder
branch
from
1b73a8c to
901cac8
Compare
… validation - Implemented the Aixcoder model architecture - Added custom tokenization logic - Completed initial validation tests (pre-training verification) - Prepared for upcoming training and fine-tuning validation - Documented usage in README for model reproduction
| # Row Linear | ||
| "aixcoder.embed_tokens.weight": partial(fn, is_column=False), | ||
| "aixcoder.layers.0.self_attn.o_proj.weight": partial(fn, is_column=False), | ||
| "aixcoder.layers.0.mlp.down_proj.weight": partial(fn, is_column=False), |
There was a problem hiding this comment.
权重的名字确定前缀是aixcoder而不是model吗，我在hugging face仓库里查到的权重名字前缀都是model?
There was a problem hiding this comment.
我们计划在星河社区开源 paddle 版本的权重，这个版本的权重前缀为 aiXcoder，接下来我会将前缀统一为 model，包括组网代码中的部分和权重。
| "aixcoder.layers.0.self_attn.q_proj.weight", | ||
| "aixcoder.layers.0.self_attn.k_proj.weight", | ||
| "aixcoder.layers.0.self_attn.v_proj.weight", | ||
| "aixcoder.layers.0.self_attn.qkv_proj.weight", |
| >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0] | ||
| "Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you." | ||
| ```""" | ||
| output_attentions = output_attentions if output_attentions is not None else self.config.output_attentions |
There was a problem hiding this comment.
output_attentions已经不再需要，辛苦在整个组网中去除这个逻辑
| def set_decoder(self, decoder): | ||
| self.aixcoder = decoder | ||
|
|
||
| def get_decoder(self): |
There was a problem hiding this comment.
以上get_xxx/set_xxx函数，请在PretrainedModel里确认，是否还需要写在组网里
| ] | ||
|
|
||
|
|
||
| class AixcoderPretrainedModel(PretrainedModel): |
There was a problem hiding this comment.
近期llama组网有重构更新，预计本周之内会合入，辛苦按照新组网的标准更新一下aiXcoder的组网，尤其需要关注_init_weights等在新组网中多余的部分
#2770
There was a problem hiding this comment.
请按照库中其他模型的方式，添加test_modeling单测文件，覆盖ModelTest，IntegrationTest等case。
https://github.com/PaddlePaddle/PaddleFormers/blob/develop/tests/transformers/qwen3/test_modeling.py
其他单测文件可以移除
There was a problem hiding this comment.
4 participants
✨ Pull Request Summary
🚀 Implemented Features
README.🧩 Code Quality
🧪 To‑Do
testsfolder.If any
codecovcoverage issues arise, please include corresponding test cases first.📂 PR Type
🧱 PR Changes
📝 Description
This PR introduces the AiXcoder model to PaddleFormers, including its complete model architecture, tokenizer, and validation for both pre-training and SFT/finetuning workflows.
All code components have passed
pre-commitchecks and are aligned with PaddleFormers’ coding and documentation standards.It expands the model zoo with a reproducible AiXcoder training pipeline ready for integration and further evaluation.
🧑💻 Checklist
pre-commitlint and format checksREADME, configs, or tutorials)