Optimize the Startup Configuration Process for the Hunyuan DiT Model #200
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
为待解决问题5:优化混元DiT模型启动配置流程(中级难度),提供解决方案
配置目录结构
参考MMEngine/config的配置风格和代码优化了混元DiT模型的启动配置流程,将配置参数按照数据、模型和启动流程划分,使用py文件配置模型参数;在需要新增配置文件时,可引用默认配置
新增
hydit/configs
目录用于存储启动配置文件,目录结构如下:启动流程
在加载配置时,为了保留原有的代码结构,新增配置加载文件
hydit/config_engine.py
,在train_deepspeed.py
中仅修改了函数get_args
的引用模块由于全参数训练和仅训练Lora都使用的deepspeed,所以新增
train_deepspeed.sh
脚本启动训练,启动命令如下:其中,
config
参数传递的为训练配置文件相对路径