为什么换了distillation模型后速度没有明显提升？我使用的是4090的gpu，cuda版本为11.8

[wutianzhe1997]

我的命令是
python sample_t2i.py --infer-mode fa --prompt "渔舟唱晚" --load-key distill --infer-steps 30

[WARNING] async_io requires the dev libaio .so object and headers but these were not found.
[WARNING] async_io: please install the libaio-devel package with yum
[WARNING] If libaio is already installed (perhaps from source), try setting the CFLAGS and LDFLAGS environment variables to where it can be found.
[WARNING] Please specify the CUTLASS repo directory as environment variable $CUTLASS_PATH
[WARNING] sparse_attn requires a torch version >= 1.5 and < 2.0 but detected 2.0
[WARNING] using untested triton version (2.0.0), only 1.0.0 is known to be compatible
2024-07-29 20:58:06.409 | INFO | hydit.inference:init:161 - Got text-to-image model root path: ckpts/t2i
2024-07-29 20:58:06.410 | INFO | hydit.inference:init:170 - Loading CLIP Text Encoder...
/home/cxdev/anaconda3/envs/HunyuanDiT_last/lib/python3.8/site-packages/torch/_utils.py:776: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
return self.fget.get(instance, owner)()
2024-07-29 20:58:08.567 | INFO | hydit.inference:init:173 - Loading CLIP Text Encoder finished
2024-07-29 20:58:08.568 | INFO | hydit.inference:init:176 - Loading CLIP Tokenizer...
2024-07-29 20:58:08.617 | INFO | hydit.inference:init:179 - Loading CLIP Tokenizer finished
2024-07-29 20:58:08.617 | INFO | hydit.inference:init:182 - Loading T5 Text Encoder and T5 Tokenizer...
You are using the default legacy behaviour of the <class 'transformers.models.t5.tokenization_t5.T5Tokenizer'>. This is expected, and simply means that the legacy (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set legacy=False. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in huggingface/transformers#24565
/home/cxdev/anaconda3/envs/HunyuanDiT_last/lib/python3.8/site-packages/transformers/convert_slow_tokenizer.py:550: UserWarning: The sentencepiece tokenizer that you are converting to a fast tokenizer uses the byte fallback option which is not implemented in the fast tokenizers. In practice this means that the fast version of the tokenizer can produce unknown tokens whereas the sentencepiece version would have converted these unknown tokens into a sequence of byte tokens matching the original piece of text.
warnings.warn(
You are using a model of type mt5 to instantiate a model of type t5. This is not supported for all configurations of models and can yield errors.
2024-07-29 20:58:23.349 | INFO | hydit.inference:init:187 - Loading t5_text_encoder and t5_tokenizer finished
2024-07-29 20:58:23.349 | INFO | hydit.inference:init:190 - Loading VAE...
2024-07-29 20:58:23.561 | INFO | hydit.inference:init:193 - Loading VAE finished
2024-07-29 20:58:23.561 | INFO | hydit.inference:init:197 - Building HunYuan-DiT model...
2024-07-29 20:58:23.611 | INFO | hydit.modules.models:init:214 - Enable Flash Attention.
2024-07-29 20:58:23.975 | INFO | hydit.modules.models:init:253 - Number of tokens: 4096
2024-07-29 20:58:43.166 | INFO | hydit.inference:load_torch_weights:295 - Loading torch model ckpts/t2i/model/pytorch_model_distill.pt...
2024-07-29 20:58:45.400 | INFO | hydit.inference:init:225 - Loading torch model finished
2024-07-29 20:58:45.400 | INFO | hydit.inference:init:250 - Loading inference pipeline...
2024-07-29 20:58:45.404 | DEBUG | hydit.inference:get_pipeline:128 - Using sampler: ddpm with scheduler: DDPMScheduler {
"_class_name": "DDPMScheduler",
"_diffusers_version": "0.30.0.dev0",
"beta_end": 0.02,
"beta_schedule": "scaled_linear",
"beta_start": 0.00085,
"clip_sample": false,
"clip_sample_range": 1.0,
"dynamic_thresholding_ratio": 0.995,
"num_train_timesteps": 1000,
"prediction_type": "v_prediction",
"rescale_betas_zero_snr": false,
"sample_max_value": 1.0,
"steps_offset": 1,
"thresholding": false,
"timestep_spacing": "leading",
"trained_betas": null,
"variance_type": "fixed_small"
}

2024-07-29 20:58:45.428 | INFO | hydit.inference:init:252 - Loading pipeline finished
2024-07-29 20:58:45.428 | INFO | hydit.inference:init:256 - ==================================================
2024-07-29 20:58:45.428 | INFO | hydit.inference:init:257 - Model is ready.
2024-07-29 20:58:45.429 | INFO | hydit.inference:init:258 - ==================================================
2024-07-29 20:58:45.429 | INFO | main:inferencer:21 - Loading DialogGen model (for prompt enhancement)...
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:06<00:00, 1.66s/it]
2024-07-29 20:58:54.814 | INFO | main:inferencer:23 - DialogGen model loaded.
2024-07-29 20:58:54.814 | INFO | main::34 - Prompt Enhancement...
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results.
Setting pad_token_id to eos_token_id:2 for open-end generation.
2024-07-29 20:59:02.857 | INFO | main::39 - Enhanced prompt: 一条小渔舟在夕阳下，悠悠地在平静的湖面上航行，船夫正在享受这份宁静。远处是渐渐暮色沉沉的天空，湖面波光粼粼，风格是中国水墨画。
2024-07-29 20:59:02.858 | INFO | main::44 - Generating images...
2024-07-29 20:59:02.858 | INFO | hydit.inference:predict:379 - Input (height, width) = (1024, 1024)
2024-07-29 20:59:02.858 | INFO | hydit.inference:predict:384 - Align to 16: (height, width) = (1024, 1024)
2024-07-29 20:59:02.859 | DEBUG | hydit.inference:predict:439 -
prompt: 渔舟唱晚
enhanced prompt: 一条小渔舟在夕阳下，悠悠地在平静的湖面上航行，船夫正在享受这份宁静。远处是渐渐暮色沉沉的天空，湖面波光粼粼，风格是中国水墨画。
seed: 42
(height, width): (1024, 1024)
negative_prompt: 错误的眼睛，糟糕的人脸，毁容，糟糕的艺术，变形，多余的肢体，模糊的颜色，模糊，重复，病态，残缺，
batch_size: 1
guidance_scale: 6.0
infer_steps: 30
image_meta_size: None

100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:11<00:00, 2.64it/s]
2024-07-29 20:59:14.872 | DEBUG | hydit.inference:predict:476 - Success, time: 12.013107776641846
2024-07-29 20:59:15.332 | INFO | main::73 - Save to results/23.png
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using tokenizers before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using tokenizers before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using tokenizers before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)
huggingface/tokenizers: The current process just got forked, after parallelism has already been used. Disabling parallelism to avoid deadlocks...
To disable this warning, you can either:
- Avoid using tokenizers before the fork if possible
- Explicitly set the environment variable TOKENIZERS_PARALLELISM=(true | false)