Environment
OS: Windows
GPU: NVIDIA RTX A6000
Compute Capability: 8.6 (SM86)
PyTorch: 2.5.1
CUDA: 12.1
Triton: 3.7.1
Model: Qwen-Image-Edit-2509
Quantization: FP8 E4M3FN Scaled
Task: I2I
Resolution: 832x480
use_bfloat16: True
dit_quantized: True
dit_quant_scheme: fp8-sgl
infer_steps: 8
Diffusers version: 0.36.0.dev0
Checkpoint
I am using the following checkpoint:
qwen_image_edit_2509_fp8_e4m3fn_scaled.safetensors
The checkpoint is successfully found and loaded:
E:\project\Qwen-Image-Lightning\Qwen-Image-Edit-2509\qwen_image_edit_2509_fp8_e4m3fn_scaled.safetensors
Loading weights from:
E:\project\Qwen-Image-Lightning\Qwen-Image-Edit-2509\qwen_image_edit_2509_fp8_e4m3fn_scaled.safetensors
Problem
The model successfully initializes CUDA, loads the FP8 checkpoint, text encoder, and VAE.
However, inference fails at the first DiT step.
The relevant traceback is:
File "lightx2v/models/networks/qwen_image/infer/pre_infer.py", line 72, in infer
hidden_states = weights.img_in.apply(hidden_states.squeeze(0))
File "lightx2v/common/ops/mm/mm_weight.py", line 344, in apply
return torch.addmm(
self._get_actual_bias(),
input_tensor,
self._get_actual_weight(),
out=output_tensor
)
RuntimeError: self and mat2 must have the same dtype,
but got BFloat16 and Float8_e4m3fn
Environment
OS: Windows
GPU: NVIDIA RTX A6000
Compute Capability: 8.6 (SM86)
PyTorch: 2.5.1
CUDA: 12.1
Triton: 3.7.1
Model: Qwen-Image-Edit-2509
Quantization: FP8 E4M3FN Scaled
Task: I2I
Resolution: 832x480
use_bfloat16: True
dit_quantized: True
dit_quant_scheme: fp8-sgl
infer_steps: 8
Diffusers version: 0.36.0.dev0
Checkpoint
I am using the following checkpoint:
qwen_image_edit_2509_fp8_e4m3fn_scaled.safetensors
The checkpoint is successfully found and loaded:
The model successfully initializes CUDA, loads the FP8 checkpoint, text encoder, and VAE.
However, inference fails at the first DiT step.
The relevant traceback is: