Skip to content

[Bug] Qwen-Image-Edit-2509 Scaled FP8 inference fails with BF16/FP8 dtype mismatch on RTX A6000 #1439

Description

@zddpk-cyber

Environment

OS: Windows
GPU: NVIDIA RTX A6000
Compute Capability: 8.6 (SM86)
PyTorch: 2.5.1
CUDA: 12.1
Triton: 3.7.1
Model: Qwen-Image-Edit-2509
Quantization: FP8 E4M3FN Scaled
Task: I2I
Resolution: 832x480
use_bfloat16: True
dit_quantized: True
dit_quant_scheme: fp8-sgl
infer_steps: 8
Diffusers version: 0.36.0.dev0

Checkpoint

I am using the following checkpoint:

qwen_image_edit_2509_fp8_e4m3fn_scaled.safetensors

The checkpoint is successfully found and loaded:

E:\project\Qwen-Image-Lightning\Qwen-Image-Edit-2509\qwen_image_edit_2509_fp8_e4m3fn_scaled.safetensors

Loading weights from:
E:\project\Qwen-Image-Lightning\Qwen-Image-Edit-2509\qwen_image_edit_2509_fp8_e4m3fn_scaled.safetensors
Problem 

The model successfully initializes CUDA, loads the FP8 checkpoint, text encoder, and VAE.

However, inference fails at the first DiT step.

The relevant traceback is:

File "lightx2v/models/networks/qwen_image/infer/pre_infer.py", line 72, in infer
    hidden_states = weights.img_in.apply(hidden_states.squeeze(0))

File "lightx2v/common/ops/mm/mm_weight.py", line 344, in apply
    return torch.addmm(
        self._get_actual_bias(),
        input_tensor,
        self._get_actual_weight(),
        out=output_tensor
    )

RuntimeError: self and mat2 must have the same dtype,
but got BFloat16 and Float8_e4m3fn

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions