Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

t5xxl bf16 produces invalid code when compiled #19932

Open
KyleHerndon opened this issue Feb 6, 2025 · 0 comments
Open

t5xxl bf16 produces invalid code when compiled #19932

KyleHerndon opened this issue Feb 6, 2025 · 0 comments
Labels
bug 🐞 Something isn't working

Comments

@KyleHerndon
Copy link
Contributor

KyleHerndon commented Feb 6, 2025

What happened?

I'm still working on finding a minimal repro, but I'm running into issues working with a full Flux pipeline and isolated the issue to t5xxl. When I replace t5xxl_bf16 with a fp32 variant, everything works fine, but with bf16 the images don't follow the prompt.
t5 bf16 mlir
t5 fp32 mlir

Steps to reproduce your issue

~/kherndon/iree-build/tools/iree-compile --iree-hal-target-backends=rocm --iree-hip-target=gfx942 t5xxl_bf16.mlir -o t5xxl.vmfb

What component(s) does this issue relate to?

Compiler

Version information

278e63a

Additional context

I was also able to run this model correctly with bf16 eager through sharktank, but after compilation it fails.

@KyleHerndon KyleHerndon added the bug 🐞 Something isn't working label Feb 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug 🐞 Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant