Research OneDiff inference speed optimization [30 LPT]

[rickstaa]

Overview

Inference speed is crucial for our applications, and we aim to enhance the performance of our T2I and I2I pipelines. Previous optimizations like DeepCache have improved inference times but at the expense of image quality. We are interested in exploring whether another optimization, OneDiff, can provide better results. As we are currently focused on the core AI subnet software, we are inviting community builders to conduct this research in exchange for a software bounty. If OneDiff proves effective, we will create another bounty to implement this optimization in the T2I and I2I pipelines. The successful researcher will have the first opportunity to take on this new bounty, allowing them to implement this potentially valuable feature for the community and accelerate our pipelines ⚡.

Required Skillset

• Intermediate proficiency in Python.
• Optional: Experience with the Hugginface diffusers library.

Bounty Requirements

• The submission must include a dedicated report outlining the inference speedup and output quality with and without the OneDiff optimization.
• The report should cover each of the models listed below in the Text-to-Image and Image-to-Image pipelines.
• Inference times should be averaged over 10 runs, and comparison images for each run should be provided.

The following items are NOT included in this bounty:

• This bounty does not include the Image-to-video and upscaling pipelines. These will be addressed in a subsequent bounty.
• This bounty is solely focused on conducting research for this optimization method. The implementation will be addressed in a subsequent bounty.

Required models

Text-to-Image

• SG161222/RealVisXL_V4.0_Lightning
• SG161222/RealVisXL_V4.0

Image-to-image

• timbrooks/instruct-pix2pix
• SG161222/RealVisXL_V4.0

Implementation Tips

Avoid testing OneFl directly on the livepeer/ai-worker repository, as its complexity may hinder your process. Instead, create a simple inference Python script for your tests.

How to Apply

Warning

Please wait for the issue to be assigned to you before starting work. To prevent duplication of effort, submissions for unassigned issues will not be accepted.

1. Express Your Interest: Comment on the issue to let us know you're interested.
2. Wait for Review: Our team will review expressions of interest within 14 days and select the best candidate.
3. Get Assigned: If selected, we'll assign the GitHub issue to you.
4. Start Working: Begin your work! For help or guidance, join the #🛋│developer-lounge channel on our Discord server.
5. Submit Your Work: Create a pull request in the relevant repository and request a review.
6. Notify Us: Comment on the GitHub issue when your pull request is ready for review.
7. Receive Your Bounty: Once your pull request is approved, we'll arrange the bounty payment.
8. Gain Recognition: Your valuable contributions will be showcased in our project's changelog.

Thank you for your interest in contributing to our project 💛!

[lukiod]

@rickstaa i am interested in this i request you to assign me in this .

[rickstaa]

@rickstaa i am interested in this i request you to assign me in this .

@lukiod, thank you for your interest in implementing this bounty. I reviewed your GitHub profile and your experience with Python. I'm happy to announce that you are accepted to work on this bounty and qualify for the 25 LPT reward 🎉. Welcome to the Livepeer open-source community 🚀. I have provided the necessary information in this issue, but if anything is unclear, please don't hesitate to comment below or reach out in the #🛋│developer-lounge channel on our Discord server.

[rickstaa]

I increased the bounty amount to 30 LPT since due to complications in the OneDiff software the task took a lot longer than estimated.

[lukiod]

here is my report https://github.com/lukiod/T2I-and-I2I-Report

[rickstaa]

here is my report https://github.com/lukiod/T2I-and-I2I-Report

High quality report 🚀. Thanks again for your OpenSource contribution to the Livepeer ecosystem. I will contact you in DM to ensure you get your bounty reward 🪙.