End-to-End Implementation Bounty for generic image-to-image pipeline [2300]

[rickstaa]

Overview

The rapid evolution of AI-driven models has introduced remarkable capabilities that can significantly enhance creative workflows. Among these, tasks such as image inpainting, outpainting, and sketch-to-image generation stand out as transformative tools for creators. In the past, we launched individual pipelines for each of these tasks through community bounties. However, after thorough evaluation, we have identified that a unified approach—a generic image-to-image-generic pipeline—would provide a more efficient solution. This consolidation will reduce code complexity, streamline feature integration, and enhance scalability within the Livepeer AI Network. 🔥

Problem

We are now seeking the community’s support to implement this critical enhancement. By developing a unified image-to-image-generic pipeline, we aim to simplify workflows for image transformation tasks and improve the developer experience. This upgrade will make it easier for developers to incorporate advanced image-processing capabilities into their applications while maintaining flexibility and efficiency.

Desired Solution

This initiative aligns with our vision of establishing the Livepeer AI Network as the infrastructure backbone for creative platforms, empowering creators and developers alike. We are excited to collaborate with the community to bring this vision to life and continue driving innovation within the Livepeer ecosystem. 🚀

Bounty Requirements

1. Implementation: Create a functional /image-to-image-generic route and pipeline within the AI-worker repository. This new pipeline should be accessible through Docker on port ____. Also, develop the necessary code within the go-livepeer repository to integrate access to the image-to-image-generic pipeline from the ai-worker component. This includes implementing the payment logic and job routing to ensure seamless operation within the network.

2. Functionality: The pipeline must accept an original image and other related parameters for the tasks such as prompt and image masks and output the modified image according to the user request. Ensure that users can submit AI job requests to the network in a manner consistent with other AI-Network features.

Example request:

`curl -X POST "https://your-gateway-url/image-to-image-generic -F pipeline="image-to-image-generic” -F model_id=""     -F prompt="" -F mask_image=""`

Applicant Requirements

• Proficiency in FastAPI.
• Strong experience in Python.
• Familiarity with Docker and Livepeer AI Network.
• Basic understanding of Go programming and Hugging Face models is advantageous.

Scope Exclusions

None

Implementation Tips

Getting started with unifying the tasks, refer to the PR #231 and PR #250 to see how the base structure for the individual pipelines are setup. You can also explore the following pull requests to see how other image handling pipelines were implemented:

• PR #223

In some cases, you might encounter dependencies conflicts and not be able to integrate the new pipeline directly into the regular AI Runner. If this occurs, you can follow the approach outlined in SAM2 PR to create a custom container for the pipeline. This approach uses the regular AI runner as the base while keeping the base container lean.

To streamline development, keep these best practices in mind:

• Use Developer Documentation: Leverage developer documentation for the worker and runner that provides tips for mocking pipelines and direct debugging, which can streamline the development process. Similarly, developer documentation for the installation of go-livepeer and the general Livepeer documentation for example usage and setup instructions offer valuable insights that can expedite your development process, including automatic scripts for orchestrators and gateways.
• Update OpenAPI Specification: Execute the runner/gen_openapi.py script to generate an updated OpenAPI specification.
• Generate Go-Livepeer Bindings: In the main repository directory, execute the make command to generate the necessary bindings, ensuring compatibility with the go-livepeer repository.
• Build Binaries: Run the make command in the main repository folder to generate Livepeer binaries. This will allow you to test your implementation and ensure it integrates smoothly.
• Create Docker Images: Build Docker images of Livepeer and test them using appropriate tools and settings to identify any edge cases or bugs. This step is crucial for ensuring robustness and reliability in your implementation.

Additional Resources

None

How to Apply

1. Express Your Interest: Fill out this form, making sure to specify the bounty you are interested in
2. Wait for Review: Our team will review expressions of interest and select the best candidate.
3. Get Assigned: If selected, we'll contact you and assign the bounty to you.
4. Start Working: Dive into your task! If you need assistance or guidance, join the discussions in the #developer-lounge channel on our Discord server.
5. Submit Your Work: Create a pull request in the relevant repository and request a review.
6. Notify Us: Ping us on Discord when you’re pull request is ready for review.
7. Receive Your Bounty: We'll arrange the bounty payment once your pull request is approved.
8. Gain Recognition: Your valuable contributions will be showcased in our project's changelog.

Contact Information

For questions or clarifications, please contact: [email protected]

FAQs

How is payment made?

Bounties are paid out in LPT on the Arbitrum network. On acceptance of your application, we will ask for an Arbitrum-compatible wallet address from you for which we can send payment.

[rickstaa]

Duplicate of #72.