Good First Issue List

[pan-x-c]

Good First Issue List

Trinity-RFT is a flexible and modular Reinforcement Fine-Tuning framework. We welcome contributions from the community to help improve and expand the framework.

As a starting point, here are some good first issues that new contributors can work on:

1. Implement a New Workflow

   • Description: Create a new agentic workflow to tackle specific tasks.
   • Difficulty: Easy or Medium
   • Labels: good first issue, workflow
   • Resources: Check out the Workflow Development Guide for step-by-step instructions.
   • Examples:
     • Gomoku, Sudoku, or other interesting game playing workflows
     • Workflows to solve benchmarks like Multi-hop QA, MuSiQue, etc.
     • Workflows to adapt to popular agent frameworks like LangChain, AutoGen, etc.
     • ...

2. Implement a New RL Algorithm

   • Description: Implement a new reinforcement learning algorithm to improve training efficiency or performance.
   • Difficulty: Medium
   • Labels: good first issue, algorithm
   • Resources: Refer to the Algorithm Development Guide for implementation tutorials.
   • Examples:
     • Implement newly proposed RL algorithms, e.g., M2PO, BAPO.
     • Optimize existing algorithms by improving their efficiency or stability.
     • ...

3. Implement a New Experience Operator

   • Description: Develop a new operator for experience data filtering, augmentation, or reward shaping.
   • Difficulty: Easy or Medium
   • Labels: good first issue, operator
   • Resources: See the Operator Development Guide for guidance.
   • Examples:
     • Implement an operator to filter out low-quality experiences based on predefined criteria.
     • Implement an operator to refine rewards by comparing experiences generated from different runs of the same task.
     • ...

4. Improve Examples and Documentation

   • Description: Enhance the existing examples and documentation to help new users get started with Trinity-RFT.
   • Difficulty: Easy
   • Labels: good first issue, documentation
   • Resources: Check the existing Examples and Documentation for areas of improvement.
   • Examples:
     • Add examples for workflows or algorithms implemented but not yet documented in the examples directory.
     • Improve existing documentation for clarity and completeness.
     • ...

Besides these tasks for beginners, we also have more challenging issues for experienced contributors, such as:

• Reduce the bubble caused by decoupled Explorer / Trainer to improve resource utilization.
• Improve the efficiency of the experience buffer.
• Add partial rollout support to the Explorer to avoid resource waste caused by the long-tail effect of rollouts in agentic RL scenarios.
• Add popular inference backends like SGLang.
• ...

If you're interested in working on any of these issues, please feel free to comment on the issue or open a pull request. We look forward to your contributions!

[AdnanQureshi3]

Hey, I am intrested working on operator issue.

[pan-x-c]

Hey, I am intrested working on operator issue.

@AdnanQureshi3 Thank you for your attention. You can start a new Issue or PR describing the specific operator issue you want to address. If you encounter any problems during development, you can also provide feedback in the corresponding Issue or PR, and we will provide assistance.