Repository navigation
Good First Issue List #470
Description
Good First Issue List
Trinity-RFT is a flexible and modular Reinforcement Fine-Tuning framework. We welcome contributions from the community to help improve and expand the framework.
As a starting point, here are some good first issues that new contributors can work on:
-
Implement a New Workflow
- Description: Create a new agentic workflow to tackle specific tasks.
- Difficulty: Easy or Medium
- Labels:
good first issue,workflow - Resources: Check out the Workflow Development Guide for step-by-step instructions.
- Examples:
- Gomoku, Sudoku, or other interesting game playing workflows
- Workflows to solve benchmarks like Multi-hop QA, MuSiQue, etc.
- Workflows to adapt to popular agent frameworks like LangChain, AutoGen, etc.
- ...
-
Implement a New RL Algorithm
- Description: Implement a new reinforcement learning algorithm to improve training efficiency or performance.
- Difficulty: Medium
- Labels:
good first issue,algorithm - Resources: Refer to the Algorithm Development Guide for implementation tutorials.
- Examples:
-
Implement a New Experience Operator
- Description: Develop a new operator for experience data filtering, augmentation, or reward shaping.
- Difficulty: Easy or Medium
- Labels:
good first issue,operator - Resources: See the Operator Development Guide for guidance.
- Examples:
- Implement an operator to filter out low-quality experiences based on predefined criteria.
- Implement an operator to refine rewards by comparing experiences generated from different runs of the same task.
- ...
-
Improve Examples and Documentation
- Description: Enhance the existing examples and documentation to help new users get started with Trinity-RFT.
- Difficulty: Easy
- Labels:
good first issue,documentation - Resources: Check the existing Examples and Documentation for areas of improvement.
- Examples:
- Add examples for workflows or algorithms implemented but not yet documented in the examples directory.
- Improve existing documentation for clarity and completeness.
- ...
Besides these tasks for beginners, we also have more challenging issues for experienced contributors, such as:
- Reduce the bubble caused by decoupled Explorer / Trainer to improve resource utilization.
- Improve the efficiency of the experience buffer.
- Add partial rollout support to the Explorer to avoid resource waste caused by the long-tail effect of rollouts in agentic RL scenarios.
- Add popular inference backends like SGLang.
- ...
If you're interested in working on any of these issues, please feel free to comment on the issue or open a pull request. We look forward to your contributions!
Activity
AdnanQureshi3 commentedon Jan 10, 2026
Hey, I am intrested working on operator issue.
pan-x-c commentedon Jan 10, 2026
@AdnanQureshi3 Thank you for your attention. You can start a new Issue or PR describing the specific operator issue you want to address. If you encounter any problems during development, you can also provide feedback in the corresponding Issue or PR, and we will provide assistance.
Add low-quality experience filter operator #470 (#473)
artaasd95 commentedon May 21, 2026
Hi, I want to start development on this project!
I have forked this repo, started onboarding and after I have get familiar with the project, I will open an issue or perform PR or I will ask any information I want.
This comment is for making sure that you are accepting new contributions and it is for you information!
Thanks in advance!
pan-x-c commentedon May 22, 2026
@artaasd95 Thank you for your interest in Trinity-RFT and for considering contributing to the project.
We are still open to new contributions. However, please note that we are currently planning changes to some core modules, especially the Buffer and Trainer parts. Because of this, we will not accept new PRs related to Operators or other Buffer-related components in the short term.
For algorithm-related extensions, we recommend opening an issue first before starting implementation. This helps us discuss whether the proposal fits the current roadmap and avoid unnecessary development effort.
Thank you for your understanding and support.
artaasd95 commentedon May 22, 2026
@pan-x-c Thank you for your response!
Sure! I want to work on the RL algorithms part so I will do as you instructed.
sarthakkondawar2007 commentedon Aug 25, 2026
Hi,
I'd like to work on implementing a Gomoku agentic workflow as part of this good first issue.
Honorllq commentedon Aug 27, 2026
Hi! I’d like to confirm whether a narrowly scoped MuSiQue workflow contribution would be useful here.
I propose starting with MuSiQue-Answerable as a plugin-first, fully offline workflow over the benchmark-provided indexed paragraphs—no external search service or LLM judge. A sample would be converted into a Trinity task containing the question and local paragraphs; the model would interact for a bounded number of turns and return a final answer plus supporting paragraph indices. Reward would follow MuSiQue’s deterministic normalization: maximum answer EM/F1 over the gold answer and aliases, together with supporting-paragraph F1.
I would add unit tests for official-schema conversion, alias-aware normalization, support scoring, output parsing, and turn/tool bounds; then validate a sample with
trinity debugand a short smoke run showing at least some non-zero rewards. I found no existing MuSiQue issue, workflow, claim, or PR in this repository.Would this MuSiQue-Answerable/plugin-first scope be acceptable, and would you prefer it to remain an example/plugin initially or be registered under
trinity/common/workflowsafter the debug evidence is available?KunyangZhang commentedon Sep 13, 2026
Following the operator invitation in this issue: I opened #620 proposing a
repetition_filteroperator for degenerate rollout responses.
Short version of the gap:
reward_filter,reward_std_filter,dapo_dynamic_sampling,mask_response_truncatedandinvalid_reward_filterall decide on reward, group variance ortruncation status — none of them looks at the response itself. Repetition collapse (the policy
repeating a pattern instead of terminating) is a common failure mode of on-policy rollouts on
long-horizon agent tasks, and those trajectories currently enter the buffer like any other.
The proposal is a
duplicate_ngrams / total_ngramsratio computed over the model-generatedtokens only (
prompt_lengthexcluded, observation tokens excluded whenaction_maskispresent), dropping experiences above a configurable threshold, with
filtered_count/repeat_ratio/*metrics. Implementation plus unit tests are ready locally(
pytest tests/buffer/repetition_filter_test.py→ 11 passed) and implemented onExperienceOperatorV1as the operator guide asks.Three points I would like your call on before opening the PR, all listed in #620: whether this
belongs in the buffer layer at all, whether to drop the experience or only zero its
action_mask(likemask_response_truncateddoes), and the default thresholds.Add GSM8K examples for RLOO, REINFORCE++, GSPO and SAPO
KunyangZhang commentedon Sep 15, 2026
For item 4 (Improve Examples and Documentation): RLOO, REINFORCE++, GSPO and SAPO were implemented and listed in the support table but had no example config, so I opened #622 adding
examples/<algo>_gsm8k/for each and filling in the links in both READMEs and the docs tables (the CISPO link was also still showing-). Each config is the grpo_gsm8k one with only the experiment name andalgorithm_typechanged, and they are covered bytest_all_examples_are_valid.