Enhance Buffer Data Processing Pipeline #175
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
There was a problem hiding this comment.
Summary of Changes
Hello @pan-x-c, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
I've implemented a significant enhancement to the experience processing flow by introducing a new pipeline architecture. This change allows for flexible and modular manipulation of experience data through a series of operators. The previous AddStrategy concept has been integrated into this new operator framework, streamlining the algorithm module. This update provides a more robust and extensible system for handling experience data, making it easier to implement and test complex data transformations.
Highlights
- New Experience Pipeline: I've introduced a new
ExperiencePipelineandExperienceOperatorinterface. This allows for a more modular and extensible way to process experience data, enabling users to define custom operations that run in a pipeline. - Refactoring and Merging
AddStrategy: The functionality previously handled byAddStrategyhas been merged into theOperatorconcept, specifically enhancingAdvantageFnto handle group-based advantage calculations. This refactoring simplifies the algorithm module and reduces functional overlap. - Updated Developer Documentation: The documentation has been thoroughly updated to reflect these significant changes, including new guidelines for developers on how to write and integrate their own custom
Operators. - Enhanced Test Coverage: Comprehensive unit tests have been added for both the new
ExperiencePipelineand the refactoredAdvantageFncomponents, ensuring the stability and correctness of these core changes.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
Contributor
There was a problem hiding this comment.
Code Review
This pull request introduces a significant enhancement to the data processing capabilities by implementing an ExperiencePipeline. This new pipeline, composed of ExperienceOperators, provides a flexible and extensible way to process experience data. As part of this refactoring, the AddStrategy has been merged into the new Operator concept, simplifying the architecture. The changes are well-reflected in the updated documentation and tests. My review focuses on a couple of areas for potential improvement, including addressing a TODO for metrics and another for performance optimization.
Collaborator
Author
|
/unittest-all |
Summary
Tests
Github Test Reporter by CTRF 💚 |
Collaborator
Author
|
/unittest-all |
Summary
Failed Tests
Tests
Github Test Reporter by CTRF 💚 |
Collaborator
Author
|
/unittest-module-trainer |
Collaborator
Author
|
/unittest-all |
Summary
Tests
Github Test Reporter by CTRF 💚 |
Collaborator
Author
|
/unittest-trainer |
Summary
Tests
Github Test Reporter by CTRF 💚 |
Collaborator
Author
|
/unittest-module-trainer |
Summary
Tests
Github Test Reporter by CTRF 💚 |
Collaborator
Author
|
/unittest-all |
Summary
Tests
Github Test Reporter by CTRF 💚 |
Collaborator
Author
|
/unittest-all |
Summary
Tests
Github Test Reporter by CTRF 💚 |
Collaborator
Author
|
/unittest-all |
Summary
Failed Tests
Tests
Github Test Reporter by CTRF 💚 |
Collaborator
Author
|
/unittest-module-common |
Summary
Tests
Github Test Reporter by CTRF 💚 |
yaochaorui
pushed a commit
to yaochaorui/Trinity-RFT
that referenced
this pull request
Aug 27, 2025
Co-authored-by: Yilun Huang <lielin.hyl@alibaba-inc.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Implementation of #158
ExperienceOperatorinterface, allowing users to implement this interface to process experience data in a pipeline manner.ExperiencePipelineactor, responsible for continuously running operators.AddStrategyinto Operator to avoid functional overlap.TaskPipeline, responsible for pre-processing the raw task datasets before exploring. (Only support using Data-Juicer operators for now)Checklist
Please check the following items before code is ready to be reviewed.