feat(eval): filter reported scores and metrics - #130
Merged
Abhijeet Prasad (AbhiPrasad) merged 3 commits intoAug 20, 2026
Conversation
Add independent `report_scores` and `report_metrics` action inputs so PR comments can focus on selected evaluation results. Render scores and metrics in separate tables while preserving the existing report-all behavior when filters are omitted. Resolves https://linear.app/braintrustdata/issue/SDK-196/add-a-method-to-specify-which-metrics-are-reported-in-the-pr-comment
Abhijeet Prasad (AbhiPrasad)
requested review from
Stephen Belanger (Qard) and
Andrew Kent (realark)
August 19, 2026 23:17
Trigger eval workflows from the pull request event so the action has a PR to comment on. Limit push-triggered evals to `main` to avoid duplicate runs on feature branches.
Braintrust eval reportSay Hi Bot Python (abhijeet@braintrustdata.com-1787181833)
|
Braintrust eval reportSay Hi Bot Python (abhijeet@braintrustdata.com-1787181835)
|
Braintrust eval report
|
Braintrust eval reportConsole logging (HEAD-1787181836)
My Evaluation (HEAD-1787181836)
|
Braintrust eval report
|
Keep scores and metrics visually distinct with labeled sections while sharing a single table header for a more compact PR comment.
Stephen Belanger (Qard)
approved these changes
Aug 20, 2026
Braintrust eval reportSay Hi Bot Python (main-1787230576)
|
Braintrust eval reportSay Hi Bot Python (main-1787230581)
|
Braintrust eval report
|
Braintrust eval reportConsole logging (main-1787230587)
My Evaluation (main-1787230587)
|
Braintrust eval report
|
Abhijeet Prasad (AbhiPrasad)
deleted the
abhi-feat-sdk-196-filter-pr-metrics
branch
September 1, 2026 21:28
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
report_scoresandreport_metricsaction inputs to constraint PR comment output for eval action.Example:
Resolves https://linear.app/braintrustdata/issue/SDK-196/add-a-method-to-specify-which-metrics-are-reported-in-the-pr-comment