Skip to content

Conversation

@Tanuj-Taneja1
Copy link
Collaborator

Description

Fixes #3278

Checklist

Go over all the following points, and put an x in all the boxes that apply.

  • I have read the CONTRIBUTION guide (required)
  • I have linked this PR to an issue using the Development section on the right sidebar or by adding Fixes #issue-number in the PR description (required)
  • I have checked if any dependencies need to be added or updated in pyproject.toml and uv lock
  • I have updated the tests accordingly (required for a bug fix or a new feature)
  • I have updated the documentation if needed:
  • I have added examples if this is a new feature

If you are unsure about any of these, don't hesitate to ask. We are here to help!

@github-actions github-actions bot added the Review Required PR need to be reviewed label Oct 29, 2025
@coderabbitai
Copy link
Contributor

coderabbitai bot commented Oct 29, 2025

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch docs/benchmark

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@fengju0213
Copy link
Collaborator

@Tanuj-Taneja1 thanks for this pr!

Copy link
Collaborator

@fengju0213 fengju0213 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Tanuj-Taneja1 thanks for your contribution!

@Tanuj-Taneja1
Copy link
Collaborator Author

Hi @fengju0213
Just a question do we need to run all the test for a docs pr. I think although github actions is free for open source projects we can still reduce the api usage.
Not sure how much a single set of test cost but if it is a considerable amount on scale we can maybe skip more tests that are not related to docs

@Wendong-Fan
Copy link
Member

Hi @fengju0213 Just a question do we need to run all the test for a docs pr. I think although github actions is free for open source projects we can still reduce the api usage. Not sure how much a single set of test cost but if it is a considerable amount on scale we can maybe skip more tests that are not related to docs

thanks @Tanuj-Taneja1 , it's really a great suggestion, please feel free to open a new issue to isolate tests for changes unrelated to functional code (e.g., doc updates)

Copy link
Member

@Wendong-Fan Wendong-Fan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we also need to add this new doc to index.rst file, also let's use benchmark.md instead of Benchmark.md to align with other docs' naming
cc @fengju0213

3. `run()`: Execute benchmark and populate `self._results`
4. Optional: Override `train`, `valid`, `test` properties

## References
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems we missing the reference for BrowseComp Benchmark cc @fengju0213

@Wendong-Fan Wendong-Fan merged commit c8d5374 into master Nov 2, 2025
11 of 12 checks passed
@Wendong-Fan Wendong-Fan deleted the docs/benchmark branch November 2, 2025 21:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Review Required PR need to be reviewed

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

[Feature Request] DOCS regarding benchmark

4 participants