Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Benchmarking & Leaderboard #31

Open
TheEimer opened this issue Jun 5, 2024 · 0 comments
Open

Benchmarking & Leaderboard #31

TheEimer opened this issue Jun 5, 2024 · 0 comments
Labels
enhancement New feature or request
Milestone
ARLBench 0.2.0

Comments

@TheEimer
Copy link
Collaborator

TheEimer commented Jun 5, 2024

Ideally we'll be able to provide some sort of running leaderboard for ARLBench - but at least we should get all optimizers we should sensibly compare and benchmark them. These include in my opinion:

  • SMAC (BOHB + BBO)
  • PBT/2, potentially even BG-PBT
  • DEHB
  • RS (multiple sizes?)
  • Standard HB
@TheEimer TheEimer added the enhancement New feature or request label Jun 5, 2024
@TheEimer TheEimer added this to the ARLBench 0.2.0 milestone Jun 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
ARLBench: Road to 1.0.0
Status: Ready
Milestone
ARLBench 0.2.0
Development

No branches or pull requests
1 participant
@TheEimer