Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Benchmarking #96

Merged
merged 34 commits into from
Jul 3, 2024
Merged

Benchmarking #96

merged 34 commits into from
Jul 3, 2024

Commits on Apr 15, 2024

  1. Add first version of benchmarking script

    @Bronzila
    Bronzila committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    4cc7d38 View commit details
    Browse the repository at this point in the history

Commits on Apr 17, 2024

  1. [WIP] Add initial HPOBench benchmark

    @Bronzila
    Bronzila committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    9ce8d23 View commit details
    Browse the repository at this point in the history

  2. Add OD and surrogate benchmark

    @Bronzila
    Bronzila committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    a876520 View commit details
    Browse the repository at this point in the history

  3. [WIP] Add proper min/max fidelity loading

    @Bronzila
    Bronzila committed Apr 17, 2024
    Configuration menu
    Copy the full SHA
    9469123 View commit details
    Browse the repository at this point in the history

Commits on Apr 18, 2024

  1. Adjust args

    @Bronzila
    Bronzila committed Apr 18, 2024
    Configuration menu
    Copy the full SHA
    7f7778f View commit details
    Browse the repository at this point in the history

Commits on Apr 23, 2024

  1. Add benchmarking setup script

    @Bronzila
    Bronzila committed Apr 23, 2024
    Configuration menu
    Copy the full SHA
    0100e3a View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2024

  1. Sort out dependencies and clean up benchmarking script

    @Bronzila
    Bronzila committed Apr 29, 2024
    Configuration menu
    Copy the full SHA
    5253f3e View commit details
    Browse the repository at this point in the history

Commits on May 7, 2024

  1. Adjust folder structure and implement mfpbench benchmark

    @Bronzila
    Bronzila committed May 7, 2024
    Configuration menu
    Copy the full SHA
    23c3274 View commit details
    Browse the repository at this point in the history

Commits on May 8, 2024

  1. Add setup guide

    @Bronzila
    Bronzila committed May 8, 2024
    Configuration menu
    Copy the full SHA
    3f3c67b View commit details
    Browse the repository at this point in the history

Commits on May 10, 2024

  1. Add virtual env creation section to benchmarking script

    @Bronzila
    Bronzila committed May 10, 2024
    Configuration menu
    Copy the full SHA
    4097ab4 View commit details
    Browse the repository at this point in the history

Commits on May 13, 2024

  1. Adjust benchmarking setup

    @Bronzila
    Bronzila committed May 13, 2024
    Configuration menu
    Copy the full SHA
    f0ef67f View commit details
    Browse the repository at this point in the history

Commits on May 15, 2024

  1. Fix version for MFPbench and commit for HPOBench

    @Bronzila
    Bronzila committed May 15, 2024
    Configuration menu
    Copy the full SHA
    59ffbc3 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'benchmarking' of https://github.com/automl/DEHB into be… 

    …nchmarking
    @Bronzila
    Bronzila committed May 15, 2024
    Configuration menu
    Copy the full SHA
    2b08333 View commit details
    Browse the repository at this point in the history

Commits on May 21, 2024

  1. Integrate more MFPBench benchmarks + [WIP] MFPBench benchmarking setup

    @Bronzila
    Bronzila committed May 21, 2024
    Configuration menu
    Copy the full SHA
    3b9ab0b View commit details
    Browse the repository at this point in the history

Commits on May 22, 2024

  1. Add PD1, MFH benchmarks and result table/plot generation

    @Bronzila
    Bronzila committed May 22, 2024
    Configuration menu
    Copy the full SHA
    11662aa View commit details
    Browse the repository at this point in the history

Commits on May 23, 2024

  1. bump up mfpbench version

    @Bronzila
    Bronzila committed May 23, 2024
    Configuration menu
    Copy the full SHA
    f240ef0 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. Make summary generation parametrizable

    @Bronzila
    Bronzila committed May 28, 2024
    Configuration menu
    Copy the full SHA
    ef88faa View commit details
    Browse the repository at this point in the history

Commits on May 30, 2024

  1. Adjust seeding and add CountingOnes benchmark

    @Bronzila
    Bronzila committed May 30, 2024
    Configuration menu
    Copy the full SHA
    8e510eb View commit details
    Browse the repository at this point in the history

Commits on May 31, 2024

  1. Add version sorting for result table

    @Bronzila
    Bronzila committed May 31, 2024
    Configuration menu
    Copy the full SHA
    c00946b View commit details
    Browse the repository at this point in the history

Commits on Jun 5, 2024

  1. Add benchmarking script for cluster

    @Bronzila
    Bronzila committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    6ece84f View commit details
    Browse the repository at this point in the history

  2. Adjust script to fit benchmarking doc names

    @Bronzila
    Bronzila committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    21d8633 View commit details
    Browse the repository at this point in the history

  3. Minor adjustments

    @Bronzila
    Bronzila committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    83a7896 View commit details
    Browse the repository at this point in the history

  4. Add benchmarking files to gitignore and adjust benchmarking script

    @Bronzila
    Bronzila committed Jun 5, 2024
    Configuration menu
    Copy the full SHA
    ec64e5e View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. Remove jahs from benchmarking script

    @Bronzila
    Bronzila committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    69cdc88 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. Remove JAHS from default choice for MFPBench

    @Bronzila
    Bronzila committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    e1f38f5 View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2024

  1. Add extra picky dependenciesfor hpobench setup

    @Bronzila
    Bronzila committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    cba4eee View commit details
    Browse the repository at this point in the history

  2. Refactor HPOBench benchmark to use tabular ml benchmarks

    @Bronzila
    Bronzila committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    0a7f79f View commit details
    Browse the repository at this point in the history

  3. Remove unnecessary cd command

    @Bronzila
    Bronzila committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    957410e View commit details
    Browse the repository at this point in the history

  4. Add PR template and benchmark instructions for PRs to CONTRIBUTING

    @Bronzila
    Bronzila committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    39afd7f View commit details
    Browse the repository at this point in the history

  5. Update benchmarking markdown

    @Bronzila
    Bronzila committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    6f49cd0 View commit details
    Browse the repository at this point in the history

  6. Run all benchmarks per default

    @Bronzila
    Bronzila committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    2c23bf3 View commit details
    Browse the repository at this point in the history

  7. Adjust benchmark scripts to final version

    @Bronzila
    Bronzila committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    1bee014 View commit details
    Browse the repository at this point in the history

  8. Add benchmark results for all dehb versions

    @Bronzila
    Bronzila committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    bfadd8b View commit details
    Browse the repository at this point in the history

  9. Merge branch 'benchmarking' of https://github.com/automl/DEHB into be… 

    …nchmarking
    @Bronzila
    Bronzila committed Jul 3, 2024
    Configuration menu
    Copy the full SHA
    7dc6912 View commit details
    Browse the repository at this point in the history