HyperSweeper

Hydra sweeper integration of our favorite optimization packages, utilizing ask-and-tell interfaces.

• Free software: BSD license
• Documentation: https://automl.github.io/hypersweeper

Installation

We recommend installing hypersweeper via a uv virtual environment:

pip install uv
uv venv --python 3.10
source .venv/bin/activate
make install

For extra dependencies, add them like this:

uv sync --extra dev --extra carps

Note that CARP-S requires you to install benchmarks and optimizers on your own, e.g. by running:

python -m carps.build.make optimizer_smac

The full optimizer options are:

optimizer_smac optimizer_dehb optimizer_nevergrad optimizer_optuna optimizer_ax optimizer_skopt optimizer_synetune

Basic Usage

To use the sweeper, you need to specify a target function with a hydra interface (see our examples). Then you can add one of the Hypersweeper variations as a sweeper and run with the '-m' flag to start the optimization. This will start a sequential run of your selected optimizer. If you want to use Hypersweeper on a cluster, you should additionally add a launcher, e.g. the submitit launcher for slurm.

As an example, take black-box optimization for Branin using SMAC. Simply run:

python examples/branin.py -m

You should see the launched configurations in the terminal. The results are located in 'tmp', including a record of each run, the final config and a full runhistory. For more information, see our example ReadMe.

Current Sweeper Integrations

Hypersweeper currently interfaces the following optimizers:

• SMAC
• HEBO
• Nevergrad
• CARP-S (which contains many different optimizers in itself)

We also implement sweepers from scratch for:

• Random Search
• Grid Search (for collecting analysis data)
• PBT, PB2 and BGPBT as adaptations of the original code
• Local parameter importance (LPI)
• Ablation Paths

We also have an interface for NePS, but due to them using an old configspace version, this sweeper is not usable out of the box and requires at least a change to the configspace version, possibly even some smaller changes to the config parsing.

Cite Us

If you use Hypersweeper in your project, please cite us:

@misc{eimer24,
  author    = {T. Eimer},
  title     = {Hypersweeper},
  year      = {2024},
  url = {https://github.com/automl/hypersweeper},