You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Some time ago, while browsing the state of the art, I stumbled upon this idea and I can't for the life of me remember which algo introduced it, in which paper it was published or which packages implement it. I could have sworn it was BOHB and SMAC3, but it turns out I was wrong.
The main idea was to treat the budget parameter similarly to how SMAC3 treats instance features, i.e., to train the surrogate model on it, as well as on instance features and hyperparameters. When maximizing the acquisition function, we'd only care about the predictions at max_budget.
As such, a slice along the budget dimension in the cost surface modeled by the RF would effectively represent an estimate for configurations' learning curve. This way, the underlying surrogate model would also provide learning curve prediction (extrapolation), and costs measured at lower budgets would improve estimations for which configs will maximize the acquisition function at max_budget.
In this case, it would also help to provide more datapoints to constrain the surrogate model, so it would make sense to report cost(s) after every unit increment of the budget (i.e., after every epoch), rather then just at the budgets at which the multi-fidelity intensifier judges whether to keep running or cut short.
The text was updated successfully, but these errors were encountered:
Some time ago, while browsing the state of the art, I stumbled upon this idea and I can't for the life of me remember which algo introduced it, in which paper it was published or which packages implement it. I could have sworn it was BOHB and SMAC3, but it turns out I was wrong.
The main idea was to treat the budget parameter similarly to how SMAC3 treats instance features, i.e., to train the surrogate model on it, as well as on instance features and hyperparameters. When maximizing the acquisition function, we'd only care about the predictions at
max_budget
.As such, a slice along the budget dimension in the cost surface modeled by the RF would effectively represent an estimate for configurations' learning curve. This way, the underlying surrogate model would also provide learning curve prediction (extrapolation), and costs measured at lower budgets would improve estimations for which configs will maximize the acquisition function at
max_budget
.In this case, it would also help to provide more datapoints to constrain the surrogate model, so it would make sense to report cost(s) after every unit increment of the budget (i.e., after every epoch), rather then just at the budgets at which the multi-fidelity intensifier judges whether to keep running or cut short.
The text was updated successfully, but these errors were encountered: