Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feat] Continue SMAC runs #18

Open
becktepe opened this issue Sep 5, 2024 · 1 comment
Open

[Feat] Continue SMAC runs #18

becktepe opened this issue Sep 5, 2024 · 1 comment
Assignees

Comments

@becktepe
Copy link
Contributor

becktepe commented Sep 5, 2024

As of now, the hypersweeper does not allow for continuing SMAC runs because it overwrites previous runs by initializing the job index with zero for every optimization run instead of using the current config index from SMAC when submitting a new batch.

Example:
Run SMAC with the HS for 100 trials. After 10 trials, something crashes.
Runhistory: configs [0-9]
Sweep directories: [0-9]

Now, if we want to continue the run, i.e., start a new HS optimization:
Runhistory: configs [0-9] from previous run + configs [10-19]
Sweep directories: [0-9] again -> this is the problem, we use the same sweep directories again for different configs

One solution could be so use the configuration index from SMAC if it is contained in the object returned by smac.ask(). Otherwise the length of the run history could be used to determine the next

@TheEimer
Copy link
Contributor

Sounds good, but I don't think the TrialInfo we get from smac.ask() actually contains the config ID. @benjamc any idea how to best get that info? Is it in the configuration?

@TheEimer TheEimer added this to the RL Maintenance 01/25 milestone Nov 27, 2024
@TheEimer TheEimer self-assigned this Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants