Skip to content

Commit

Permalink
update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
danrgll committed Jun 17, 2024
1 parent 659614f commit f23c882
Show file tree
Hide file tree
Showing 14 changed files with 45 additions and 53 deletions.
6 changes: 4 additions & 2 deletions docs/doc_yamls/customizing_neps_optimizer.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# Customizing NePS Searcher
run_pipeline:
path: path/to/your/run_pipeline.py # Path to the function file
name: example_pipeline # Function name within the file
Expand All @@ -12,9 +13,10 @@ pipeline_space:
epochs: 50

root_directory: path/to/results # Directory for result storage
max_evaluations_total: 20 # Budget
max_evaluations_total: 20 # Budget
searcher:
strategy: bayesian_optimization # name linked with neps keywords, more information click here..?
strategy: bayesian_optimization # key for neps searcher
name: "my_bayesian" # optional; changing the searcher_name for better recognition
# Specific arguments depending on the searcher
initial_design_size: 7
surrogate_model: gp
Expand Down
6 changes: 3 additions & 3 deletions docs/doc_yamls/defining_hooks.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Basic NEPS Configuration Example
# Hooks
run_pipeline:
path: path/to/your/run_pipeline.py # Path to the function file
name: example_pipeline # Function name within the file
Expand All @@ -17,8 +17,8 @@ pipeline_space:
batch_size: 64

root_directory: path/to/results # Directory for result storage
max_evaluations_total: 20 # Budget
max_evaluations_total: 20 # Budget

pre_load_hooks:
hook1: path/to/your/hooks.py # (function_name: Path to the function's file)
hook2: path/to/your/hooks.py # Different function name from the same file source
hook2: path/to/your/hooks.py # Different function name 'hook2' from the same file source
2 changes: 1 addition & 1 deletion docs/doc_yamls/full_configuration_template.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ pipeline_space:
learning_rate:
lower: 1e-5
upper: 1e-1
log: True # Log scale for learning rate
log: True
epochs:
lower: 5
upper: 20
Expand Down
7 changes: 4 additions & 3 deletions docs/doc_yamls/loading_own_optimizer.yaml
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
# Loading Optimizer Class
run_pipeline:
path: path/to/your/run_pipeline.py # Path to the function file
name: example_pipeline # Function name within the file
Expand All @@ -12,10 +13,10 @@ pipeline_space:
epochs: 50

root_directory: path/to/results # Directory for result storage
max_evaluations_total: 20 # Budget
max_evaluations_total: 20 # Budget
searcher:
path: path/to/your/searcher.py # Path to the class
name: CustomOptimizer # class name within the file
path: path/to/your/searcher.py # Path to the class
name: CustomOptimizer # class name within the file
# Specific arguments depending on your searcher
initial_design_size: 7
surrogate_model: gp
Expand Down
6 changes: 3 additions & 3 deletions docs/doc_yamls/loading_pipeline_space_dict.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# Loading pipeline space from a python dict
run_pipeline:
path: path/to/your/run_pipeline.py # Path to the function file
name: example_pipeline # Function name within the file
name: example_pipeline # Function name within the file

pipeline_space:
path: path/to/your/search_space.py # Path to the dict file
name: pipeline_space # Name of the dict instance
name: pipeline_space # Name of the dict instance

root_directory: path/to/results # Directory for result storage
max_evaluations_total: 20 # Budget
max_evaluations_total: 20 # Budget
2 changes: 1 addition & 1 deletion docs/doc_yamls/outsourcing_optimizer.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,6 @@ pipeline_space:
epochs: 50

root_directory: path/to/results # Directory for result storage
max_evaluations_total: 20 # Budget
max_evaluations_total: 20 # Budget

searcher: path/to/your/searcher_setup.yaml
4 changes: 2 additions & 2 deletions docs/doc_yamls/outsourcing_pipeline_space.yaml
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Pipeline space settings from YAML
# Pipeline space settings from separate YAML
run_pipeline:
path: path/to/your/run_pipeline.py # Path to the function file
name: example_pipeline # Function name within the file

pipeline_space: path/to/your/pipeline_space.yaml

root_directory: path/to/results # Directory for result storage
max_evaluations_total: 20 # Budget
max_evaluations_total: 20 # Budget

2 changes: 1 addition & 1 deletion docs/doc_yamls/pipeline_space.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# pipeline_space including priors and fidelity
# Pipeline_space including priors and fidelity
learning_rate:
lower: 1e-5
upper: 1e-1
Expand Down
2 changes: 1 addition & 1 deletion docs/doc_yamls/simple_example_including_run_pipeline.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Simple NePS configuration including run_pipeline
run_pipeline:
path: path/to/your/run_pipeline.py # Path to the function file
name: example_pipeline # Function name within the file
name: example_pipeline # Function name within the file

pipeline_space:
learning_rate:
Expand Down
15 changes: 9 additions & 6 deletions docs/reference/declarative_usage.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
!!! note "Work in Progress"
This document is currently a work in progress and may contain incomplete or preliminary information.

## Introduction
### Configuring with YAML
Configure your experiments using a YAML file, which serves as a central reference for setting up your project.
This approach simplifies sharing, reproducing, and modifying configurations.
This approach simplifies sharing, reproducing and modifying configurations.
#### Simple YAML Example
Below is a straightforward YAML configuration example for NePS covering the required arguments.
=== "config.yaml"
Expand All @@ -27,7 +25,7 @@ Below is a straightforward YAML configuration example for NePS covering the requ
```


#### Including `run_pipeline` in config.yaml for External Referencing
#### Including `run_pipeline` in `run_args` for External Referencing
In addition to setting experimental parameters via YAML, this configuration example also specifies the pipeline function
and its location, enabling more flexible project structures.
=== "config.yaml"
Expand Down Expand Up @@ -65,7 +63,7 @@ but also advanced settings for more complex setups.

The `searcher` key used in the YAML configuration corresponds to the same keys used for selecting an optimizer directly
through `neps.run`. For a detailed list of integrated optimizers, see [here](optimizers.md#list-available-searchers)
!!! note "Note on Undefined Keys"
!!! note "Note on undefined keys in `run_args` (config.yaml)"
Not all configurations are explicitly defined in this template. Any undefined key in the YAML file is mapped to
the internal default settings of NePS. This ensures that your experiments can run even if certain parameters are
omitted.
Expand Down Expand Up @@ -158,7 +156,10 @@ search spaces must be loaded via a dictionary, which is then referenced in the `


### Integrating Custom Optimizers
You can also load your own custom optimizer and change its arguments in `config.yaml`.
For people who want to write their own optimizer class as a subclass of the base optimizer, you can load your own
custom optimizer class and define its arguments in `config.yaml`.

Note: You can still overwrite arguments via searcher_kwargs of `neps.run` like for the internal searchers.
=== "config.yaml"
```yaml
--8<-- "docs/doc_yamls/loading_own_optimizer.yaml"
Expand All @@ -173,6 +174,8 @@ You can also load your own custom optimizer and change its arguments in `config.
neps.run(run_args="path/to/your/config.yaml")
```



### Adding Custom Hooks to Your Configuration
Define hooks in your YAML configuration to extend the functionality of your experiment.
=== "config.yaml"
Expand Down
36 changes: 12 additions & 24 deletions docs/reference/neps_run.md
Original file line number Diff line number Diff line change
Expand Up @@ -190,11 +190,7 @@ Any new workers that come online will automatically pick up work and work togeth

## YAML Configuration
You have the option to configure all arguments using a YAML file through [`neps.run(run_args=...)`][neps.api.run].
For more on yaml usage, please visit the dedicated [page on usage of YAML with NePS](../reference/yaml_usage.md).

!!! example "In Progress"

This feature is currently in development and is subject to change.
For more on yaml usage, please visit the dedicated [page on usage of YAML with NePS](../reference/declarative_usage.md).

Parameters not explicitly defined within this file will receive their default values.

Expand All @@ -203,18 +199,17 @@ Parameters not explicitly defined within this file will receive their default va

```yaml
# path/to/your/config.yaml
run_args:
run_pipeline:
path: "path/to/your/run_pipeline.py" # File path of the run_pipeline function
name: "name_of_your_run_pipeline" # Function name
pipeline_space: "path/to/your/search_space.yaml" # Path of the search space yaml file
root_directory: "neps_results" # Output directory for results
max_evaluations_total: 100
post_run_summary: # Defaults applied if left empty
searcher: "bayesian_optimization"
searcher_kwargs:
initial_design_size: 5
surrogate_model: "gp"
run_pipeline:
path: "path/to/your/run_pipeline.py" # File path of the run_pipeline function
name: "name_of_your_run_pipeline" # Function name
pipeline_space: "path/to/your/search_space.yaml" # Path of the search space yaml file
root_directory: "neps_results" # Output directory for results
max_evaluations_total: 100
post_run_summary: # Defaults applied if left empty
searcher:
strategy: "bayesian_optimization"
initial_design_size: 5
surrogate_model: "gp"
```

=== "Python"
Expand All @@ -223,13 +218,6 @@ Parameters not explicitly defined within this file will receive their default va
neps.run(run_args="path/to/your/config.yaml")
```

!!! warning

Currently we have a strict usage for `run_args`.
So you can define either all arguments by providing them directly to neps.run or via the yaml file.
This might change in the future.
If you use yaml, directly provided arguments get overwritten either by the defined yaml config or the default value.

## Handling Errors
Things go wrong during optimization runs and it's important to consider what to do in these cases.
By default, NePS will halt the optimization process when an error but you can choose to `ignore_errors=`, providing a `loss_value_on_error=` and `cost_value_on_error=` to control what values should be reported to the optimization process.
Expand Down
4 changes: 2 additions & 2 deletions docs/reference/optimizers.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,8 @@ The library will then load your custom settings and use them for optimization.
Here's the format of a custom YAML (`custom_bo.yaml`) configuration using `Bayesian Optimization` as an example:

```yaml
algorithm: bayesian_optimization
name: my_custom_bo # # optional; otherwise, your searcher will be named after your YAML file, here 'custom_bo'.
strategy: bayesian_optimization
name: my_custom_bo # optional; otherwise, your searcher will be named after your YAML file, here 'custom_bo'.
# Specific arguments depending on the searcher
initial_design_size: 7
surrogate_model: gp
Expand Down
3 changes: 0 additions & 3 deletions docs/reference/pipeline_space.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,9 +128,6 @@ the NePS will automatically infer the data type based on the value provided.

If none of these hold, an error will be raised.

Details on how YAML interpret boolean values can be found later on,
[here](#important-note-on-yaml-data-type-interpretation)


## Using ConfigSpace

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,8 @@ pipeline_space:
root_directory: path/to/results/custominizing_neps_optimizer # Directory for result storage
max_evaluations_total: 20 # Budget
searcher:
strategy: bayesian_optimization # name linked with neps keywords, more information click here..?
strategy: bayesian_optimization
name: "my_bayesian"
# Specific arguments depending on the searcher
initial_design_size: 7
surrogate_model: gp
Expand Down

0 comments on commit f23c882

Please sign in to comment.