Road to Declarative Usage

[danrgll]

Pipeline Space defined via yaml:

• Hyperparameters are supported
• Architecture Parameters are not yet supported (See Architecture parameter support in yaml-defined pipeline_space #66 )
  • see, example on how to use grammar search spaces with YAMLs [TODO]
    • Low priority until it's needed by someone or we have a clean Architecture Parameters setup that we can understand.
    • TODO: Write up a simple example python file which we would like to be able to pass in and use.
• User Feedback: Store pipeline_space into yaml file with full description (See Generate a pipeline_space.yaml as feedback on which parameter type your configurations have been converted to #84 )

neps.run arguments defined via yaml (using 'run_args'):

• All arguments can be loaded via a yaml file (See Load neps arguments from a .yaml file using a new argument run_args #61 )

• Current Limitations:

  • Strict usage of 'run_args' (See Enhance flexibility of 'run_args' to allow mixed usage of direct and YAML-based arguments #67 )
  • Missing documentation for Declarative Usage (See Add section for Declarative Usage in Documentation #68 )
  • Enable argument modification for custom optimizer class that has been loaded via yaml (See Enable argument modification for custom optimizer class that has been loaded via yaml #69 )
  • [ ] ConfigSpace currently not supported via yaml
    • Not an intended usage
  • Using Custom optimizer via yaml currently only works if pipeline_space is also provided via yaml (See Enable the use of dict for pipeline_space when the custom optimizer is defined via yaml #87 )

• Improvements:

  • Standardize yaml definitions for custom optimizer configurations (See Standardize yaml path definitions for the neps arguments #70 )
  • Enable optional consolidation of multiple yaml files into one (See Enable optional consolidation of multiple yaml files into one #73 )

Follow-up feature:

• Develop CLI for simplified yaml file management + running neps #74

Miscellaneous:

• Debate on the choice of the name "run_args" #86