How to properly generate a multi-level composition?

[celsofranssa]

Currently, my hydra config is organized as follows:

configs/
├── config.yaml
├── data
│   ├── IMDB.yaml
│   └── REUT.yaml
└── model
    ├── BERT.yaml
    ├── GPT.yaml
    └── loss
        ├── CrossEntropyLoss.yaml
        └── TripletMarginLoss.yaml

config.yaml:

defaults:
  - model: BERT
  - data: IMDB

tasks: [ "fit", "eval" ]

The dataset (IMDB.yaml and REUT.yaml) settings are in the format:

name: IMDB
dir: resource/dataset/imdb_reviews/
folds: [0,1,2,3,4]
max_length: 256
num_classes: 10

The model (BERT.yaml and GPT.yaml) settings are in the format:

defaults:
  - loss: TripletMarginLoss

name: BERT
architecture: bert-base-uncased
lr: 5e-5
tokenizer:
  architecture: ${model.architecture}

And finally, the loss function settings (CrossEntropyLoss.yaml and TripletMarginLoss.yam) adopt the following structure:

_target_: source.loss.TripletMarginLoss.TripletMarginLoss
params:
  name: TripletMarginLoss
  margin: 1.0
  eps: 1e-6
  reduction: mean

Running the following entry point:

@hydra.main(config_path="configs/", config_name="config.yaml")
def my_app(params):
    OmegaConf.resolve(params)
    print(
        f"Params:\n"
        f"{OmegaConf.to_yaml(params)}\n")

if __name__ == '__main__':
    my_app()

# python main.py

generates the correct config composition:

tasks:
- fit
- eval
model:
  loss:
    _target_: source.loss.TripletMarginLoss.TripletMarginLoss
    params:
      name: TripletMarginLoss
      margin: 1.0
      eps: 1.0e-06
      reduction: mean
  name: BERT
  architecture: bert-base-uncased
  lr: 5.0e-05
  tokenizer:
    architecture: bert-base-uncased
data:
  name: IMDB
  dir: resource/dataset/imdb_reviews/
  folds:
  - 0
  - 1
  - 2
  - 3
  - 4
  max_length: 256
  num_classes: 10

However, overriding the loss function generates the wrong config:

python main.py model.loss=CrossEntropyLoss

tasks:
- fit
- eval
model:
  loss: CrossEntropyLoss
  name: BERT
  architecture: bert-base-uncased
  lr: 5.0e-05
  tokenizer:
    architecture: bert-base-uncased
data:
  name: IMDB
  dir: resource/dataset/imdb_reviews/
  folds:
  - 0
  - 1
  - 2
  - 3
  - 4
  max_length: 256
  num_classes: 10

Therefore, how to correctly generate a multi-level composition?

[Jasha10]

Let me cross-link to Omry's answer on StackOverflow.

[celsofranssa]

Let me cross-link to Omry's answer on StackOverflow.

Thank you! It's just the documentation I was looking for.

However, since most sub-configs are done using ".", using "/" on the command line loses cohesion.
How to structure the configs to keep "$python main.py model.loss=CrossEntropyLoss"?

[Jasha10]

I don't think there's an easy way to do that. As you have seen, writing model.loss=CrossEntropyLoss will overwrite only the model.loss key in your config.
If you want to load the CrossEntropyLoss.yaml file using the defaults list, you will need to use the slash notation model/loss=CrossEntropyLoss.

[celsofranssa]

Ok, thank you @Jasha10.