Document Butane local files features

[dustymabe]

This might exist but I couldn't find it easily so I'll just put this here. We should add some examples of the local files feature of fcct:

variant: fcos
version: 1.1.0
storage:
  # call with --files-dir=/path/to/files/zram-generator
  trees:
    - local: fakeroot

$ tree /path/to/files/zram-generator/fakeroot/
/path/to/files/zram-generator/fakeroot
└── etc
    └── systemd
        ├── system
        │   └── [email protected]
        ├── system-generators
        │   └── zram-generator
        └── zram-generator.conf
$ cat fcct_zram_generator.yaml | fcct --files-dir=/path/to/files/zram-generator

Another example this time merging configs:

variant: fcos
version: 1.1.0
ignition:
  config:
    merge:
      - local: ./generated-files/master.ign

cat resources/fcct-control-plane.yaml | fcct -d ./ -o ./generated-files/control-plane-processed.ign

[bgilbert]

https://github.com/coreos/fcct/blob/master/docs/examples.md#storage-and-files

We don't have an example of merging a local Ignition config, though.

[Okeanos]

I am interested in understanding the ignition.config.merge feature mentioned in the specs – from an ignition rendering perspective it looks like an additional ignition.config.merge.{compression|source} object is added to the output when using local as source. However, the documentation here is very sparse on actual behaviour (ignition itself is more verbose but doesn't have working examples with actual output to demonstrate behaviour either).

Am I correct in my understanding that an embedded config (referred to as ignition.merge.source) is the child config and the one doing the embedding, e.g. the one I would refer to in the actual butane call (butane --strict --files-dir ./embed parent.yaml > rendered.ign.json), is the parent config according to the ignition documentation?

If so, values in the parent are used unless a child value for the same (identifiable) key exists, correct? Doesn't this make specifying defaults, that can be overwritten in the "actual" config, "impossible"? A workaround would be not making large "common/default" embeddable pieces but multiple smaller pieces and only using those that don't need to be overwritten, right?
Personally, I would prefer something where I could inherit from a common ancestor and overwrite defaults as necessary instead of the (assumed) inverse.

[bgilbert]

Am I correct in my understanding that an embedded config (referred to as ignition.merge.source) is the child config and the one doing the embedding, e.g. the one I would refer to in the actual butane call (butane --strict --files-dir ./embed parent.yaml > rendered.ign.json), is the parent config according to the ignition documentation?

Yup, that's right. Note that the child config is an Ignition config, not a Butane config - i.e., if you want to write it with Butane, you need to transpile the child config separately.

If so, values in the parent are used unless a child value for the same (identifiable) key exists, correct? Doesn't this make specifying defaults, that can be overwritten in the "actual" config, "impossible"?

Children override parents, yes. The idea is that the top-level config should merge what it needs from child configs, with later child configs overriding earlier ones. For example, your config graph might be:

• user config
  • site-wide defaults
  • cluster defaults
    • configuration for standard service 1
    • configuration for standard service 2
  • RAID configuration for this hardware type
  • workload-specific defaults

If you really need to have the last set of parameters (the workload-specific defaults above) inline, you can use Butane's ignition.config.merge.local (or .inline), though the resulting config is harder to read.

[Okeanos]

Am I correct in my understanding that an embedded config (referred to as ignition.merge.source) is the child config and the one doing the embedding, e.g. the one I would refer to in the actual butane call (butane --strict --files-dir ./embed parent.yaml > rendered.ign.json), is the parent config according to the ignition documentation?

Yup, that's right. Note that the child config is an Ignition config, not a Butane config - i.e., if you want to write it with Butane, you need to transpile the child config separately.

Uhhh, that's … good to know because I just ran butane --strict --files-dir ./embed parent.yaml > rendered.ign.json locally and used ignition.config.merge.local to refer to a Butane yaml file inside of parent.yaml – no errors during transpiling. I would assume that the actual ignition process during installation would throw a hissy fit as a result? Imho that doesn't make for a good user experience. (In particular because the butane spec isn't very clear on what's allowed here to begin with and I naively assumed butane spec in yaml would be fine.)

Children override parents, yes. The idea is that the top-level config should merge what it needs from child configs, with later child configs overriding earlier ones. For example, your config graph might be:

• user config

  • site-wide defaults

  • cluster defaults

    • configuration for standard service 1
    • configuration for standard service 2

  • RAID configuration for this hardware type

  • workload-specific defaults

If you really need to have the last set of parameters (the workload-specific defaults above) inline, you can use Butane's ignition.config.merge.local (or .inline), though the resulting config is harder to read.

I can work with that, just wanted to make sure I understood the process correctly before moving stuff around.

[bgilbert]

Uhhh, that's … good to know because I just ran butane --strict --files-dir ./embed parent.yaml > rendered.ign.json locally and used ignition.config.merge.local to refer to a Butane yaml file inside of parent.yaml – no errors during transpiling. I would assume that the actual ignition process during installation would throw a hissy fit as a result? Imho that doesn't make for a good user experience.

Yup, you're right. Butane treats the merged config as opaque right now. That makes sense for remote configs, but probably doesn't make sense for local ones where we already have the config contents.

coreos/butane#118 tracks the ability to merge Butane configs, and I just filed coreos/butane#275 to validate local merged configs.