Skip to content

Latest commit

 

History

History
64 lines (62 loc) · 5.36 KB

journey_of_a_config_variable.md

File metadata and controls

64 lines (62 loc) · 5.36 KB
Raw

The journey of a config variable in openstack-k8s-operators

Our goal is to pass extra configuration from the CI job definition to the nova-compute process running on the EDPM compute node. E.g. we need to pass the following

[libvirt]
cpu_mode = custom
cpu_models = Nehalem

to nova-compute in CI to allow live migration between slightly different compute node models.

The config goes through the following journey:

  1. The snippet starts its life in the .zuul.yaml defining the CI job
  2. That config snippet is copied into a .conf file. by the ansible code in ci_framework
  3. Then the filepath is passed to the edpm_deploy_prep makefile target of install_yamls via theDATAPLANE_EXTRA_NOVA_CONFIG_FILE ENV variable.
  4. The install_yamls target uses kustomize to generate a ConfigMap that will store the config snippet in k8s.
  5. The ConfigMap gets created in k8s during the early phases of the EDPM deployment by the joint effort of install_yamls and ci_framework.
  6. A new OpenStackDataPlaneService CR referencing the osp.edpm.nova ansible playbook is created from the default nova service sample to reference the new ConfigMap. We need to do this as the default service CR is not modifiable.
  7. The default OpenStackDataPlaneNodeSet CR is changed to use the new OpenStackDataPlaneService CR instead of the default one.
  8. When the OpenStackDataPlaneDeployment CR is created the openstack-operator translates the OpenStackDataPlaneService CR to an OpenStackAnsibleEE CR and ensures that the ConfigMap is translated to an item in the extraVolumes field of the OpenStackAnsibleEE CR.
  9. The openstack-ansibleee-operator creates a k8s Job from the OpenStackAnsibleEE CR and instructs k8s to mount the ConfigMap to the Pod created from the Job.
  10. The osp.edpm.nova ansible playbook runs the edpm_nova ansible role in the k8s Pod where our ConfigMap content is mounted as a file now.
  11. The edpm_nova role copies every config file to the EDPM node (hypervisor) into /var/lib/openstack/config/nova directory then starts the nova_compute container with podman mounting the /var/lib/openstack/config/nova directory into it.
  12. Kolla in the nova_compute container copies the config files to /etc/nova/nova.conf.d/ directory and starts the nova-compute binary and the config snippet gets loaded by oslo.config and the magic happens.