From 197b88a0c36d584729d793961f39d625ee8ae85f Mon Sep 17 00:00:00 2001 From: Vincent Date: Mon, 11 Mar 2024 21:01:45 -0600 Subject: [PATCH 01/18] Playbook support (1.3.0) (#4) * Add playbook support * Update documentation * Dynamically load collection name where possible (init) --- README.md | 98 ++++++++++++++++++- galaxy.yml | 2 +- roles/init/files/create.yml | 33 ------- roles/init/tasks/asserts.yml | 10 +- roles/init/tasks/auto.yml | 45 ++++++--- roles/init/tasks/main.yml | 10 +- roles/init/templates/collections.yml.j2 | 4 +- roles/init/templates/create.yml.j2 | 6 +- .../destroy.yml => templates/destroy.yml.j2} | 4 +- roles/init/templates/molecule.yml.j2 | 11 ++- roles/prepare_controller/tasks/main.yml | 6 +- roles/prepare_controller/tasks/playbook.yml | 7 ++ 12 files changed, 172 insertions(+), 64 deletions(-) delete mode 100644 roles/init/files/create.yml rename roles/init/{files/destroy.yml => templates/destroy.yml.j2} (60%) create mode 100644 roles/prepare_controller/tasks/playbook.yml diff --git a/README.md b/README.md index ffbaefa..e0eb0e3 100644 --- a/README.md +++ b/README.md @@ -8,8 +8,35 @@ It provides tooling to create Molecule testing scenarios via the `init` role, an When utilizing an image with systemd support (systemd packages are installed, etc.), the `docker_platform` role supports the creation of Docker containers with a functional Systemd implementation, which can be used to test Ansible code that makes use of Systemd services or related functionality. +# What is Molecule? + +> Molecule project is designed to aid in the development and testing of Ansible roles. + +Molecule is a testing platform for your Ansible projects that enables testing of your code both during development and after release via CI infrastructure. + +Some resources on Molecule can be found here: +* [Developing and Testing Ansible Roles with Molecule and Podman - Part 1](https://www.ansible.com/blog/developing-and-testing-ansible-roles-with-molecule-and-podman-part-1) +* [Testing your Ansible roles with Molecule](https://www.jeffgeerling.com/blog/2018/testing-your-ansible-roles-molecule) +* [Ansible Collections: Role Tests with Molecule](https://ericsysmin.com/2020/04/30/ansible-collections-role-tests-with-molecule/) +* [Introducing Ansible Molecule with Ansible Automation Platform](https://developers.redhat.com/articles/2023/09/13/introducing-ansible-molecule-ansible-automation-platform#an_automation_testing_framework_built_for_the_enterprise) + +> [!WARNING] +> Some [fairly significant changes](https://ansible.readthedocs.io/projects/molecule/next/) have been made in Molecule v6. Most noticable among these are likely to be that `ansible` is now the only driver included by default (previously called `delegated`), and that the `molecule init` command now only supports creation of scenarios, not Ansible roles. +> +> This [RedHat article](https://developers.redhat.com/articles/2023/09/13/introducing-ansible-molecule-ansible-automation-platform#) has some more information on this change. +> +> When reading the above referenced articles, keep in mind their publishing dates, and that there may have been breaking changes to Molecule's functionality since that time! + # Using this collection +The following roles are provided: + +* [init](roles/init) - Initialize the Molecule testing framework for a project +* [docker_platform](roles/docker_platform) - Create a docker-based test platform for Molecule +* [prepare_controller](roles/prepare_controller) - Prepare a molecule controller to run local code tests + +The recommended way to use this collection is to provision Molecule scenarios using the [init role](roles/init). The `init` role provides template configurations that will work in various project types. + ## Host Requirements The host from which this collection is run (workstation, CI instance, etc.) must meet the following requirements: @@ -35,12 +62,13 @@ Docker CE can be installed by following the appropriate [installation instructio ## Project requirements -Roles within this collection will attempt to discover what type of project they are being utilized in. This is enabled by setting the appropriate `project_type` configuration variable to `auto`. The project type can also be explicitly specified if this is desired. +The `init` role from this collection will attempt to discover what type of project it is being utilized in. This is enabled by setting the `init_project_type` configuration variable to `auto`. The project type can also be explicitly specified if this is desired. Supported project types: -* `role` * `collection` * `monolith` +* `playbook` +* `role` When used with a role or collection, the Galaxy meta information for the role must be configured! @@ -146,9 +174,11 @@ or if your `collections/requirements.yml` includes this collection: ansible-galaxy collection install -p ./collections -r ./collections/requirements.yml ``` -#### Testing roles within a monolithic project +#### Testing roles and playbooks within a monolithic project + +When configuring molecule testing for individual roles or playbooks within a monolithic project (creating a `roles//molecule` or `playbooks//molecule` directory), take care _not_ to name the scenario "default", as there is already a "default" scenario for the monolithic project itself if you have created `molecule/default` as described above! Instead, name your role scenario with a unique name. -When configuring molecule testing for individual roles within a monolithic project (creating a `roles//molecule` directory), take care _not_ to name the scenario "default", as there is already a "default" scenario for the monolithic project itself if you have created `molecule/default` as described above! Instead, name your role scenario with a unique name. +For example (role): ```bash ROLE_NAME=your_role @@ -157,6 +187,66 @@ wget -P molecule/role-$ROLE_NAME https://raw.githubusercontent.com/syndr/ansible ansible-playbook molecule/role-$ROLE_NAME/init.yml ``` +Note that in this circumstance, you will need to specify the scenario name in order to run molecule against it (as it is not named `default`). + +Running the `molecule list` command will provide you an overview of the available scenarios + +```bash +❯ molecule list +INFO Running pb-example_playbook > list + ╷ ╷ ╷ ╷ ╷ + Instance Name │ Driver Name │ Provisioner Name │ Scenario Name │ Created │ Converged +╶────────────────────┼─────────────┼──────────────────┼─────────────────────┼─────────┼───────────╴ + docker-rockylinux9 │ default │ ansible │ pb-example_playbook │ false │ false + ╵ ╵ ╵ ╵ ╵ +``` + + +And running the full test suite for this playbook would be done as: + +```bash +molecule test -s pb-example_playbook +``` + +While running just the "converge" steps (IE: during development) would be: + +```bash +molecule converge -s pb-example_playbook +``` + +> [!TIP] +> The `molecule list` command will show multiple scenarios when run in the root of a monolithic project that also has molecule configured on individual playbooks or roles contained within it. Note that you will, however, still need to be in the appropriate role or playbook directory in order to successfully run these! + +### Playbooks + +Playbook configurations are similar to the `monolith` project type noted above, and are typically contained within monolithic projects. A project directory is considered a playbook if it contains a `tasks/` folder, but no role `meta/main.yml` configuration, and no `playbooks/` subdirectory. + +A playbook project configuration may look like: + +``` +playbooks +├── your_playbook +│   ├── main.yml +│   ├── README.md +│   ├── tasks +│   │   ├── asserts.yml +│   │   ├── main.yml +│   │   └── standard.yml +│   └── vars +└── [...] +``` + +Playbook configuration adds the following directories to the role path configuration (paths relative to the playbook `main.yml` or equivilant file): + +* `./roles` +* `./../roles` +* `./../../roles` + +It also adds the following directories to the collection path configuration (paths relative to the playbook `main.yml` or equivilant file): +* `./collections` +* `./../collections` +* `./../../collections` + # Contributing Pull requests are welcomed! diff --git a/galaxy.yml b/galaxy.yml index 129d7eb..3e9da7c 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -8,7 +8,7 @@ namespace: syndr name: molecule # The version of the collection. Must be compatible with semantic versioning -version: 1.2.1 +version: 1.3.0 # The path to the Markdown (.md) readme file. This path is relative to the root of the collection readme: README.md diff --git a/roles/init/files/create.yml b/roles/init/files/create.yml deleted file mode 100644 index 2a226ef..0000000 --- a/roles/init/files/create.yml +++ /dev/null @@ -1,33 +0,0 @@ ---- -- name: Create - hosts: localhost - gather_facts: false - tasks: - - name: Create platform - ansible.builtin.include_role: - name: syndr.molecule.docker_platform - vars: - docker_platform_name: "{{ item.name }}" - docker_platform_image: "{{ item.image }}" - docker_platform_systemd: "{{ item.systemd | default(false) }}" - docker_platform_modify_image: "{{ item.modify_image | default(false) }}" - docker_platform_privileged: "{{ item.privileged | default (false) }}" - docker_platform_state: present - loop: "{{ molecule_yml.platforms }}" - loop_control: - label: item.name - -# We want to avoid errors like "Failed to create temporary directory" -- name: Validate that inventory was refreshed - hosts: molecule - gather_facts: false - tasks: - - name: Check uname - ansible.builtin.raw: uname -a - register: result - changed_when: false - - - name: Display uname info - ansible.builtin.debug: - msg: "{{ result.stdout }}" - diff --git a/roles/init/tasks/asserts.yml b/roles/init/tasks/asserts.yml index f426dab..3b8920f 100644 --- a/roles/init/tasks/asserts.yml +++ b/roles/init/tasks/asserts.yml @@ -11,6 +11,14 @@ fail_msg: Global configuration option for init role is not sane success_msg: Sanity check passed +- name: Check for collection identification + ansible.builtin.assert: + that: + - __init_collection_config.collection_info.namespace is truthy + - __init_collection_config.collection_info.name is truthy + fail_msg: Collection ID information not found! This role must be run from within its collection! + success_msg: Collection ID found + - name: Check for project file paths ansible.builtin.stat: path: "{{ __init_item }}" @@ -28,7 +36,7 @@ fail_msg: Specified file path does not exist! loop: "{{ __init_filepath_stat.results }}" loop_control: - label: "{{ __init_item.item }}" + label: "{{ __init_item.stat.path }}" loop_var: __init_item - name: Check for secret if specified diff --git a/roles/init/tasks/auto.yml b/roles/init/tasks/auto.yml index fe3583c..8d22471 100644 --- a/roles/init/tasks/auto.yml +++ b/roles/init/tasks/auto.yml @@ -1,15 +1,15 @@ --- # Automagically deturmine what type of Ansible project this is -- name: Check for role meta configuration +- name: Check for role meta/main.yml ansible.builtin.stat: path: "{{ init_project_dir }}/meta/main.yml" - register: __init_rolepath_stat + register: __init_role_meta_stat -- name: Check for collection Galaxy configuration +- name: Check for collection galaxy.yml ansible.builtin.stat: path: "{{ init_project_dir }}/galaxy.yml" - register: __init_collectionpath_stat + register: __init_collection_galaxy_stat - name: Check for monolith block: @@ -30,16 +30,26 @@ - __init_monoroles_stat.stat.exists is true - __init_monoplaybooks_stat.stat.exists is true +- name: Check for playbook-specific files + block: + - name: Check for playbook tasks directory + ansible.builtin.stat: + path: "{{ init_project_dir }}/tasks" + register: __init_playbook_tasks_stat + - name: Perform convergence dance block: - name: Detect crossed streams ansible.builtin.assert: - that: __init_rolepath_stat.stat.exists is true and - __init_collectionpath_stat.stat.exists is false and + that: __init_role_meta_stat.stat.exists is true and + __init_collection_galaxy_stat.stat.exists is false and __init_monolith is false or - __init_collectionpath_stat.stat.exists is true or + __init_collection_galaxy_stat.stat.exists is true or __init_monolith is true and - __init_rolepath_stat.stat.exists is false + __init_role_meta_stat.stat.exists is false or + __init_playbook_tasks_stat.stat.exists is true and + __init_role_meta_stat.stat.exists is false and + __init_monolith is false success_msg: Known repository format detected! fail_msg: No known repository type detected. 😵 @@ -47,21 +57,30 @@ ansible.builtin.set_fact: init_project_type: role when: - - __init_rolepath_stat.stat.exists is true - - __init_collectionpath_stat.stat.exists is false - - __init_monolith is false + - __init_role_meta_stat.stat.exists is true + - __init_collection_galaxy_stat.stat.exists is false + - __init_monolith is false - name: Collection repository detected ansible.builtin.set_fact: init_project_type: collection - when: __init_collectionpath_stat.stat.exists is true + when: + - __init_collection_galaxy_stat.stat.exists is true + + - name: Playbook repository detected + ansible.builtin.set_fact: + init_project_type: playbook + when: + - __init_playbook_tasks_stat.stat.exists is true + - __init_role_meta_stat.stat.exists is false + - __init_monolith is false - name: Monolith repository detected ansible.builtin.set_fact: init_project_type: monolith when: - __init_monolith is true - - __init_rolepath_stat.stat.exists is false + - __init_role_meta_stat.stat.exists is false - name: Autodetection succeeded ansible.builtin.assert: diff --git a/roles/init/tasks/main.yml b/roles/init/tasks/main.yml index 4dc65cc..34902e5 100644 --- a/roles/init/tasks/main.yml +++ b/roles/init/tasks/main.yml @@ -1,6 +1,12 @@ --- # tasks file for init +- name: Load parent collection config + # NOTE: This will break if this role is moved outside of the collection! + ansible.builtin.include_vars: + file: "{{ role_path }}/../../MANIFEST.json" + name: __init_collection_config + - name: Check configuration data ansible.builtin.include_tasks: "{{ role_path }}/tasks/asserts.yml" @@ -24,7 +30,9 @@ loop: - collections.yml - requirements.yml + - create.yml - prepare.yml + - destroy.yml loop_control: loop_var: __init_item @@ -35,12 +43,10 @@ mode: 0644 backup: "{{ init_file_backup }}" loop: - - create.yml - converge.yml - side_effect.yml - verify.yml - cleanup.yml - - destroy.yml loop_control: loop_var: __init_item diff --git a/roles/init/templates/collections.yml.j2 b/roles/init/templates/collections.yml.j2 index e650b44..196bdad 100644 --- a/roles/init/templates/collections.yml.j2 +++ b/roles/init/templates/collections.yml.j2 @@ -1,6 +1,6 @@ --- collections: - - community.docker - - syndr.molecule + - name: community.docker + - name: syndr.molecule diff --git a/roles/init/templates/create.yml.j2 b/roles/init/templates/create.yml.j2 index fd4e4fa..7798591 100644 --- a/roles/init/templates/create.yml.j2 +++ b/roles/init/templates/create.yml.j2 @@ -1,13 +1,13 @@ --- -{% raw %} - name: Create hosts: localhost gather_facts: false tasks: - name: Create platform ansible.builtin.include_role: - name: syndr.molecule.docker_platform + name: {{ __init_collection_config.collection_info.namespace }}.{{ __init_collection_config.collection_info.name }}.docker_platform vars: +{% raw %} docker_platform_name: "{{ item.name }}" docker_platform_image: "{{ item.image }}" docker_platform_systemd: "{{ item.systemd | default(false) }}" @@ -36,7 +36,7 @@ - name: Load system facts ansible.builtin.setup: - filters: + filter: - ansible_service_mgr - name: Wait for systemd to complete initialization. diff --git a/roles/init/files/destroy.yml b/roles/init/templates/destroy.yml.j2 similarity index 60% rename from roles/init/files/destroy.yml rename to roles/init/templates/destroy.yml.j2 index 88c729d..9569108 100644 --- a/roles/init/files/destroy.yml +++ b/roles/init/templates/destroy.yml.j2 @@ -6,8 +6,10 @@ tasks: - name: Remove platform ansible.builtin.include_role: - name: syndr.molecule.docker_platform + name: {{ __init_collection_config.collection_info.namespace }}.{{ __init_collection_config.collection_info.name }}.docker_platform vars: +{% raw %} docker_platform_name: "{{ inventory_hostname }}" docker_platform_state: absent +{% endraw %} diff --git a/roles/init/templates/molecule.yml.j2 b/roles/init/templates/molecule.yml.j2 index 026745e..90591be 100644 --- a/roles/init/templates/molecule.yml.j2 +++ b/roles/init/templates/molecule.yml.j2 @@ -38,10 +38,15 @@ provisioner: {% if init_ansible_secret_path is truthy %} vault_password_file: {{ init_ansible_secret_path }} {% endif %} -{% if __init_monolith %} +{% if init_project_type == 'monolith' %} env: - ANSIBLE_ROLES_PATH: ${PWD}/roles:/usr/share/ansible/roles:/etc/ansible/roles:~/.ansible/roles - ANSIBLE_COLLECTIONS_PATH: ${PWD}/collections:/usr/share/ansible/collections:~/.ansible/collections + ANSIBLE_ROLES_PATH: /usr/share/ansible/roles:/etc/ansible/roles:~/.ansible/roles:${PWD}/roles + ANSIBLE_COLLECTIONS_PATH: /usr/share/ansible/collections:~/.ansible/collections:${PWD}/collections +{% endif %} +{% if init_project_type == 'playbook' %} + env: + ANSIBLE_ROLES_PATH: /usr/share/ansible/roles:/etc/ansible/roles:~/.ansible/roles:${PWD}/roles:${PWD}/../roles:${PWD}/../../roles + ANSIBLE_COLLECTIONS_PATH: /usr/share/ansible/collections:~/.ansible/collections:${PWD}/collections:${PWD}/../collections:${PWD}/../../collections {% endif %} scenario: create_sequence: diff --git a/roles/prepare_controller/tasks/main.yml b/roles/prepare_controller/tasks/main.yml index 41e286e..030ba96 100644 --- a/roles/prepare_controller/tasks/main.yml +++ b/roles/prepare_controller/tasks/main.yml @@ -8,7 +8,7 @@ - name: Verify project type ansible.builtin.assert: that: - - prepare_controller_project_type is in ['role', 'collection', 'monolith'] + - prepare_controller_project_type is in ['role', 'collection', 'monolith', 'playbook'] fail_msg: Unsupported project type specified! success_msg: Project type is supported @@ -20,6 +20,10 @@ ansible.builtin.include_tasks: "{{ role_path }}/tasks/collection.yml" when: prepare_controller_project_type == 'collection' +- name: Configure controller for playbook project type + ansible.builtin.include_tasks: "{{ role_path }}/tasks/playbook.yml" + when: prepare_controller_project_type == 'playbook' + - name: Configure controller for monolith project type ansible.builtin.include_tasks: "{{ role_path }}/tasks/monolith.yml" when: prepare_controller_project_type == 'monolith' diff --git a/roles/prepare_controller/tasks/playbook.yml b/roles/prepare_controller/tasks/playbook.yml new file mode 100644 index 0000000..d91316d --- /dev/null +++ b/roles/prepare_controller/tasks/playbook.yml @@ -0,0 +1,7 @@ +--- +# Configure molecule to run in the context of a standalone playbook + +- name: Standalone playbook configuration + ansible.builtin.debug: + msg: Make sure that you have run the 'init' role from this collection, or have otherwise set role and collection paths appropriately in Molecule configuration! + From 2ca973b3c0d6a54b27d02f3c2bbaf597137aeb20 Mon Sep 17 00:00:00 2001 From: syndr Date: Mon, 11 Mar 2024 21:12:32 -0600 Subject: [PATCH 02/18] Add galaxy publish ci --- .github/workflows/publish.yml | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 .github/workflows/publish.yml diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml new file mode 100644 index 0000000..6718cd5 --- /dev/null +++ b/.github/workflows/publish.yml @@ -0,0 +1,20 @@ +--- + +name: Deploy Collection + +on: + release: + types: + - published + +jobs: + deploy: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + + - name: Build and Deploy Collection + uses: artis3n/ansible_galaxy_collection@v2 + with: + api_key: '${{ secrets.GALAXY_API_KEY }}' + From 06d3246b5a67a6b7eadae2347d2ef300ad5c98eb Mon Sep 17 00:00:00 2001 From: syndr Date: Mon, 11 Mar 2024 21:16:48 -0600 Subject: [PATCH 03/18] Push to galaxy on all releases --- .github/workflows/publish.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 6718cd5..6287178 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -5,7 +5,7 @@ name: Deploy Collection on: release: types: - - published + - created jobs: deploy: From a16752737b22021a1d7b34ea60b67a376b89f476 Mon Sep 17 00:00:00 2001 From: Vincent Date: Tue, 12 Mar 2024 23:23:54 -0600 Subject: [PATCH 04/18] Hostvars support, cgroup fixes, templates (#5) Add support for specifying Ansible hostvars on a platform. Fix cgroup namespace for docker platform. Add more recommended configuration to prepare.yml template. Add more documentation. --- README.md | 45 +++++++++++++++++++++ galaxy.yml | 2 +- molecule/default/collections.yml | 4 +- molecule/default/converge.yml | 4 +- molecule/default/create.yml | 3 +- molecule/default/molecule.yml | 2 + roles/docker_platform/defaults/main.yml | 4 ++ roles/docker_platform/tasks/create.yml | 2 +- roles/docker_platform/tasks/present.yml | 6 ++- roles/init/tasks/asserts.yml | 8 ---- roles/init/tasks/main.yml | 6 --- roles/init/templates/collections.yml.j2 | 2 +- roles/init/templates/create.yml.j2 | 35 +++++++++++------ roles/init/templates/destroy.yml.j2 | 2 +- roles/init/templates/molecule.yml.j2 | 1 + roles/init/templates/prepare.yml.j2 | 52 +++++++++++++++++++++++-- 16 files changed, 137 insertions(+), 41 deletions(-) diff --git a/README.md b/README.md index e0eb0e3..3bf9527 100644 --- a/README.md +++ b/README.md @@ -8,6 +8,22 @@ It provides tooling to create Molecule testing scenarios via the `init` role, an When utilizing an image with systemd support (systemd packages are installed, etc.), the `docker_platform` role supports the creation of Docker containers with a functional Systemd implementation, which can be used to test Ansible code that makes use of Systemd services or related functionality. +# Table of Contents + +- [Ansible Collection - syndr.molecule](#ansible-collection---syndrmolecule) +- [What is Molecule?](#what-is-molecule) +- [Using this collection](#using-this-collection) + - [Host Requirements](#host-requirements) + - [Project requirements](#project-requirements) + - [Standalone roles](#standalone-roles) + - [Collections](#collections) + - [Monoliths](#monoliths) + - [Testing roles and playbooks within a monolithic project](#testing-roles-and-playbooks-within-a-monolithic-project) + - [Playbooks](#playbooks) +- [Using Molecule](#using-molecule) + - [Ansible Tags](#ansible-tags) +- [Contributing](#contributing) + # What is Molecule? > Molecule project is designed to aid in the development and testing of Ansible roles. @@ -27,6 +43,8 @@ Some resources on Molecule can be found here: > > When reading the above referenced articles, keep in mind their publishing dates, and that there may have been breaking changes to Molecule's functionality since that time! +More tips on using Molecule can be found [below](#using-molecule). + # Using this collection The following roles are provided: @@ -247,6 +265,33 @@ It also adds the following directories to the collection path configuration (pat * `./../collections` * `./../../collections` +# Using Molecule + +The most common Molecule commands that you will likely use are: + +```bash +molecule create # Create the test infrastructure, as defined in molecule.yml +molecule converge # Run the plays from converge.yml (launch your role/playbook) +molecule verify # Run the plays from verify.yml (test for desired state) +molecule test # Run the full test sequence +``` + +## Ansible Tags + +If [tags](https://docs.ansible.com/ansible/latest/playbook_guide/playbooks_tags.html) are used in your code to enable/disable certian functionality, they must be specified on the command line when running Molecule commands. To do so, use the `--` [command line option](https://ansible.readthedocs.io/projects/molecule/usage/#test-sequence-commands) to pass commands through to `ansible-playbook`. + +For example: + +```bash +molecule test -- --tags the-cheese +``` + +Or running `converge` using a non-default scenario: + +```bash +molecule converge -s pb-the_toaster -- --tags sourdough +``` + # Contributing Pull requests are welcomed! diff --git a/galaxy.yml b/galaxy.yml index 3e9da7c..9a8d9ae 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -8,7 +8,7 @@ namespace: syndr name: molecule # The version of the collection. Must be compatible with semantic versioning -version: 1.3.0 +version: 1.3.1 # The path to the Markdown (.md) readme file. This path is relative to the root of the collection readme: README.md diff --git a/molecule/default/collections.yml b/molecule/default/collections.yml index e650b44..196bdad 100644 --- a/molecule/default/collections.yml +++ b/molecule/default/collections.yml @@ -1,6 +1,6 @@ --- collections: - - community.docker - - syndr.molecule + - name: community.docker + - name: syndr.molecule diff --git a/molecule/default/converge.yml b/molecule/default/converge.yml index f904f40..e7bfe26 100644 --- a/molecule/default/converge.yml +++ b/molecule/default/converge.yml @@ -46,7 +46,7 @@ ansible.builtin.set_fact: test_prepare_fact: "{{ ansible_local.molecule.test_prepare_fact }}" - - name: Add your project test configuration here + - name: Dump the vars ansible.builtin.debug: - msg: Typically this will be via the ansible.builtin.include_role module or via import_playbook + var: testvar diff --git a/molecule/default/create.yml b/molecule/default/create.yml index 2a226ef..db9f212 100644 --- a/molecule/default/create.yml +++ b/molecule/default/create.yml @@ -11,7 +11,8 @@ docker_platform_image: "{{ item.image }}" docker_platform_systemd: "{{ item.systemd | default(false) }}" docker_platform_modify_image: "{{ item.modify_image | default(false) }}" - docker_platform_privileged: "{{ item.privileged | default (false) }}" + docker_platform_privileged: "{{ item.privileged | default(false) }}" + docker_platform_hostvars: "{{ item.hostvars | default({}) }}" docker_platform_state: present loop: "{{ molecule_yml.platforms }}" loop_control: diff --git a/molecule/default/molecule.yml b/molecule/default/molecule.yml index f8be0bc..bdd51ca 100644 --- a/molecule/default/molecule.yml +++ b/molecule/default/molecule.yml @@ -13,6 +13,8 @@ platforms: systemd: True modify_image: False privileged: False + hostvars: + testvar: stuff provisioner: name: ansible log: True diff --git a/roles/docker_platform/defaults/main.yml b/roles/docker_platform/defaults/main.yml index dc0369d..bb5812d 100644 --- a/roles/docker_platform/defaults/main.yml +++ b/roles/docker_platform/defaults/main.yml @@ -33,3 +33,7 @@ docker_platform_privileged: false # A list of tmpfs filesystem paths to be passed to the container docker_platform_tmpfs: [] +# Ansible hostvars that should be associated with the ansible test "host" created by this role +# stored in the format `name: value` +docker_platform_hostvars: {} + diff --git a/roles/docker_platform/tasks/create.yml b/roles/docker_platform/tasks/create.yml index 6d7e4ff..7bdc234 100644 --- a/roles/docker_platform/tasks/create.yml +++ b/roles/docker_platform/tasks/create.yml @@ -55,7 +55,7 @@ - name: Build docker volume list ansible.builtin.set_fact: - __docker_platform_volume_list: "{{ docker_platform_volumes + ['/sys/fs/cgroup/molecule-ci.scope:/sys/fs/cgroup:rw'] + __docker_platform_volume_list: "{{ docker_platform_volumes + ['/sys/fs/cgroup:/sys/fs/cgroup:rw'] if docker_platform_systemd else docker_platform_volumes }}" diff --git a/roles/docker_platform/tasks/present.yml b/roles/docker_platform/tasks/present.yml index 10ca1ee..9e445e6 100644 --- a/roles/docker_platform/tasks/present.yml +++ b/roles/docker_platform/tasks/present.yml @@ -53,13 +53,15 @@ - name: Add container to molecule_inventory vars: + __docker_platform_inventory_partial_hostvars: "{{ { + 'ansible_connection': 'community.docker.docker' + } | combine(docker_platform_hostvars, recursive=true) }}" __docker_platform_inventory_partial_yaml: | all: children: molecule: hosts: - "{{ docker_platform_name }}": - ansible_connection: community.docker.docker + "{{ docker_platform_name }}": {{ __docker_platform_inventory_partial_hostvars }} ansible.builtin.set_fact: __docker_platform_molecule_inventory: > {{ __docker_platform_current_molecule_inventory | default({}) | combine(__docker_platform_inventory_partial_yaml | from_yaml, recursive=true) }} diff --git a/roles/init/tasks/asserts.yml b/roles/init/tasks/asserts.yml index 3b8920f..307793e 100644 --- a/roles/init/tasks/asserts.yml +++ b/roles/init/tasks/asserts.yml @@ -11,14 +11,6 @@ fail_msg: Global configuration option for init role is not sane success_msg: Sanity check passed -- name: Check for collection identification - ansible.builtin.assert: - that: - - __init_collection_config.collection_info.namespace is truthy - - __init_collection_config.collection_info.name is truthy - fail_msg: Collection ID information not found! This role must be run from within its collection! - success_msg: Collection ID found - - name: Check for project file paths ansible.builtin.stat: path: "{{ __init_item }}" diff --git a/roles/init/tasks/main.yml b/roles/init/tasks/main.yml index 34902e5..778a914 100644 --- a/roles/init/tasks/main.yml +++ b/roles/init/tasks/main.yml @@ -1,12 +1,6 @@ --- # tasks file for init -- name: Load parent collection config - # NOTE: This will break if this role is moved outside of the collection! - ansible.builtin.include_vars: - file: "{{ role_path }}/../../MANIFEST.json" - name: __init_collection_config - - name: Check configuration data ansible.builtin.include_tasks: "{{ role_path }}/tasks/asserts.yml" diff --git a/roles/init/templates/collections.yml.j2 b/roles/init/templates/collections.yml.j2 index 196bdad..3513c62 100644 --- a/roles/init/templates/collections.yml.j2 +++ b/roles/init/templates/collections.yml.j2 @@ -2,5 +2,5 @@ collections: - name: community.docker - - name: syndr.molecule + - name: {{ ansible_collection_name }} diff --git a/roles/init/templates/create.yml.j2 b/roles/init/templates/create.yml.j2 index 7798591..f33754b 100644 --- a/roles/init/templates/create.yml.j2 +++ b/roles/init/templates/create.yml.j2 @@ -3,9 +3,9 @@ hosts: localhost gather_facts: false tasks: - - name: Create platform + - name: Create docker platform(s) ansible.builtin.include_role: - name: {{ __init_collection_config.collection_info.namespace }}.{{ __init_collection_config.collection_info.name }}.docker_platform + name: {{ ansible_collection_name }}.docker_platform vars: {% raw %} docker_platform_name: "{{ item.name }}" @@ -14,7 +14,9 @@ docker_platform_modify_image: "{{ item.modify_image | default(false) }}" docker_platform_modify_image_buildpath: "{{ item.modify_image_buildpath | default(molecule_ephemeral_directory + '/build') }}" docker_platform_privileged: "{{ item.privileged | default (false) }}" + docker_platform_hostvars: "{{ item.hostvars | default({}) }}" docker_platform_state: present + when: item.type == 'docker' loop: "{{ molecule_yml.platforms }}" loop_control: label: item.name @@ -39,15 +41,24 @@ filter: - ansible_service_mgr - - name: Wait for systemd to complete initialization. - ansible.builtin.command: systemctl is-system-running - register: systemctl_status - until: > - 'running' in systemctl_status.stdout or - 'degraded' in systemctl_status.stdout - retries: 30 - delay: 5 + - name: Check on Systemd + block: + - name: Wait for systemd to complete initialization. + ansible.builtin.command: systemctl is-system-running + register: systemctl_status + until: > + 'running' in systemctl_status.stdout or + 'degraded' in systemctl_status.stdout + retries: 30 + delay: 5 + changed_when: false + failed_when: systemctl_status.rc > 1 + + - name: Check systemd status + ansible.builtin.assert: + that: + - systemctl_status.stdout == 'running' + fail_msg: Systemd-enabled container does not have a healthy Systemd! + success_msg: Systemd is running when: ansible_service_mgr == 'systemd' - changed_when: false - failed_when: systemctl_status.rc > 1 diff --git a/roles/init/templates/destroy.yml.j2 b/roles/init/templates/destroy.yml.j2 index 9569108..4254478 100644 --- a/roles/init/templates/destroy.yml.j2 +++ b/roles/init/templates/destroy.yml.j2 @@ -6,7 +6,7 @@ tasks: - name: Remove platform ansible.builtin.include_role: - name: {{ __init_collection_config.collection_info.namespace }}.{{ __init_collection_config.collection_info.name }}.docker_platform + name: {{ ansible_collection_name }}.docker_platform vars: {% raw %} docker_platform_name: "{{ inventory_hostname }}" diff --git a/roles/init/templates/molecule.yml.j2 b/roles/init/templates/molecule.yml.j2 index 90591be..9aca6f8 100644 --- a/roles/init/templates/molecule.yml.j2 +++ b/roles/init/templates/molecule.yml.j2 @@ -19,6 +19,7 @@ platforms: modify_image_buildpath: {{ init_platform.config.modify_image_buildpath }} {% endif %} privileged: {{ init_platform.config.privileged | default(false) }} + hostvars: {} {% endif %} {% endfor %} provisioner: diff --git a/roles/init/templates/prepare.yml.j2 b/roles/init/templates/prepare.yml.j2 index 203f9f4..80582fb 100644 --- a/roles/init/templates/prepare.yml.j2 +++ b/roles/init/templates/prepare.yml.j2 @@ -2,6 +2,7 @@ - name: Prepare controller for execution hosts: localhost + tags: always tasks: - name: Configure for standalone role testing ansible.builtin.include_role: @@ -11,10 +12,50 @@ - name: Prepare target host for execution hosts: molecule + tags: always tasks: - - name: Add your host preparation tasks here! - ansible.builtin.debug: - msg: "IE: adding system users, installing required packages, etc." + ## + # Creating an admin service account for Molecule/Ansible to use for testing + # + # - If you run Ansible as a service account (you should) on your hosts and + # not as root, it is wise to also test as a non-root user! + # + # - To use this account, add the following to any plays targeting test + # infrastructure (such as in converge.yml): + # + # vars: + # ansible_user: molecule_runner + ## +{% raw %} + - name: Create ansible service account + vars: + molecule_user: molecule_runner + block: + - name: Create ansible group + ansible.builtin.group: + name: "{{ molecule_user }}" + + - name: Create ansible user + ansible.builtin.user: + name: "{{ molecule_user }}" + group: "{{ molecule_user }}" + + - name: Sudoers.d directory exists + ansible.builtin.file: + path: /etc/sudoers.d + state: directory + owner: root + group: root + mode: 0751 + + - name: Ansible user has sudo + ansible.builtin.copy: + content: | + {{ molecule_user }} ALL=(ALL) NOPASSWD: ALL + dest: /etc/sudoers.d/ansible + owner: root + group: root + mode: 0600 - name: "Save vars to host (IE: generated test credentials, etc.)" become: true @@ -27,7 +68,6 @@ group: root mode: 0744 -{% raw %} - name: Persistent data saved to local Ansible facts ansible.builtin.copy: dest: /etc/ansible/facts.d/molecule.fact @@ -37,3 +77,7 @@ mode: 0644 {% endraw %} + - name: Add your host preparation tasks here! + ansible.builtin.debug: + msg: "IE: adding system users, installing required packages, etc." + From d79a261b25ede07333913edc68b05c94849d76ba Mon Sep 17 00:00:00 2001 From: Vincent Date: Sun, 24 Mar 2024 16:05:35 -0600 Subject: [PATCH 05/18] Add separate `platform` role (#6) Fix the `docker_platform` role so that it works correctly when more than one platform is specified in molecule.yml. Move non-provider-specific platform configuration to a `platform` role. This is a breaking change for this version, as functions of `docker_platform` are now provided by platform. Replace any references to `docker_platform` in playbooks using this collection with `platform`, options stay the same. --- .github/workflows/latest.yml | 23 +++ .github/workflows/publish.yml | 4 +- LICENSE | 22 +++ galaxy.yml | 4 +- molecule/default/collections.yml | 5 + molecule/default/converge.yml | 84 ++++++++++- molecule/default/create.yml | 39 ++++-- molecule/default/molecule.yml | 22 ++- molecule/default/prepare.yml | 125 ++++++++++++++++- roles/docker_platform/README.md | 2 +- roles/docker_platform/tasks/absent.yml | 1 + roles/docker_platform/tasks/create.yml | 124 ----------------- roles/docker_platform/tasks/present.yml | 177 +++++++++++++++--------- roles/init/templates/collections.yml.j2 | 4 +- roles/init/templates/create.yml.j2 | 15 +- roles/init/templates/prepare.yml.j2 | 2 +- roles/platform/README.md | 38 +++++ roles/platform/defaults/main.yml | 15 ++ roles/platform/handlers/main.yml | 2 + roles/platform/meta/main.yml | 52 +++++++ roles/platform/tasks/inventory.yml | 74 ++++++++++ roles/platform/tasks/main.yml | 11 ++ roles/platform/tasks/provision.yml | 31 +++++ roles/platform/tests/inventory | 2 + roles/platform/tests/test.yml | 5 + roles/platform/vars/main.yml | 7 + 26 files changed, 662 insertions(+), 228 deletions(-) create mode 100644 .github/workflows/latest.yml create mode 100644 LICENSE delete mode 100644 roles/docker_platform/tasks/create.yml create mode 100644 roles/platform/README.md create mode 100644 roles/platform/defaults/main.yml create mode 100644 roles/platform/handlers/main.yml create mode 100644 roles/platform/meta/main.yml create mode 100644 roles/platform/tasks/inventory.yml create mode 100644 roles/platform/tasks/main.yml create mode 100644 roles/platform/tasks/provision.yml create mode 100644 roles/platform/tests/inventory create mode 100644 roles/platform/tests/test.yml create mode 100644 roles/platform/vars/main.yml diff --git a/.github/workflows/latest.yml b/.github/workflows/latest.yml new file mode 100644 index 0000000..c5604e9 --- /dev/null +++ b/.github/workflows/latest.yml @@ -0,0 +1,23 @@ +--- + +name: Update `latest` tag +on: + release: + types: [published] + +jobs: + run: + runs-on: ubuntu-latest + + steps: + - name: Checkout repository + uses: actions/checkout@v4 + + - name: Run latest-tag + uses: EndBug/latest-tag@latest + with: + ref: latest + description: This tag is automatically generated on new releases. + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 6287178..fb2b080 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -1,15 +1,15 @@ --- name: Deploy Collection - on: release: types: - created - jobs: deploy: runs-on: ubuntu-latest + if: | + github.event.release.prerelase == false steps: - uses: actions/checkout@v4 diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..bc05ea7 --- /dev/null +++ b/LICENSE @@ -0,0 +1,22 @@ +MIT License + +Copyright (c) 2024 syndr + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. + diff --git a/galaxy.yml b/galaxy.yml index 9a8d9ae..c5a9fd4 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -8,7 +8,7 @@ namespace: syndr name: molecule # The version of the collection. Must be compatible with semantic versioning -version: 1.3.1 +version: 1.4.0-dev # The path to the Markdown (.md) readme file. This path is relative to the root of the collection readme: README.md @@ -56,7 +56,7 @@ repository: https://github.com/syndr/ansible-collection-molecule #homepage: http://example.com # The URL to the collection issue tracker -issues: https://github.com/syndr/ansible-collection-molecule/issues +#issues: https://github.com/syndr/ansible-collection-molecule/issues # A list of file glob-like patterns used to filter any files or directories that should not be included in the build # artifact. A pattern is matched from the relative path of the file or directory of the collection directory. This diff --git a/molecule/default/collections.yml b/molecule/default/collections.yml index 196bdad..5dcfda8 100644 --- a/molecule/default/collections.yml +++ b/molecule/default/collections.yml @@ -1,6 +1,11 @@ --- collections: + - name: community.general - name: community.docker + #- name: git+https://github.com/syndr/ansible-collection-molecule.git + # type: git + # version: latest - name: syndr.molecule + version: 1.4.0-dev diff --git a/molecule/default/converge.yml b/molecule/default/converge.yml index e7bfe26..a8e01e0 100644 --- a/molecule/default/converge.yml +++ b/molecule/default/converge.yml @@ -18,6 +18,8 @@ - name: Converge hosts: molecule + vars: + ansible_user: molecule_runner tasks: - name: Check uname ansible.builtin.raw: uname -a @@ -42,11 +44,81 @@ var: ansible_facts.ansible_local verbosity: 1 - - name: Load preparation facts - ansible.builtin.set_fact: - test_prepare_fact: "{{ ansible_local.molecule.test_prepare_fact }}" + - name: Test local fact exists + ansible.builtin.assert: + that: + - ansible_local.molecule.test_prepare_fact is defined + fail_msg: Something went wrong with storing local facts! + success_msg: Local fact exists - - name: Dump the vars - ansible.builtin.debug: - var: testvar + - name: Run the init role + vars: + test_dir: /tmp/molecule-init + test_init_wget_path: "https://raw.githubusercontent.com/syndr/ansible-collection-molecule/main/roles/init/files/init.yml" + test_collection_dir: "{{ test_dir }}/ci_testing/test_collection" + test_role_dir: "{{ test_dir }}/ci_testing/test_collection" + block: + - name: Local work dir exists + ansible.builtin.file: + path: "{{ test_dir }}" + state: directory + + - name: Test with a collection + block: + - name: Test collection is created + ansible.builtin.command: + chdir: "{{ test_dir }}" + cmd: ansible-galaxy collection init ci_testing.test_collection + creates: "{{ test_collection_dir }}" + + - name: Molecule scenario dir exists for collection + ansible.builtin.file: + path: "{{ test_collection_dir }}/molecule/default" + state: directory + + - name: Molecule init playbook exists for collection + ansible.builtin.command: + chdir: "{{ test_collection_dir }}" + cmd: wget -P molecule/default {{ test_init_wget_path }} + creates: "{{ test_collection_dir }}/molecule/default/init.yml" + + - name: Run init playbook on collection + # TODO: Actually check idempotence on this + ansible.builtin.command: + chdir: "{{ test_collection_dir }}" + cmd: ansible-playbook molecule/default/init.yml + changed_when: false + + - name: Test with a role + block: + - name: Test role is created + ansible.builtin.command: + chdir: "{{ test_dir }}" + cmd: ansible-galaxy role init test_role + creates: "{{ test_role_dir }}" + + - name: Molecule scenario dir exists for role + ansible.builtin.file: + path: "{{ test_role_dir }}/molecule/default" + state: directory + + - name: Molecule init playbook exists for role + ansible.builtin.command: + chdir: "{{ test_role_dir }}" + cmd: wget -P molecule/default {{ test_init_wget_path }} + creates: "{{ test_role_dir }}/molecule/default/init.yml" + + - name: Run init playbook on role + # TODO: Actually check idempotence on this + ansible.builtin.command: + chdir: "{{ test_role_dir }}" + cmd: ansible-playbook molecule/default/init.yml + changed_when: false + + - name: Test with a monolith + block: + # TODO: Find/create a public monolith repo we can use to test this + - name: "TODO: Write monolith test" + ansible.builtin.debug: + msg: This test hasn't been written! You get a cookie if you can fix that! 🍪 diff --git a/molecule/default/create.yml b/molecule/default/create.yml index db9f212..fda8be2 100644 --- a/molecule/default/create.yml +++ b/molecule/default/create.yml @@ -5,15 +5,12 @@ tasks: - name: Create platform ansible.builtin.include_role: - name: syndr.molecule.docker_platform + name: syndr.molecule.platform vars: - docker_platform_name: "{{ item.name }}" - docker_platform_image: "{{ item.image }}" - docker_platform_systemd: "{{ item.systemd | default(false) }}" - docker_platform_modify_image: "{{ item.modify_image | default(false) }}" - docker_platform_privileged: "{{ item.privileged | default(false) }}" - docker_platform_hostvars: "{{ item.hostvars | default({}) }}" - docker_platform_state: present + platform_name: "{{ item.name }}" + platform_state: present + platform_type: "{{ item.type }}" + platform_molecule_cfg: "{{ item }}" loop: "{{ molecule_yml.platforms }}" loop_control: label: item.name @@ -32,3 +29,29 @@ ansible.builtin.debug: msg: "{{ result.stdout }}" + - name: Load system facts + ansible.builtin.setup: + filter: + - ansible_service_mgr + + - name: Check on Systemd + block: + - name: Wait for systemd to complete initialization. + ansible.builtin.command: systemctl is-system-running + register: systemctl_status + until: > + 'running' in systemctl_status.stdout or + 'degraded' in systemctl_status.stdout + retries: 30 + delay: 5 + changed_when: false + failed_when: systemctl_status.rc > 1 + + - name: Check systemd status + ansible.builtin.assert: + that: + - systemctl_status.stdout == 'running' + fail_msg: Systemd-enabled container does not have a healthy Systemd! + success_msg: Systemd is running + when: ansible_service_mgr == 'systemd' + diff --git a/molecule/default/molecule.yml b/molecule/default/molecule.yml index bdd51ca..db2a5bd 100644 --- a/molecule/default/molecule.yml +++ b/molecule/default/molecule.yml @@ -8,13 +8,27 @@ driver: managed: true login_cmd_template: 'docker exec -ti {instance} bash' platforms: - - name: instance - image: geerlingguy/docker-${MOLECULE_GEERLINGGUY_DISTRO:-rockylinux9}-ansible:latest + - name: docker-rockylinux9 + type: docker + image: geerlingguy/docker-rockylinux9-ansible:latest systemd: True modify_image: False privileged: False - hostvars: - testvar: stuff + hostvars: {} + - name: docker-fedora39 + type: docker + image: geerlingguy/docker-fedora39-ansible:latest + systemd: True + modify_image: False + privileged: False + hostvars: {} + - name: docker-ubuntu2204 + type: docker + image: geerlingguy/docker-ubuntu2204-ansible:latest + systemd: True + modify_image: False + privileged: False + hostvars: {} provisioner: name: ansible log: True diff --git a/molecule/default/prepare.yml b/molecule/default/prepare.yml index 4c3f16c..229ae43 100644 --- a/molecule/default/prepare.yml +++ b/molecule/default/prepare.yml @@ -9,12 +9,57 @@ vars: prepare_controller_project_type: collection + - name: Archive this project for transfer + community.general.archive: + path: "{{ playbook_dir }}/../../" + dest: "{{ molecule_ephemeral_directory }}/project.tar" + format: tar + - name: Prepare target host for execution hosts: molecule + tags: always + vars: + molecule_user: molecule_runner tasks: - - name: Add your host preparation tasks here! - ansible.builtin.debug: - msg: "IE: adding system users, installing required packages, etc." + ## + # Creating an admin service account for Molecule/Ansible to use for testing + # + # - If you run Ansible as a service account (you should) on your hosts and + # not as root, it is wise to also test as a non-root user! + # + # - To use this account, add the following to any plays targeting test + # infrastructure (such as in converge.yml): + # + # vars: + # ansible_user: molecule_runner + ## + - name: Create ansible service account + block: + - name: Create ansible group + ansible.builtin.group: + name: "{{ molecule_user }}" + + - name: Create ansible user + ansible.builtin.user: + name: "{{ molecule_user }}" + group: "{{ molecule_user }}" + + - name: Sudoers.d directory exists + ansible.builtin.file: + path: /etc/sudoers.d + state: directory + owner: root + group: root + mode: 0751 + + - name: Ansible user has sudo + ansible.builtin.copy: + content: | + {{ molecule_user }} ALL=(ALL) NOPASSWD: ALL + dest: /etc/sudoers.d/ansible + owner: root + group: root + mode: 0600 - name: "Save vars to host (IE: generated test credentials, etc.)" become: true @@ -25,8 +70,7 @@ state: directory owner: root group: root - mode: 0744 - + mode: 0755 - name: Persistent data saved to local Ansible facts ansible.builtin.copy: @@ -36,3 +80,74 @@ group: root mode: 0644 + - name: Collect host facts + ansible.builtin.setup: + filter: + - ansible_distribution* + + - name: Install packages + # NOTE: If testing with a distro not mentioned in this block, you'll need to add code for it here! + block: + - name: Install packages for ubuntu + block: + - name: Install prereqs (Ubuntu) + ansible.builtin.apt: + update_cache: true + pkg: + - software-properties-common + - dirmngr + + - name: Install Ansible PPA (Ubuntu) + ansible.builtin.apt_repository: + repo: "ppa:ansible/ansible" + + - name: Install packages (Ubuntu) + ansible.builtin.apt: + pkg: + - ansible + - wget + when: ansible_distribution | lower == 'ubuntu' + + - name: Install packages (RHEL Family) + ansible.builtin.package: + name: + - ansible-core + - wget + when: ansible_distribution_file_variety | lower == 'redhat' + + - name: Verify application availibility + ansible.builtin.command: + cmd: "{{ item }}" + changed_when: false + loop: + - ansible-playbook --version + - wget --version + + - name: Add the working version of this project to test host + become: true + become_user: "{{ molecule_user }}" + vars: + collection_namespace: "{{ (lookup('file', playbook_dir ~ '/../../galaxy.yml') | from_yaml).namespace }}" + collection_name: "{{ (lookup('file', playbook_dir ~ '/../../galaxy.yml') | from_yaml).name }}" + block: + - name: Local user collections path exists + ansible.builtin.file: + path: ~/.ansible/collections/ansible_collections + state: directory + + - name: Project dir does not exist (Clean Slate 󰩸) + ansible.builtin.file: + path: ~/.ansible/collections/ansible_collections/{{ collection_namespace }}/{{ collection_name }} + state: absent + + - name: Recreate project dir + ansible.builtin.file: + path: ~/.ansible/collections/ansible_collections/{{ collection_namespace }}/{{ collection_name }} + state: directory + + - name: Deploy project to host + ansible.builtin.unarchive: + src: "{{ molecule_ephemeral_directory }}/project.tar" + dest: ~/.ansible/collections/ansible_collections/{{ collection_namespace }}/{{ collection_name }} + copy: true + diff --git a/roles/docker_platform/README.md b/roles/docker_platform/README.md index e8529ed..5d61f4b 100644 --- a/roles/docker_platform/README.md +++ b/roles/docker_platform/README.md @@ -54,7 +54,7 @@ Configuration that should not require modification: docker_platform_molecule_ephemeral_directory: "{{ molecule_ephemeral_directory }}" ``` -Molecule variables expected: +Molecule variables expected: - `molecule_ephemeral_directory` Dependencies diff --git a/roles/docker_platform/tasks/absent.yml b/roles/docker_platform/tasks/absent.yml index d19690e..cf53114 100644 --- a/roles/docker_platform/tasks/absent.yml +++ b/roles/docker_platform/tasks/absent.yml @@ -11,6 +11,7 @@ state: absent auto_remove: true +# TODO: Remove just this host, not the whole inventory - name: Remove dynamic molecule inventory delegate_to: localhost block: diff --git a/roles/docker_platform/tasks/create.yml b/roles/docker_platform/tasks/create.yml deleted file mode 100644 index 7bdc234..0000000 --- a/roles/docker_platform/tasks/create.yml +++ /dev/null @@ -1,124 +0,0 @@ ---- -# Create a docker container for use by molecule -# -# Expected to be called in a loop with `platform` defined as the loop var -# (loop off of `platforms` list in molecule.yml) - - -- name: Docker image needs to be customized - block: - - name: Check build path - ansible.builtin.stat: - path: "{{ docker_platform_modify_image_buildpath }}" - register: __docker_platform_buildpath_stat - - - name: Build directory doesn't exist - block: - - name: Create build directory - ansible.builtin.file: - path: "{{ docker_platform_modify_image_buildpath }}" - state: directory - mode: 0755 - - - name: Copy templates - ansible.builtin.template: - src: templates/{{ __docker_platform_item }} - dest: "{{ docker_platform_modify_image_buildpath}}/{{ __docker_platform_item | regex_replace('\\.j2$', '') }}" - loop: - - bash.service.j2 - - entrypoint.sh.j2 - - Dockerfile.j2 - loop_control: - loop_var: __docker_platform_item - when: __docker_platform_buildpath_stat.stat.exists is false - - - name: Build local image name - ansible.builtin.set_fact: - __docker_platform_built_image_name: "molecule-local-build/{{ docker_platform_image | split(':') | first | split('/') | last }}-custom" - - - name: Docker image is built - community.docker.docker_image: - name: "{{ __docker_platform_built_image_name }}" - build: - path: "{{ docker_platform_modify_image_buildpath }}" - cache_from: "{{ docker_platform_image }}" - source: build - force_source: true # Always build a new image when this is run - tag: latest - register: image_build_output - - - name: Show image build details - ansible.builtin.debug: - var: image_build_output - verbosity: 1 - when: docker_platform_modify_image - -- name: Build docker volume list - ansible.builtin.set_fact: - __docker_platform_volume_list: "{{ docker_platform_volumes + ['/sys/fs/cgroup:/sys/fs/cgroup:rw'] - if docker_platform_systemd - else docker_platform_volumes }}" - -- name: Runtime docker container is present and running - community.docker.docker_container: - name: "{{ docker_platform_name }}" - image: "{{ __docker_platform_built_image_name | default(docker_platform_image) }}" - state: started - command: "{{ docker_platform_command }}" - log_driver: json-file - hostname: molecule-ci-{{ docker_platform_name }} - init: false - cgroupns_mode: "{{ 'host' if docker_platform_systemd is true else 'private' }}" - privileged: "{{ docker_platform_privileged }}" - tmpfs: "{{ docker_platform_tmpfs + ['/run'] if docker_platform_systemd else docker_platform_tmpfs }}" - volumes: "{{ __docker_platform_volume_list }}" - register: __docker_platform_create_result - -- name: Print some info - ansible.builtin.debug: - msg: "{{ __docker_platform_create_result }}" - verbosity: 1 - -- name: Fail if container is not running - block: - - name: Retrieve container log - ansible.builtin.command: - cmd: docker logs {{ __docker_platform_create_result.container.Name }} - changed_when: false - register: __docker_platform_logfile_cmd - - - name: Display container log - ansible.builtin.fail: - msg: "{{ __docker_platform_logfile_cmd.stdout ~ __docker_platform_logfile_cmd.stderr }}" - when: > - __docker_platform_create_result.container.State.ExitCode != 0 or - not __docker_platform_create_result.container.State.Running - -- name: Systemd status is healthy - block: - - name: System service manager is Systemd - ansible.builtin.assert: - that: - - "ansible_service_mgr == 'systemd'" - fail_msg: Systemd is enabled, but container service manager isn't Systemd! Is this a Systemd-enabled container? - - - name: Systemd has completed initialization - ansible.builtin.command: - cmd: systemctl is-system-running - register: __docker_platform_systemctl_status - until: > - 'running' in __docker_platform_systemctl_status.stdout or - 'degraded' in __docker_platform_systemctl_status.stdout - retries: 30 - delay: 5 - changed_when: false - failed_when: __docker_platform_systemctl_status.rc > 1 - - - name: System is healthy - ansible.builtin.assert: - that: - - "'running' in __docker_platform_systemctl_status.stdout" - success_msg: Systemd is healthy - fail_msg: Systemd is unhealthy - when: docker_platform_systemd - diff --git a/roles/docker_platform/tasks/present.yml b/roles/docker_platform/tasks/present.yml index 9e445e6..1dc97c3 100644 --- a/roles/docker_platform/tasks/present.yml +++ b/roles/docker_platform/tasks/present.yml @@ -1,84 +1,133 @@ --- +# Create a docker container for use by molecule +# +# Expected to be called in a loop with `platform` defined as the loop var +# (loop off of `platforms` list in molecule.yml) + - name: Initialize state ansible.builtin.set_fact: # Number of times this role has been included during this playbook run - __docker_platform_run_count: "{{ __docker_platform_run_count | default(0) + 1 }}" + __docker_platform_run_count: "{{ __docker_platform_run_count | default(0) | int + 1 }}" - name: Load system facts ansible.builtin.setup: filter: - ansible_service_mgr -- name: Create instance docker container - ansible.builtin.include_tasks: "{{ role_path }}/tasks/create.yml" - -- name: Load existing instance configuration +- name: Docker image needs to be customized block: - - name: Load existing instance configuration file - ansible.builtin.slurp: - src: "{{ docker_platform_molecule_ephemeral_directory }}/instance_config.yml" - register: __docker_platform_current_instance_config_b64 - ignore_errors: true + - name: Check build path + ansible.builtin.stat: + path: "{{ docker_platform_modify_image_buildpath }}" + register: __docker_platform_buildpath_stat - - name: Decode instance configuration data - ansible.builtin.set_fact: - __docker_platform_current_instance_config: "{{ __docker_platform_current_instance_config_b64.content | default('') | b64decode | from_yaml }}" - when: __docker_platform_run_count | int > 1 - -- name: Write instance config file - ansible.builtin.copy: - # This is very basic - just needs an item there to show as managed with docker config - content: | - {% if __docker_platform_current_instance_config is defined %} - {{ __docker_platform_current_instance_config | to_yaml }} - {% endif %} - - instance: {{ docker_platform_name }} - connection: docker - dest: "{{ docker_platform_molecule_ephemeral_directory }}/instance_config.yml" - mode: "0600" - -- name: Load existing molecule inventory - block: - - name: Load existing molecule inventory file - ansible.builtin.slurp: - src: "{{ docker_platform_molecule_ephemeral_directory }}/molecule_inventory.yml" - register: __docker_platform_current_molecule_inventory_b64 - ignore_errors: true + - name: Build directory doesn't exist + block: + - name: Create build directory + ansible.builtin.file: + path: "{{ docker_platform_modify_image_buildpath }}" + state: directory + mode: 0755 - - name: Decode instance configuration data + - name: Copy templates + ansible.builtin.template: + src: templates/{{ __docker_platform_item }} + dest: "{{ docker_platform_modify_image_buildpath}}/{{ __docker_platform_item | regex_replace('\\.j2$', '') }}" + loop: + - bash.service.j2 + - entrypoint.sh.j2 + - Dockerfile.j2 + loop_control: + loop_var: __docker_platform_item + when: __docker_platform_buildpath_stat.stat.exists is false + + - name: Build local image name ansible.builtin.set_fact: - __docker_platform_current_molecule_inventory: "{{ __docker_platform_current_molecule_inventory_b64.content | default({}) | b64decode | from_yaml }}" - when: __docker_platform_run_count | int > 1 - -- name: Add container to molecule_inventory - vars: - __docker_platform_inventory_partial_hostvars: "{{ { - 'ansible_connection': 'community.docker.docker' - } | combine(docker_platform_hostvars, recursive=true) }}" - __docker_platform_inventory_partial_yaml: | - all: - children: - molecule: - hosts: - "{{ docker_platform_name }}": {{ __docker_platform_inventory_partial_hostvars }} + __docker_platform_built_image_name: "molecule-local-build/{{ docker_platform_image | split(':') | first | split('/') | last }}-custom" + + - name: Docker image is built + community.docker.docker_image: + name: "{{ __docker_platform_built_image_name }}" + build: + path: "{{ docker_platform_modify_image_buildpath }}" + cache_from: "{{ docker_platform_image }}" + source: build + force_source: true # Always build a new image when this is run + tag: latest + register: image_build_output + + - name: Show image build details + ansible.builtin.debug: + var: image_build_output + verbosity: 1 + when: docker_platform_modify_image + +- name: Build docker volume list ansible.builtin.set_fact: - __docker_platform_molecule_inventory: > - {{ __docker_platform_current_molecule_inventory | default({}) | combine(__docker_platform_inventory_partial_yaml | from_yaml, recursive=true) }} + __docker_platform_volume_list: "{{ docker_platform_volumes + ['/sys/fs/cgroup:/sys/fs/cgroup:rw'] + if docker_platform_systemd + else docker_platform_volumes }}" + +- name: "{{ docker_platform_name }} docker container is present and running" + community.docker.docker_container: + name: "{{ docker_platform_name }}" + image: "{{ __docker_platform_built_image_name | default(docker_platform_image) }}" + state: started + command: "{{ docker_platform_command }}" + log_driver: json-file + hostname: molecule-ci-{{ docker_platform_name }} + init: false + cgroupns_mode: "{{ 'host' if docker_platform_systemd is true else 'private' }}" + privileged: "{{ docker_platform_privileged }}" + tmpfs: "{{ docker_platform_tmpfs + ['/run', '/run/lock'] if docker_platform_systemd else docker_platform_tmpfs }}" + volumes: "{{ __docker_platform_volume_list }}" + register: __docker_platform_create_result + +- name: Print creation output + ansible.builtin.debug: + msg: "{{ __docker_platform_create_result }}" + verbosity: 1 -- name: Write molecule inventory file - ansible.builtin.copy: - content: | - {{ __docker_platform_molecule_inventory | to_yaml }} - dest: "{{ docker_platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" - mode: "0600" +- name: Fail if is not running + block: + - name: Retrieve {{ docker_platform_name }} container log + ansible.builtin.command: + cmd: docker logs {{ __docker_platform_create_result.container.Name }} + changed_when: false + register: __docker_platform_logfile_cmd + + - name: Container {{ docker_platform_name }} failed to start + ansible.builtin.fail: + msg: "{{ __docker_platform_logfile_cmd.stdout ~ __docker_platform_logfile_cmd.stderr }}" + when: > + __docker_platform_create_result.container.State.ExitCode != 0 or + not __docker_platform_create_result.container.State.Running -- name: Force inventory refresh - ansible.builtin.meta: refresh_inventory +- name: "{{ docker_platform_name }} Systemd status is healthy" + block: + - name: System service manager is Systemd + ansible.builtin.assert: + that: + - "ansible_service_mgr == 'systemd'" + fail_msg: Systemd is enabled, but container service manager isn't Systemd! Is this a Systemd-enabled container? -- name: Fail if molecule group is missing - ansible.builtin.assert: - that: "'molecule' in groups" - fail_msg: | - molecule group was not found inside inventory groups: {{ groups }} + - name: Systemd has completed initialization + ansible.builtin.command: + cmd: systemctl is-system-running + register: __docker_platform_systemctl_status + until: > + 'running' in __docker_platform_systemctl_status.stdout or + 'degraded' in __docker_platform_systemctl_status.stdout + retries: 30 + delay: 5 + changed_when: false + failed_when: __docker_platform_systemctl_status.rc > 1 + - name: "{{ docker_platform_name }} Systemd is healthy" + ansible.builtin.assert: + that: + - "'running' in __docker_platform_systemctl_status.stdout" + success_msg: Systemd is healthy + fail_msg: Systemd is unhealthy + when: docker_platform_systemd diff --git a/roles/init/templates/collections.yml.j2 b/roles/init/templates/collections.yml.j2 index 3513c62..83d66c4 100644 --- a/roles/init/templates/collections.yml.j2 +++ b/roles/init/templates/collections.yml.j2 @@ -2,5 +2,7 @@ collections: - name: community.docker - - name: {{ ansible_collection_name }} + - name: git+https://github.com/syndr/ansible-collection-molecule.git + type: git + version: latest diff --git a/roles/init/templates/create.yml.j2 b/roles/init/templates/create.yml.j2 index f33754b..c203a3f 100644 --- a/roles/init/templates/create.yml.j2 +++ b/roles/init/templates/create.yml.j2 @@ -5,18 +5,13 @@ tasks: - name: Create docker platform(s) ansible.builtin.include_role: - name: {{ ansible_collection_name }}.docker_platform + name: {{ ansible_collection_name }}.platform vars: {% raw %} - docker_platform_name: "{{ item.name }}" - docker_platform_image: "{{ item.image }}" - docker_platform_systemd: "{{ item.systemd | default(false) }}" - docker_platform_modify_image: "{{ item.modify_image | default(false) }}" - docker_platform_modify_image_buildpath: "{{ item.modify_image_buildpath | default(molecule_ephemeral_directory + '/build') }}" - docker_platform_privileged: "{{ item.privileged | default (false) }}" - docker_platform_hostvars: "{{ item.hostvars | default({}) }}" - docker_platform_state: present - when: item.type == 'docker' + platform_name: "{{ item.name }}" + platform_state: present + platform_type: "{{ item.type }}" + platform_molecule_cfg: "{{ item }}" loop: "{{ molecule_yml.platforms }}" loop_control: label: item.name diff --git a/roles/init/templates/prepare.yml.j2 b/roles/init/templates/prepare.yml.j2 index 80582fb..53fd233 100644 --- a/roles/init/templates/prepare.yml.j2 +++ b/roles/init/templates/prepare.yml.j2 @@ -66,7 +66,7 @@ state: directory owner: root group: root - mode: 0744 + mode: 0755 - name: Persistent data saved to local Ansible facts ansible.builtin.copy: diff --git a/roles/platform/README.md b/roles/platform/README.md new file mode 100644 index 0000000..225dd44 --- /dev/null +++ b/roles/platform/README.md @@ -0,0 +1,38 @@ +Role Name +========= + +A brief description of the role goes here. + +Requirements +------------ + +Any pre-requisites that may not be covered by Ansible itself or the role should be mentioned here. For instance, if the role uses the EC2 module, it may be a good idea to mention in this section that the boto package is required. + +Role Variables +-------------- + +A description of the settable variables for this role should go here, including any variables that are in defaults/main.yml, vars/main.yml, and any variables that can/should be set via parameters to the role. Any variables that are read from other roles and/or the global scope (ie. hostvars, group vars, etc.) should be mentioned here as well. + +Dependencies +------------ + +A list of other roles hosted on Galaxy should go here, plus any details in regards to parameters that may need to be set for other roles, or variables that are used from other roles. + +Example Playbook +---------------- + +Including an example of how to use your role (for instance, with variables passed in as parameters) is always nice for users too: + + - hosts: servers + roles: + - { role: username.rolename, x: 42 } + +License +------- + +BSD + +Author Information +------------------ + +An optional section for the role authors to include contact information, or a website (HTML is not allowed). diff --git a/roles/platform/defaults/main.yml b/roles/platform/defaults/main.yml new file mode 100644 index 0000000..70657a0 --- /dev/null +++ b/roles/platform/defaults/main.yml @@ -0,0 +1,15 @@ +--- +# defaults file for platform + +# Name of this Molecule platform +platform_name: instance + +# Whether this platform should be deployed on the current system (present/absent) +platform_state: present + +# What type of platform should be deployed +platform_type: docker + +# Molecule platform configuration +platform_molecule_cfg: {} + diff --git a/roles/platform/handlers/main.yml b/roles/platform/handlers/main.yml new file mode 100644 index 0000000..a68801b --- /dev/null +++ b/roles/platform/handlers/main.yml @@ -0,0 +1,2 @@ +--- +# handlers file for platform diff --git a/roles/platform/meta/main.yml b/roles/platform/meta/main.yml new file mode 100644 index 0000000..c572acc --- /dev/null +++ b/roles/platform/meta/main.yml @@ -0,0 +1,52 @@ +galaxy_info: + author: your name + description: your role description + company: your company (optional) + + # If the issue tracker for your role is not on github, uncomment the + # next line and provide a value + # issue_tracker_url: http://example.com/issue/tracker + + # Choose a valid license ID from https://spdx.org - some suggested licenses: + # - BSD-3-Clause (default) + # - MIT + # - GPL-2.0-or-later + # - GPL-3.0-only + # - Apache-2.0 + # - CC-BY-4.0 + license: license (GPL-2.0-or-later, MIT, etc) + + min_ansible_version: 2.1 + + # If this a Container Enabled role, provide the minimum Ansible Container version. + # min_ansible_container_version: + + # + # Provide a list of supported platforms, and for each platform a list of versions. + # If you don't wish to enumerate all versions for a particular platform, use 'all'. + # To view available platforms and versions (or releases), visit: + # https://galaxy.ansible.com/api/v1/platforms/ + # + # platforms: + # - name: Fedora + # versions: + # - all + # - 25 + # - name: SomePlatform + # versions: + # - all + # - 1.0 + # - 7 + # - 99.99 + + galaxy_tags: [] + # List tags for your role here, one per line. A tag is a keyword that describes + # and categorizes the role. Users find roles by searching for tags. Be sure to + # remove the '[]' above, if you add tags to this list. + # + # NOTE: A tag is limited to a single word comprised of alphanumeric characters. + # Maximum 20 tags per role. + +dependencies: [] + # List your role dependencies here, one per line. Be sure to remove the '[]' above, + # if you add dependencies to this list. diff --git a/roles/platform/tasks/inventory.yml b/roles/platform/tasks/inventory.yml new file mode 100644 index 0000000..88692d3 --- /dev/null +++ b/roles/platform/tasks/inventory.yml @@ -0,0 +1,74 @@ +--- +# Add a host to the Molecule inventory + +- name: Load existing instance configuration + when: __platform_run_count | int > 1 + block: + - name: Load existing instance configuration file + ansible.builtin.slurp: + src: "{{ platform_molecule_ephemeral_directory }}/instance_config.yml" + register: __platform_current_instance_config_b64 + ignore_errors: true + + - name: Decode instance configuration data + ansible.builtin.set_fact: + __platform_current_instance_config: "{{ __platform_current_instance_config_b64.content | default('') | b64decode | from_yaml }}" + +- name: Write {{ platform_name }} instance config file + ansible.builtin.copy: + # This is very basic - just needs an item there to show as managed with docker config + content: | + {% if __platform_current_instance_config is defined %} + {{ __platform_current_instance_config | to_yaml }} + {% endif %} + # TODO: update this from just docker for connection + - instance: {{ platform_name }} + connection: docker + dest: "{{ platform_molecule_ephemeral_directory }}/instance_config.yml" + mode: "0600" + +- name: Load existing molecule inventory + when: __platform_run_count | int > 1 + block: + - name: Load existing molecule inventory file + ansible.builtin.slurp: + src: "{{ platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" + register: __platform_current_molecule_inventory_b64 + ignore_errors: true + + - name: Decode instance configuration data + ansible.builtin.set_fact: + __platform_current_molecule_inventory: "{{ __platform_current_molecule_inventory_b64.content | default({}) | b64decode | from_yaml }}" + +- name: Add {{ platform_name }} to molecule_inventory + vars: + __platform_inventory_partial_hostvars: "{{ { + 'ansible_connection': 'community.docker.docker' + } | combine(platform_molecule_cfg.hostvars, recursive=true) }}" + __platform_inventory_partial_yaml: | + all: + children: + molecule: + hosts: + "{{ platform_name }}": {{ __platform_inventory_partial_hostvars }} + ansible.builtin.set_fact: + __platform_molecule_inventory: > + {{ __platform_current_molecule_inventory | from_yaml | default({}) | combine(__platform_inventory_partial_yaml | from_yaml, recursive=true) }} + +- name: Write {{ platform_name }} to molecule inventory file + ansible.builtin.copy: + content: | + {{ __platform_molecule_inventory | to_yaml }} + dest: "{{ platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" + mode: "0600" + +- name: Force inventory refresh + ansible.builtin.meta: refresh_inventory + +- name: Fail if molecule group is missing + ansible.builtin.assert: + that: "'molecule' in groups" + fail_msg: | + molecule group was not found inside inventory groups: {{ groups }} + + diff --git a/roles/platform/tasks/main.yml b/roles/platform/tasks/main.yml new file mode 100644 index 0000000..404984b --- /dev/null +++ b/roles/platform/tasks/main.yml @@ -0,0 +1,11 @@ +--- +# tasks file for platform + +- name: Platform is provisioned + ansible.builtin.include_tasks: "{{ role_path }}/tasks/provision.yml" + when: platform_state == 'present' + +- name: Platform is destroyed + ansible.builtin.include_tasks: "{ role_path }}/tasks/destroy.yml" + when: platform_state == 'absent' + diff --git a/roles/platform/tasks/provision.yml b/roles/platform/tasks/provision.yml new file mode 100644 index 0000000..aea6771 --- /dev/null +++ b/roles/platform/tasks/provision.yml @@ -0,0 +1,31 @@ +--- +# Create a host and requisite configuration for use by molecule +# + +- name: Initialize state + ansible.builtin.set_fact: + # Number of times this role has been included during this playbook run + __platform_run_count: "{{ __platform_run_count | default(0) | int + 1 }}" + +- name: Load system facts + ansible.builtin.setup: + filter: + - ansible_service_mgr + +- name: Configure platform for docker type + when: platform_type == 'docker' + ansible.builtin.include_role: + name: "{{ ansible_collection_name }}.docker_platform" + vars: + docker_platform_name: "{{ platform_name }}" + docker_platform_state: "{{ platform_state }}" + docker_platform_image: "{{ platform_molecule_cfg.image }}" + docker_platform_systemd: "{{ platform_molecule_cfg.systemd | default(false) }}" + docker_platform_modify_image: "{{ platform_molecule_cfg.modify_image | default(false) }}" + docker_platform_modify_image_buildpath: "{{ platform_molecule_cfg.modify_image_buildpath | default(molecule_ephemeral_directory + '/build') }}" + docker_platform_privileged: "{{ platform_molecule_cfg.privileged | default (false) }}" + docker_platform_hostvars: "{{ platform_molecule_cfg.hostvars | default({}) }}" + +- name: Configure Molecule inventory + ansible.builtin.include_tasks: "{{ role_path }}/tasks/inventory.yml" + diff --git a/roles/platform/tests/inventory b/roles/platform/tests/inventory new file mode 100644 index 0000000..878877b --- /dev/null +++ b/roles/platform/tests/inventory @@ -0,0 +1,2 @@ +localhost + diff --git a/roles/platform/tests/test.yml b/roles/platform/tests/test.yml new file mode 100644 index 0000000..360779d --- /dev/null +++ b/roles/platform/tests/test.yml @@ -0,0 +1,5 @@ +--- +- hosts: localhost + remote_user: root + roles: + - platform diff --git a/roles/platform/vars/main.yml b/roles/platform/vars/main.yml new file mode 100644 index 0000000..2cc9bf1 --- /dev/null +++ b/roles/platform/vars/main.yml @@ -0,0 +1,7 @@ +--- +# vars file for platform + +# Filesystem location of the Molecule ephemeral directory. Should not need to be updated by the user of this role! +platform_molecule_ephemeral_directory: "{{ molecule_ephemeral_directory }}" + + From 4e031f6f6c074e5991224e95a67081ac363e6d2d Mon Sep 17 00:00:00 2001 From: Vincent Date: Sun, 24 Mar 2024 16:33:48 -0600 Subject: [PATCH 06/18] Build latest tag on every commit to main (#7) --- .github/workflows/latest.yml | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/.github/workflows/latest.yml b/.github/workflows/latest.yml index c5604e9..27f8d78 100644 --- a/.github/workflows/latest.yml +++ b/.github/workflows/latest.yml @@ -1,14 +1,17 @@ --- name: Update `latest` tag -on: - release: - types: [published] +on: + push: + branches: + - main jobs: run: runs-on: ubuntu-latest - + permissions: + contents: read + packages: write steps: - name: Checkout repository uses: actions/checkout@v4 @@ -21,3 +24,4 @@ jobs: env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + From 509e22280216c36ff56d3507014397cd57b8f350 Mon Sep 17 00:00:00 2001 From: syndr Date: Sun, 24 Mar 2024 16:37:33 -0600 Subject: [PATCH 07/18] Add content permissions --- .github/workflows/latest.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/latest.yml b/.github/workflows/latest.yml index 27f8d78..82397a0 100644 --- a/.github/workflows/latest.yml +++ b/.github/workflows/latest.yml @@ -10,7 +10,7 @@ jobs: run: runs-on: ubuntu-latest permissions: - contents: read + contents: write packages: write steps: - name: Checkout repository From c42d2bc1dd2ca077dffb26240d58c16a647cf7d8 Mon Sep 17 00:00:00 2001 From: Vincent Date: Thu, 11 Apr 2024 20:14:54 -0600 Subject: [PATCH 08/18] Platform loader and ec2 support (#8) Refactor molecule.docker_platform role to handle specifically docker testing containers and not molecule inventory files Molecule inventory files are now managed by the new molecule.platform role. References to docker_platform in molecule create.yml or destroy.yml files should be updated to use this role in order to work with this version Add molecule.ec2_platform role, which allows creation and use of ephemeral ec2 instances for test environments. Update molecule.init role to support deployment of both docker and ec2 platforms. Note that only one platform type is supported per scenario! Also note that there are differences between the Molecule configuration for each platform, so this init role should be used to deploy the appropriate templates! --- README.md | 4 +- galaxy.yml | 2 +- molecule/default/collections.yml | 8 +- molecule/default/destroy.yml | 15 +- roles/docker_platform/README.md | 98 +++++----- roles/docker_platform/tasks/absent.yml | 24 +-- roles/docker_platform/tasks/present.yml | 32 ---- roles/ec2_platform/README.md | 133 ++++++++++++++ roles/ec2_platform/defaults/main.yml | 81 +++++++++ roles/ec2_platform/handlers/main.yml | 2 + roles/ec2_platform/meta/main.yml | 54 ++++++ .../molecule/role-ec2_platform/cleanup.yml | 13 ++ .../role-ec2_platform/collections.yml | 10 ++ .../molecule/role-ec2_platform/converge.yml | 52 ++++++ .../molecule/role-ec2_platform/create.yml | 58 ++++++ .../molecule/role-ec2_platform/destroy.yml | 18 ++ .../molecule/role-ec2_platform/init.yml | 10 ++ .../molecule/role-ec2_platform/molecule.yml | 71 ++++++++ .../molecule/role-ec2_platform/prepare.yml | 83 +++++++++ .../role-ec2_platform/requirements.yml | 4 + .../role-ec2_platform/side_effect.yml | 10 ++ .../molecule/role-ec2_platform/verify.yml | 21 +++ roles/ec2_platform/tasks/absent.yml | 76 ++++++++ roles/ec2_platform/tasks/main.yml | 26 +++ roles/ec2_platform/tasks/present.yml | 169 ++++++++++++++++++ roles/ec2_platform/vars/main.yml | 2 + roles/init/README.md | 20 ++- roles/init/defaults/main.yml | 20 ++- roles/init/files/init.yml | 5 + roles/init/tasks/asserts.yml | 3 + roles/init/tasks/main.yml | 70 ++++++++ roles/init/templates/collections.yml.j2 | 8 +- roles/init/templates/create.yml.j2 | 3 +- roles/init/templates/destroy.yml.j2 | 13 +- roles/init/templates/molecule.yml.j2 | 10 +- roles/init/templates/prepare.yml.j2 | 1 + roles/platform/README.md | 71 ++++++-- roles/platform/meta/main.yml | 12 +- roles/platform/tasks/deprovision.yml | 25 +++ roles/platform/tasks/inventory.yml | 152 +++++++++++++--- roles/platform/tasks/main.yml | 5 +- roles/platform/tasks/provision.yml | 12 +- roles/platform/vars/main.yml | 5 + 43 files changed, 1340 insertions(+), 171 deletions(-) create mode 100644 roles/ec2_platform/README.md create mode 100644 roles/ec2_platform/defaults/main.yml create mode 100644 roles/ec2_platform/handlers/main.yml create mode 100644 roles/ec2_platform/meta/main.yml create mode 100644 roles/ec2_platform/molecule/role-ec2_platform/cleanup.yml create mode 100644 roles/ec2_platform/molecule/role-ec2_platform/collections.yml create mode 100644 roles/ec2_platform/molecule/role-ec2_platform/converge.yml create mode 100644 roles/ec2_platform/molecule/role-ec2_platform/create.yml create mode 100644 roles/ec2_platform/molecule/role-ec2_platform/destroy.yml create mode 100644 roles/ec2_platform/molecule/role-ec2_platform/init.yml create mode 100644 roles/ec2_platform/molecule/role-ec2_platform/molecule.yml create mode 100644 roles/ec2_platform/molecule/role-ec2_platform/prepare.yml create mode 100644 roles/ec2_platform/molecule/role-ec2_platform/requirements.yml create mode 100644 roles/ec2_platform/molecule/role-ec2_platform/side_effect.yml create mode 100644 roles/ec2_platform/molecule/role-ec2_platform/verify.yml create mode 100644 roles/ec2_platform/tasks/absent.yml create mode 100644 roles/ec2_platform/tasks/main.yml create mode 100644 roles/ec2_platform/tasks/present.yml create mode 100644 roles/ec2_platform/vars/main.yml create mode 100644 roles/platform/tasks/deprovision.yml diff --git a/README.md b/README.md index 3bf9527..c044c5f 100644 --- a/README.md +++ b/README.md @@ -50,7 +50,9 @@ More tips on using Molecule can be found [below](#using-molecule). The following roles are provided: * [init](roles/init) - Initialize the Molecule testing framework for a project -* [docker_platform](roles/docker_platform) - Create a docker-based test platform for Molecule +* [platform](roles/platform) - Deploy a Molecule platform for testing +* [docker_platform](roles/docker_platform) - Used by the `platform` role to create a Docker-based test platform +* [ec2_platform](roles/ec2_platform) - Used by the `platform` role to create an EC2-based test platform * [prepare_controller](roles/prepare_controller) - Prepare a molecule controller to run local code tests The recommended way to use this collection is to provision Molecule scenarios using the [init role](roles/init). The `init` role provides template configurations that will work in various project types. diff --git a/galaxy.yml b/galaxy.yml index c5a9fd4..becb239 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -8,7 +8,7 @@ namespace: syndr name: molecule # The version of the collection. Must be compatible with semantic versioning -version: 1.4.0-dev +version: 1.4.0 # The path to the Markdown (.md) readme file. This path is relative to the root of the collection readme: README.md diff --git a/molecule/default/collections.yml b/molecule/default/collections.yml index 5dcfda8..6db7c73 100644 --- a/molecule/default/collections.yml +++ b/molecule/default/collections.yml @@ -3,9 +3,7 @@ collections: - name: community.general - name: community.docker - #- name: git+https://github.com/syndr/ansible-collection-molecule.git - # type: git - # version: latest - - name: syndr.molecule - version: 1.4.0-dev + - name: git+https://github.com/syndr/ansible-collection-molecule.git + type: git + version: latest diff --git a/molecule/default/destroy.yml b/molecule/default/destroy.yml index 88c729d..6569c07 100644 --- a/molecule/default/destroy.yml +++ b/molecule/default/destroy.yml @@ -1,13 +1,18 @@ --- - name: Perform cleanup - hosts: molecule + hosts: localhost gather_facts: false tasks: - - name: Remove platform + - name: Remove platform(s) ansible.builtin.include_role: - name: syndr.molecule.docker_platform + name: syndr.molecule.platform vars: - docker_platform_name: "{{ inventory_hostname }}" - docker_platform_state: absent + platform_name: "{{ item.name }}" + platform_state: absent + platform_type: "{{ item.type }}" + platform_molecule_cfg: "{{ item }}" + loop: "{{ molecule_yml.platforms }}" + loop_control: + label: item.name diff --git a/roles/docker_platform/README.md b/roles/docker_platform/README.md index 5d61f4b..750eddc 100644 --- a/roles/docker_platform/README.md +++ b/roles/docker_platform/README.md @@ -3,59 +3,34 @@ molecule.docker_platform Create a docker-based test platform for Molecule. -Requirements ------------- - -1. Molecule should be installed and executable from a location in the users PATH -1. Ansible should be installed, with `ansible-playbook` executable via the users PATH -1. Docker should be installed -1. The current user should be a member of the `docker` group - -Role Variables --------------- - -```yaml -# Name of this Molecule platform -docker_platform_name: instance - -# Whether this platform should be deployed on the current system (present/absent) -docker_platform_state: present +This role is intended to be used via the `molecule.platform` role that is included with this collection, and should not be referenced directly in a playbook. -# Docker image that this platform runs -docker_platform_image: "geerlingguy/docker-rockylinux9-ansible:latest" +Configuration is done via the `platforms` section of the `molecule.yml` file in your Molecule scenario directory. -# Should the provided image be modified at runtime -docker_platform_modify_image: false +Required configuration options are: -# Path to docker build files that should be used to modify the image. Files are treated as templates -# and can contain jinja2 templating language ("{{ my_var }}" etc.) -docker_platform_modify_image_buildpath: "{{ molecule_ephemeral_directory }}/build" +- `name`: Name of the platform (string) +- `type`: `docker` +- `image`: Docker image to use for the platform (string) -# Command to be executed at runtime on the container -# Leave as "" to use container default -docker_platform_command: "" +Optional configuration options are: -# Is this a SystemD enabled container? -docker_platform_systemd: true +- `systemd`: Whether the container should be started with SystemD enabled (boolean) +- `modify_image`: Whether the provided image should be modified at runtime (boolean) +- `modify_image_buildpath`: Path to Docker build files that should be used to modify the image (string) -# A list of Docker volumes that should be attached to the container -docker_platform_volumes: [] +Requirements +------------ -# Run the container in Privileged mode (greater host access, less security!) -docker_platform_privileged: false +1. Docker should be installed +1. The current user should be a member of the `docker` group -# A list of tmpfs filesystem paths to be passed to the container -docker_platform_tmpfs: [] -``` +Role Variables +-------------- -Configuration that should not require modification: -```yaml -# Filesystem location of the Molecule ephemeral directory. Should not need to be updated by the user of this role! -docker_platform_molecule_ephemeral_directory: "{{ molecule_ephemeral_directory }}" -``` +This role should not be used directly in a playbook, and should instead be used via the `molecule.platform` role. -Molecule variables expected: -- `molecule_ephemeral_directory` +Detailed information on configuration variables for this role can be found in [defaults/main.yml](defaults/main.yml). Dependencies ------------ @@ -66,36 +41,53 @@ Dependencies Example Playbook ---------------- +This role is intended to be used via the `molecule.platform` role that is included with this collection, and should not be referenced directly in a playbook. + +Configuration is done via the `platforms` section of the `molecule.yml` file in your Molecule scenario directory. + +```yaml +platforms: + - name: docker-rockylinux9 + type: docker + image: geerlingguy/docker-rockylinux9-ansible:latest + systemd: True + modify_image: False + privileged: False + hostvars: {} +``` + +To utilize this role, use the `platform` role that is included with this collection in your `create.yml` playbook! + ```yaml - name: Create hosts: localhost gather_facts: false tasks: - - name: Create platform + - name: Create platform(s) ansible.builtin.include_role: - name: syndr.molecule.docker_platform + name: syndr.molecule.platform vars: - docker_platform_name: "{{ item.name }}" - docker_platform_image: "{{ item.image }}" - docker_platform_systemd: true + platform_name: "{{ item.name }}" + platform_state: present + platform_type: "{{ item.type }}" + platform_molecule_cfg: "{{ item }}" loop: "{{ molecule_yml.platforms }}" loop_control: label: item.name -# we want to avoid errors like "Failed to create temporary directory" -- name: Validate molecule inventory +# We want to avoid errors like "Failed to create temporary directory" +- name: Validate that inventory was refreshed hosts: molecule gather_facts: false tasks: - - name: Check kernel version + - name: Check uname ansible.builtin.raw: uname -a register: result changed_when: false - - name: Display kernel info + - name: Display uname info ansible.builtin.debug: msg: "{{ result.stdout }}" - ``` diff --git a/roles/docker_platform/tasks/absent.yml b/roles/docker_platform/tasks/absent.yml index cf53114..d54f927 100644 --- a/roles/docker_platform/tasks/absent.yml +++ b/roles/docker_platform/tasks/absent.yml @@ -12,17 +12,17 @@ auto_remove: true # TODO: Remove just this host, not the whole inventory -- name: Remove dynamic molecule inventory - delegate_to: localhost - block: - - name: Remove dynamic inventory file - ansible.builtin.file: - path: "{{ docker_platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" - state: absent - - - name: Remove instance config file - ansible.builtin.file: - path: "{{ docker_platform_molecule_ephemeral_directory }}/instance_config.yml" - state: absent +#- name: Remove dynamic molecule inventory +# delegate_to: localhost +# block: +# - name: Remove dynamic inventory file +# ansible.builtin.file: +# path: "{{ docker_platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" +# state: absent +# +# - name: Remove instance config file +# ansible.builtin.file: +# path: "{{ docker_platform_molecule_ephemeral_directory }}/instance_config.yml" +# state: absent diff --git a/roles/docker_platform/tasks/present.yml b/roles/docker_platform/tasks/present.yml index 1dc97c3..c3d6d16 100644 --- a/roles/docker_platform/tasks/present.yml +++ b/roles/docker_platform/tasks/present.yml @@ -10,11 +10,6 @@ # Number of times this role has been included during this playbook run __docker_platform_run_count: "{{ __docker_platform_run_count | default(0) | int + 1 }}" -- name: Load system facts - ansible.builtin.setup: - filter: - - ansible_service_mgr - - name: Docker image needs to be customized block: - name: Check build path @@ -104,30 +99,3 @@ __docker_platform_create_result.container.State.ExitCode != 0 or not __docker_platform_create_result.container.State.Running -- name: "{{ docker_platform_name }} Systemd status is healthy" - block: - - name: System service manager is Systemd - ansible.builtin.assert: - that: - - "ansible_service_mgr == 'systemd'" - fail_msg: Systemd is enabled, but container service manager isn't Systemd! Is this a Systemd-enabled container? - - - name: Systemd has completed initialization - ansible.builtin.command: - cmd: systemctl is-system-running - register: __docker_platform_systemctl_status - until: > - 'running' in __docker_platform_systemctl_status.stdout or - 'degraded' in __docker_platform_systemctl_status.stdout - retries: 30 - delay: 5 - changed_when: false - failed_when: __docker_platform_systemctl_status.rc > 1 - - - name: "{{ docker_platform_name }} Systemd is healthy" - ansible.builtin.assert: - that: - - "'running' in __docker_platform_systemctl_status.stdout" - success_msg: Systemd is healthy - fail_msg: Systemd is unhealthy - when: docker_platform_systemd diff --git a/roles/ec2_platform/README.md b/roles/ec2_platform/README.md new file mode 100644 index 0000000..706abf9 --- /dev/null +++ b/roles/ec2_platform/README.md @@ -0,0 +1,133 @@ +molecule.ec2_platform +========= + +Create an Amazon EC2-based test platform for Molecule. + +This role is intended to be used via the `molecule.platform` role that is included with this collection, and should not be referenced directly in a playbook. + +Configuration is done via the `platforms` section of the `molecule.yml` file in your Molecule scenario directory. + +Required configuration options are: + +- `name`: The name of the platform (string) +- `type`: `ec2` +- `image`: The AMI ID to use for the instance (string) +- `region`: The AWS region to deploy the instance in (string) +- `vpc_id`: The VPC ID to deploy the instance in (string) +- `vpc_subnet_id`: The VPC subnet ID to deploy the instance in (string) + +Optional configuration options are: + +- `assign_public_ip`: Whether or not to assign a public IP to the instance (boolean) +- `aws_profile`: The AWS profile to use for authentication (string) +- `boot_wait_seconds`: The number of seconds to wait for the instance to boot (integer) +- `instance_type`: The instance type to use for the instance (string) +- `key_inject_method`: The method to use for injecting the SSH key into the instance ("cloud-init"/"ec2") +- `key_name`: The name of the SSH key pair to use for the instance (string) +- `private_key_path`: The path to the private key file for the SSH key pair (string) +- `public_key_path`: The path to the public key file for the SSH key pair (string) +- `security_group_name`: The name of the security group to use for the instance (string) +- `security_group_description`: The description of the security group to use for the instance (string) +- `security_group_rules`: A list of security group rules to apply to the instance (list of dicts) +- `security_group_rules_egress`: A list of security group egress rules to apply to the instance (list of dicts) +- `ssh_user`: The SSH user to use for connecting to the instance (string) +- `ssh_port`: The SSH port to use for connecting to the instance (integer) +- `cloud_config`: The cloud-config data to use for the instance (dictionary) +- `image_name`: The name of the image to use for the instance (string) +- `image_owner`: The owner of the image to use for the instance (string) +- `security_groups`: A list of security group names to apply to the instance (list of strings) +- `tags`: A dictionary of tags to apply to the instance (dictionary) +- `volumes`: A list of volumes to attach to the instance (list of dicts) + +Requirements +------------ + +**Python Modules** +- `boto3` + +Role Variables +-------------- + +In order to connect to AWS, you will need the following environment variables to be set: + +```bash + + export AWS_ACCESS_KEY_ID="blahblahblahblah" + export AWS_SECRET_ACCESS_KEY="hurpderpherpderpdpeypedpderpyderp" + export AWS_SESSION_TOKEN="hurpderpherpderpdpeypedpderpyderpahahwhizbanglotsofstuffblablablabla" +``` + +The `AWS_SESSION_TOKEN` variable is only required if you are using temporary credentials. + +Full role configuration options are available in the [defaults/main.yml](defaults/main.yml) file. + +Dependencies +------------ + +**Collections** +- `amazon.aws` + +Example Playbook +---------------- + +This role is intended to be used via the `molecule.platform` role that is included with this collection, and should not be referenced directly in a playbook. + +Configuration is done via the `platforms` section of the `molecule.yml` file in your Molecule scenario directory. + +```yaml +platforms: + - name: ec2-rockylinux9 + type: ec2 + image: "ami-067daee80a6d36ac0" + instance_type: "t3.micro" + region: "us-east-2" + vpc_id: "vpc-12345678" + vpc_subnet_id: "subnet-12345678" +``` + +To utilize this role, use the `platform` role that is included with this collection in your `create.yml` playbook! + +```yaml +- name: Create + hosts: localhost + gather_facts: false + tasks: + - name: Create platform(s) + ansible.builtin.include_role: + name: syndr.molecule.platform + vars: + platform_name: "{{ item.name }}" + platform_state: present + platform_type: "{{ item.type }}" + platform_molecule_cfg: "{{ item }}" + loop: "{{ molecule_yml.platforms }}" + loop_control: + label: item.name + +# We want to avoid errors like "Failed to create temporary directory" +- name: Validate that inventory was refreshed + hosts: molecule + gather_facts: false + tasks: + - name: Check uname + ansible.builtin.raw: uname -a + register: result + changed_when: false + + - name: Display uname info + ansible.builtin.debug: + msg: "{{ result.stdout }}" + +``` + + +License +------- + +MIT + +Author Information +------------------ + +- [@syndr](https://github.com/syndr/) + diff --git a/roles/ec2_platform/defaults/main.yml b/roles/ec2_platform/defaults/main.yml new file mode 100644 index 0000000..6d6f39d --- /dev/null +++ b/roles/ec2_platform/defaults/main.yml @@ -0,0 +1,81 @@ +--- +# defaults file for ec2_platform + +# Name of this Molecule platform +ec2_platform_name: instance + +# Run config handling +ec2_platform_default_run_id: "{{ lookup('password', '/dev/null chars=ascii_lowercase length=5') }}" +ec2_platform_default_run_config: + run_id: "{{ ec2_platform_default_run_id }}" + +ec2_platform_run_config_path: "{{ lookup('env', 'MOLECULE_EPHEMERAL_DIRECTORY') }}/ec2-platform-run-config.yml" +ec2_platform_run_config_from_file: "{{ (lookup('file', ec2_platform_run_config_path, errors='ignore') or '{}') | from_yaml }}" +ec2_platform_run_config: '{{ ec2_platform_default_run_config | combine(ec2_platform_run_config_from_file) }}' + +# Platform settings handling +ec2_platform_default_assign_public_ip: true +ec2_platform_default_aws_profile: "{{ lookup('env', 'AWS_PROFILE') }}" +ec2_platform_default_boot_wait_seconds: 120 +ec2_platform_default_instance_type: t3a.medium +ec2_platform_default_key_inject_method: cloud-init # valid values: [cloud-init, ec2] +ec2_platform_default_key_name: "molecule-{{ ec2_platform_run_config.run_id }}" +ec2_platform_default_private_key_path: "{{ lookup('env', 'MOLECULE_EPHEMERAL_DIRECTORY') }}/id_rsa" +ec2_platform_default_public_key_path: "{{ ec2_platform_default_private_key_path }}.pub" +ec2_platform_default_ssh_user: ansible +ec2_platform_default_ssh_port: 22 +ec2_platform_default_user_data: '' + +ec2_platform_default_security_group_name: "molecule-{{ ec2_platform_run_config.run_id }}" +ec2_platform_default_security_group_description: Ephemeral security group for Molecule instances +ec2_platform_default_security_group_rules: + - proto: tcp + from_port: "{{ ec2_platform_default_ssh_port }}" + to_port: "{{ ec2_platform_default_ssh_port }}" + cidr_ip: "0.0.0.0/0" + - proto: icmp + from_port: 8 + to_port: -1 + cidr_ip: "0.0.0.0/0" +ec2_platform_default_security_group_rules_egress: + - proto: -1 + from_port: 0 + to_port: 0 + cidr_ip: "0.0.0.0/0" + +ec2_platform_defaults: + assign_public_ip: "{{ ec2_platform_default_assign_public_ip }}" + aws_profile: "{{ ec2_platform_default_aws_profile }}" + boot_wait_seconds: "{{ ec2_platform_default_boot_wait_seconds }}" + instance_type: "{{ ec2_platform_default_instance_type }}" + key_inject_method: "{{ ec2_platform_default_key_inject_method }}" + key_name: "{{ ec2_platform_default_key_name }}" + private_key_path: "{{ ec2_platform_default_private_key_path }}" + public_key_path: "{{ ec2_platform_default_public_key_path }}" + security_group_name: "{{ ec2_platform_default_security_group_name }}" + security_group_description: "{{ ec2_platform_default_security_group_description }}" + security_group_rules: "{{ ec2_platform_default_security_group_rules }}" + security_group_rules_egress: "{{ ec2_platform_default_security_group_rules_egress }}" + ssh_user: "{{ ec2_platform_default_ssh_user }}" + ssh_port: "{{ ec2_platform_default_ssh_port }}" + cloud_config: {} + image: "" + image_name: "" + image_owner: [self] + name: "" + region: "" + security_groups: [] + tags: {} + volumes: [] + vpc_id: "" + vpc_subnet_id: "" + +# Merging defaults into a list of dicts is, it turns out, not straightforward +#ec2_platforms: >- +# {{ [ec2_platform_defaults | dict2items] +# | product(molecule_yml.platforms | map('dict2items') | list) +# | map('flatten', levels=1) +# | list +# | map('items2dict') +# | list }} + diff --git a/roles/ec2_platform/handlers/main.yml b/roles/ec2_platform/handlers/main.yml new file mode 100644 index 0000000..67429cd --- /dev/null +++ b/roles/ec2_platform/handlers/main.yml @@ -0,0 +1,2 @@ +--- +# handlers file for ec2_platform diff --git a/roles/ec2_platform/meta/main.yml b/roles/ec2_platform/meta/main.yml new file mode 100644 index 0000000..de14aaf --- /dev/null +++ b/roles/ec2_platform/meta/main.yml @@ -0,0 +1,54 @@ +galaxy_info: + role_name: ec2_platform + namespace: syndr + author: syndr + description: Provision and deprovision an EC2-based Molecule platform using Ansible. + company: UltronCORE + + # If the issue tracker for your role is not on github, uncomment the + # next line and provide a value + # issue_tracker_url: http://example.com/issue/tracker + + # Choose a valid license ID from https://spdx.org - some suggested licenses: + # - BSD-3-Clause (default) + # - MIT + # - GPL-2.0-or-later + # - GPL-3.0-only + # - Apache-2.0 + # - CC-BY-4.0 + license: MIT + + min_ansible_version: 2.16 + + # If this a Container Enabled role, provide the minimum Ansible Container version. + # min_ansible_container_version: + + # + # Provide a list of supported platforms, and for each platform a list of versions. + # If you don't wish to enumerate all versions for a particular platform, use 'all'. + # To view available platforms and versions (or releases), visit: + # https://galaxy.ansible.com/api/v1/platforms/ + # + # platforms: + # - name: Fedora + # versions: + # - all + # - 25 + # - name: SomePlatform + # versions: + # - all + # - 1.0 + # - 7 + # - 99.99 + + galaxy_tags: [] + # List tags for your role here, one per line. A tag is a keyword that describes + # and categorizes the role. Users find roles by searching for tags. Be sure to + # remove the '[]' above, if you add tags to this list. + # + # NOTE: A tag is limited to a single word comprised of alphanumeric characters. + # Maximum 20 tags per role. + +dependencies: [] + # List your role dependencies here, one per line. Be sure to remove the '[]' above, + # if you add dependencies to this list. diff --git a/roles/ec2_platform/molecule/role-ec2_platform/cleanup.yml b/roles/ec2_platform/molecule/role-ec2_platform/cleanup.yml new file mode 100644 index 0000000..26e7fa5 --- /dev/null +++ b/roles/ec2_platform/molecule/role-ec2_platform/cleanup.yml @@ -0,0 +1,13 @@ +--- +# The cleanup.yml playbook should be used to remove any test infrastructure that was created by this test process +# and is not present within the instance itself (IE: the docker container created by Molecule). For example, it +# could be used to remove AWS infrastructure created as part of this test and that should not persist. + +- name: Remove external test infrastructure + hosts: molecule + tasks: + - name: Cleanup tasks not configured + delegate_to: localhost + ansible.builtin.debug: + msg: Add your cleanup tasks here as required! + diff --git a/roles/ec2_platform/molecule/role-ec2_platform/collections.yml b/roles/ec2_platform/molecule/role-ec2_platform/collections.yml new file mode 100644 index 0000000..4fdc1bc --- /dev/null +++ b/roles/ec2_platform/molecule/role-ec2_platform/collections.yml @@ -0,0 +1,10 @@ +--- + +collections: + - name: community.docker +# - name: git+https://github.com/syndr/ansible-collection-molecule.git +# type: git +# version: latest + - name: syndr.molecule + version: 1.4.0-dev + diff --git a/roles/ec2_platform/molecule/role-ec2_platform/converge.yml b/roles/ec2_platform/molecule/role-ec2_platform/converge.yml new file mode 100644 index 0000000..f904f40 --- /dev/null +++ b/roles/ec2_platform/molecule/role-ec2_platform/converge.yml @@ -0,0 +1,52 @@ +--- +# Verify that the target code runs successfullly. +# Note that this playbook (converge.yml) must be idempotent! + +# Check that the molecule inventory is correctly configured +- name: Fail if molecule group is missing + hosts: localhost + tasks: + - name: Print host inventory groups + ansible.builtin.debug: + msg: "{{ groups }}" + + - name: Assert group existence + ansible.builtin.assert: + that: "'molecule' in groups" + fail_msg: | + molecule group was not found inside inventory groups: {{ groups }} + +- name: Converge + hosts: molecule + tasks: + - name: Check uname + ansible.builtin.raw: uname -a + register: result + changed_when: false + + - name: Verify kernel type + ansible.builtin.assert: + that: result.stdout | regex_search("^Linux") + + - name: Do preparation + block: + - name: Load local host facts + ansible.builtin.setup: + gather_subset: + - '!all' + - '!min' + - local + + - name: Show local Ansible facts + ansible.builtin.debug: + var: ansible_facts.ansible_local + verbosity: 1 + + - name: Load preparation facts + ansible.builtin.set_fact: + test_prepare_fact: "{{ ansible_local.molecule.test_prepare_fact }}" + + - name: Add your project test configuration here + ansible.builtin.debug: + msg: Typically this will be via the ansible.builtin.include_role module or via import_playbook + diff --git a/roles/ec2_platform/molecule/role-ec2_platform/create.yml b/roles/ec2_platform/molecule/role-ec2_platform/create.yml new file mode 100644 index 0000000..2815558 --- /dev/null +++ b/roles/ec2_platform/molecule/role-ec2_platform/create.yml @@ -0,0 +1,58 @@ +--- +- name: Create + hosts: localhost + gather_facts: false + tasks: + - name: Create platform(s) + ansible.builtin.include_role: + name: syndr.molecule.platform + vars: + + platform_name: "{{ item.name }}" + platform_state: present + platform_type: "{{ item.type }}" + platform_molecule_cfg: "{{ item }}" + loop: "{{ molecule_yml.platforms }}" + loop_control: + label: item.name + +# We want to avoid errors like "Failed to create temporary directory" +- name: Validate that inventory was refreshed + hosts: molecule + gather_facts: false + tasks: + - name: Check uname + ansible.builtin.raw: uname -a + register: result + changed_when: false + + - name: Display uname info + ansible.builtin.debug: + msg: "{{ result.stdout }}" + + - name: Load system facts + ansible.builtin.setup: + filter: + - ansible_service_mgr + + - name: Check on Systemd + block: + - name: Wait for systemd to complete initialization. + ansible.builtin.command: systemctl is-system-running + register: systemctl_status + until: > + 'running' in systemctl_status.stdout or + 'degraded' in systemctl_status.stdout + retries: 30 + delay: 5 + changed_when: false + failed_when: systemctl_status.rc > 1 + + - name: Check systemd status + ansible.builtin.assert: + that: + - systemctl_status.stdout == 'running' + fail_msg: Systemd-enabled container does not have a healthy Systemd! + success_msg: Systemd is running + when: ansible_service_mgr == 'systemd' + diff --git a/roles/ec2_platform/molecule/role-ec2_platform/destroy.yml b/roles/ec2_platform/molecule/role-ec2_platform/destroy.yml new file mode 100644 index 0000000..6569c07 --- /dev/null +++ b/roles/ec2_platform/molecule/role-ec2_platform/destroy.yml @@ -0,0 +1,18 @@ +--- + +- name: Perform cleanup + hosts: localhost + gather_facts: false + tasks: + - name: Remove platform(s) + ansible.builtin.include_role: + name: syndr.molecule.platform + vars: + platform_name: "{{ item.name }}" + platform_state: absent + platform_type: "{{ item.type }}" + platform_molecule_cfg: "{{ item }}" + loop: "{{ molecule_yml.platforms }}" + loop_control: + label: item.name + diff --git a/roles/ec2_platform/molecule/role-ec2_platform/init.yml b/roles/ec2_platform/molecule/role-ec2_platform/init.yml new file mode 100644 index 0000000..1da5128 --- /dev/null +++ b/roles/ec2_platform/molecule/role-ec2_platform/init.yml @@ -0,0 +1,10 @@ +--- +# Initialize a Molecule scenario for use within a role + +- name: Provision file structure + hosts: localhost + tasks: + - name: Launch provisioner + ansible.builtin.include_role: + name: syndr.molecule.init + diff --git a/roles/ec2_platform/molecule/role-ec2_platform/molecule.yml b/roles/ec2_platform/molecule/role-ec2_platform/molecule.yml new file mode 100644 index 0000000..fd0f925 --- /dev/null +++ b/roles/ec2_platform/molecule/role-ec2_platform/molecule.yml @@ -0,0 +1,71 @@ +--- +role_name_check: 0 +dependency: + name: galaxy +driver: + name: default + options: + managed: true +platforms: + - name: ec2-rockylinux9 + type: ec2 + image: ami-067daee80a6d36ac0 + region: us-east-2 + vpc_id: vpc-0eb9fd1391f4207ec + vpc_subnet_id: subnet-0aa189c0d6fc53923 + instance_type: t3.micro + hostvars: {} +provisioner: + name: ansible + log: True + playbooks: + prepare: prepare.yml + converge: converge.yml + side_effect: side_effect.yml + verify: verify.yml + cleanup: cleanup.yml + config_options: + defaults: + gathering: explicit + playbook_vars_root: top + verbosity: ${ANSIBLE_VERBOSITY:-0} +scenario: + create_sequence: + - dependency + - create + - prepare + check_sequence: + - dependency + - cleanup + - destroy + - create + - prepare + - converge + - check + - destroy + converge_sequence: + - dependency + - create + - prepare + - converge + destroy_sequence: + - dependency + - cleanup + - destroy + test_sequence: + - dependency + - cleanup + - destroy + - syntax + - create + - prepare + - converge + - idempotence + - side_effect + - verify + - cleanup + - destroy +verifier: + name: ansible + enabled: true + diff --git a/roles/ec2_platform/molecule/role-ec2_platform/prepare.yml b/roles/ec2_platform/molecule/role-ec2_platform/prepare.yml new file mode 100644 index 0000000..6d66d37 --- /dev/null +++ b/roles/ec2_platform/molecule/role-ec2_platform/prepare.yml @@ -0,0 +1,83 @@ +--- + +- name: Prepare controller for execution + hosts: localhost + tags: always + tasks: + - name: Configure for standalone role testing + ansible.builtin.include_role: + name: syndr.molecule.prepare_controller + vars: + prepare_controller_project_type: role + +- name: Prepare target host for execution + hosts: molecule + tags: always + become: true + tasks: + ## + # Creating an admin service account for Molecule/Ansible to use for testing + # + # - If you run Ansible as a service account (you should) on your hosts and + # not as root, it is wise to also test as a non-root user! + # + # - To use this account, add the following to any plays targeting test + # infrastructure (such as in converge.yml): + # + # vars: + # ansible_user: molecule_runner + ## + + - name: Create ansible service account + vars: + molecule_user: molecule_runner + block: + - name: Create ansible group + ansible.builtin.group: + name: "{{ molecule_user }}" + + - name: Create ansible user + ansible.builtin.user: + name: "{{ molecule_user }}" + group: "{{ molecule_user }}" + + - name: Sudoers.d directory exists + ansible.builtin.file: + path: /etc/sudoers.d + state: directory + owner: root + group: root + mode: 0751 + + - name: Ansible user has sudo + ansible.builtin.copy: + content: | + {{ molecule_user }} ALL=(ALL) NOPASSWD: ALL + dest: /etc/sudoers.d/ansible + owner: root + group: root + mode: 0600 + + - name: "Save vars to host (IE: generated test credentials, etc.)" + become: true + block: + - name: Ansible facts directory exists + ansible.builtin.file: + path: /etc/ansible/facts.d + state: directory + owner: root + group: root + mode: 0755 + + - name: Persistent data saved to local Ansible facts + ansible.builtin.copy: + dest: /etc/ansible/facts.d/molecule.fact + content: "{{ {'test_prepare_fact': 'this is an example!'} | to_json }}" + owner: root + group: root + mode: 0644 + + - name: Add your host preparation tasks here! + ansible.builtin.debug: + msg: "IE: adding system users, installing required packages, etc." + diff --git a/roles/ec2_platform/molecule/role-ec2_platform/requirements.yml b/roles/ec2_platform/molecule/role-ec2_platform/requirements.yml new file mode 100644 index 0000000..39b222d --- /dev/null +++ b/roles/ec2_platform/molecule/role-ec2_platform/requirements.yml @@ -0,0 +1,4 @@ +--- + +roles: [] + diff --git a/roles/ec2_platform/molecule/role-ec2_platform/side_effect.yml b/roles/ec2_platform/molecule/role-ec2_platform/side_effect.yml new file mode 100644 index 0000000..4d1d7af --- /dev/null +++ b/roles/ec2_platform/molecule/role-ec2_platform/side_effect.yml @@ -0,0 +1,10 @@ +--- +# The side effect playbook executes actions which produce side effects to the instances(s). Intended to test HA failover scenarios or the like. + +- name: Test side effects + hosts: molecule + tasks: + - name: No side effect tests configured + ansible.builtin.debug: + msg: Add side-effect tests here! + diff --git a/roles/ec2_platform/molecule/role-ec2_platform/verify.yml b/roles/ec2_platform/molecule/role-ec2_platform/verify.yml new file mode 100644 index 0000000..3237d2c --- /dev/null +++ b/roles/ec2_platform/molecule/role-ec2_platform/verify.yml @@ -0,0 +1,21 @@ +--- +# Verify that the role being tested has done what it's supposed to + +- name: Verify + hosts: molecule + tasks: + - name: Load local host facts + ansible.builtin.setup: + gather_subset: + - '!all' + - '!min' + - local + + - name: Load test data (example) + ansible.builtin.set_fact: + test_prepare_fact: "{{ ansible_local.molecule.test_prepare_fact }}" + + - name: Add your verification tasks here + ansible.builtin.debug: + msg: "IE: For a 'users' role, check that the test user exists" + diff --git a/roles/ec2_platform/tasks/absent.yml b/roles/ec2_platform/tasks/absent.yml new file mode 100644 index 0000000..d27d841 --- /dev/null +++ b/roles/ec2_platform/tasks/absent.yml @@ -0,0 +1,76 @@ +--- +# Remove ec2 test resources + +- name: Load Molecule instance config + ansible.builtin.set_fact: + __ec2_instance_molecule_instance_config: "{{ (lookup('file', molecule_instance_config, errors='ignore') or '{}') | from_yaml }}" + +- name: Validate platform configurations + ansible.builtin.assert: + that: + - ec2_platform is mapping + - ec2_platform.name is string and ec2_platform.name | length > 0 + - ec2_platform.aws_profile is string + - ec2_platform.key_inject_method is in ["cloud-init", "ec2"] + - ec2_platform.key_name is string and ec2_platform.key_name | length > 0 + - ec2_platform.region is string + - ec2_platform.security_group_name is string and ec2_platform.security_group_name | length > 0 + - ec2_platform.security_groups is sequence + - ec2_platform.vpc_id is string + - ec2_platform.vpc_subnet_id is string and ec2_platform.vpc_subnet_id | length > 0 + quiet: true + +- name: Look up subnets to determine VPCs (if needed) + amazon.aws.ec2_vpc_subnet_info: + profile: "{{ ec2_platform.aws_profile | default(omit) }}" + region: "{{ ec2_platform.region | default(omit) }}" + subnet_ids: "{{ ec2_platform.vpc_subnet_id }}" + when: not ec2_platform.vpc_id + register: __ec2_platform_subnet_info + +- name: Validate discovered information + ansible.builtin.assert: + that: ec2_platform.vpc_id or (__ec2_platform_subnet_info.subnets | length > 0) + quiet: true + fail_msg: "No VPCs found for subnet: {{ ec2_platform.vpc_subnet_id }}" + +- name: Look up EC2 instance by tag + amazon.aws.ec2_instance_info: + profile: "{{ ec2_platform.aws_profile | default(omit) }}" + region: "{{ ec2_platform.region | default(omit) }}" + filters: + "tag:molecule-run-id": "{{ ec2_platform_run_config.run_id }}" + register: __ec2_instance_info + +- name: Destroy ephemeral EC2 instances + when: __ec2_instance_info.instances | length > 0 + amazon.aws.ec2_instance: + profile: "{{ ec2_platform.aws_profile | default(omit) }}" + region: "{{ ec2_platform.region | default(omit) }}" + instance_ids: "{{ __ec2_instance_info.instances | map(attribute='instance_id') | list }}" + vpc_subnet_id: "{{ ec2_platform.vpc_subnet_id }}" + state: absent + register: __ec2_instance_destroy + +- name: Destroy ephemeral security groups (if needed) + amazon.aws.ec2_security_group: + profile: "{{ ec2_platform.aws_profile | default(omit) }}" + region: "{{ ec2_platform.region | default(omit) }}" + vpc_id: "{{ ec2_platform.vpc_id or __ec2_platform_subnet_info.subnets[0] }}" + name: "{{ ec2_platform.security_group_name }}" + state: absent + when: ec2_platform.security_groups | length == 0 + +- name: Destroy ephemeral keys (if needed) + amazon.aws.ec2_key: + profile: "{{ ec2_platform.aws_profile | default(omit) }}" + region: "{{ ec2_platform.region | default(omit) }}" + name: "{{ ec2_platform.key_name }}" + state: absent + when: ec2_platform.key_inject_method == "ec2" + +- name: Remove ec2 instance config file + ansible.builtin.file: + path: "{{ ec2_platform_run_config_path }}" + state: absent + diff --git a/roles/ec2_platform/tasks/main.yml b/roles/ec2_platform/tasks/main.yml new file mode 100644 index 0000000..02eaffe --- /dev/null +++ b/roles/ec2_platform/tasks/main.yml @@ -0,0 +1,26 @@ +--- +# tasks file for ec2_platform + +- name: 🐞 Show ec2_platform_definition + ansible.builtin.debug: + var: ec2_platform_definition + verbosity: 1 + +# Merge the defaults with any options provided to this role +- name: Generate runtime configuration + ansible.builtin.set_fact: + ec2_platform: "{{ ec2_platform_defaults | combine(ec2_platform_definition | default({})) }}" + +- name: 🦋 Show ec2_platform + ansible.builtin.debug: + var: ec2_platform + verbosity: 1 + +- name: Platform is deployed + ansible.builtin.include_tasks: "{{ role_path }}/tasks/present.yml" + when: ec2_platform_state == 'present' + +- name: Platform is not deployed + ansible.builtin.include_tasks: "{{ role_path }}/tasks/absent.yml" + when: ec2_platform_state == 'absent' + diff --git a/roles/ec2_platform/tasks/present.yml b/roles/ec2_platform/tasks/present.yml new file mode 100644 index 0000000..02a0212 --- /dev/null +++ b/roles/ec2_platform/tasks/present.yml @@ -0,0 +1,169 @@ +--- +# The ec2 platform has been created + +- name: Validate platform configuration - {{ ec2_platform.name | default('invalid') }} + ansible.builtin.assert: + that: + - ec2_platform is mapping + - ec2_platform.name is string and ec2_platform.name | length > 0 + - ec2_platform.assign_public_ip is boolean + - ec2_platform.aws_profile is string + - ec2_platform.boot_wait_seconds is integer and ec2_platform.boot_wait_seconds >= 0 + - ec2_platform.cloud_config is mapping + - ec2_platform.image is string + - ec2_platform.image_name is string + - ec2_platform.image_owner is sequence or (ec2_platform.image_owner is string and ec2_platform.image_owner | length > 0) + - ec2_platform.instance_type is string and ec2_platform.instance_type | length > 0 + - ec2_platform.key_inject_method is in ["cloud-init", "ec2"] + - ec2_platform.key_name is string and ec2_platform.key_name | length > 0 + - ec2_platform.private_key_path is string and ec2_platform.private_key_path | length > 0 + - ec2_platform.public_key_path is string and ec2_platform.public_key_path | length > 0 + - ec2_platform.region is string + - ec2_platform.security_group_name is string and ec2_platform.security_group_name | length > 0 + - ec2_platform.security_group_description is string and ec2_platform.security_group_description | length > 0 + - ec2_platform.security_group_rules is sequence + - ec2_platform.security_group_rules_egress is sequence + - ec2_platform.security_groups is sequence + - ec2_platform.ssh_user is string and ec2_platform.ssh_user | length > 0 + - ec2_platform.ssh_port is integer and ec2_platform.ssh_port in range(1, 65536) + - ec2_platform.tags is mapping + - ec2_platform.volumes is sequence + - ec2_platform.vpc_id is string + - ec2_platform.vpc_subnet_id is string and ec2_platform.vpc_subnet_id | length > 0 + quiet: true + +# TODO: Merge, not overwrite -- already does? +- name: Write run config to file + ansible.builtin.copy: + dest: "{{ ec2_platform_run_config_path }}" + content: "{{ ec2_platform_run_config | to_yaml }}" + mode: "0600" + +- name: Generate local key pairs + community.crypto.openssh_keypair: + path: "{{ ec2_platform.private_key_path }}" + type: rsa + size: 2048 + regenerate: never + register: __ec2_platform_local_keypair + +- name: Look up EC2 AMI(s) by owner and name (if image not set) + amazon.aws.ec2_ami_info: + owners: "{{ ec2_platform.image_owner }}" + filters: "{{ ec2_platform.image_filters | default({}) | combine(__ec2_platform_image_name_map) }}" + vars: + __ec2_platform_image_name_map: >- + "{% if ec2_platform.image_name is defined and ec2_platform.image_name | length > 0 %} + {{ {'name': ec2_platform.image_name} }} + {% else %}{}{% endif %}" + when: not ec2_platform.image + register: __ec2_platform_ami_info + +- name: Look up subnets to determine VPCs (if needed) + amazon.aws.ec2_vpc_subnet_info: + subnet_ids: "{{ ec2_platform.vpc_subnet_id }}" + when: not ec2_platform.vpc_id + register: __ec2_platform_subnet_info + +- name: Validate discovered information + ansible.builtin.assert: + that: + - ec2_platform.image or (__ec2_platform_ami_info.results[0].images | length > 0) + - ec2_platform.vpc_id or (__ec2_platform_subnet_info.results[0].subnets | length > 0) + quiet: true + +- name: Create ephemeral EC2 keys (if needed) + amazon.aws.ec2_key: + profile: "{{ ec2_platform.aws_profile | default(omit) }}" + region: "{{ ec2_platform.region | default(omit) }}" + name: "{{ ec2_platform.key_name }}" + key_material: "{{ __ec2_platform_local_keypair.public_key }}" + when: ec2_platform.key_inject_method == "ec2" + register: __ec2_platform_ec2_keys + +- name: Create ephemeral security groups (if needed) + amazon.aws.ec2_security_group: + profile: "{{ ec2_platform.aws_profile | default(omit) }}" + iam_instance_profile: "{{ ec2_platform.iam_instance_profile | default(omit) }}" + region: "{{ ec2_platform.region | default(omit) }}" + vpc_id: "{{ ec2_platform.vpc_id or vpc_subnet.vpc_id }}" + name: "{{ ec2_platform.security_group_name }}" + description: "{{ ec2_platform.security_group_description }}" + rules: "{{ ec2_platform.security_group_rules }}" + rules_egress: "{{ ec2_platform.security_group_rules_egress }}" + vars: + vpc_subnet: "{{ __ec2_platform_subnet_info.results[0].subnets[0] }}" + when: ec2_platform.security_groups | length == 0 + +- name: Create ephemeral EC2 instance + amazon.aws.ec2_instance: + profile: "{{ ec2_platform.aws_profile | default(omit) }}" + region: "{{ ec2_platform.region | default(omit) }}" + filters: "{{ __ec2_platform_filters }}" + instance_type: "{{ ec2_platform.instance_type }}" + image_id: "{{ __ec2_platform_image_id }}" + vpc_subnet_id: "{{ ec2_platform.vpc_subnet_id }}" + security_groups: "{{ __ec2_platform_security_groups }}" + network: + assign_public_ip: "{{ ec2_platform.assign_public_ip }}" + volumes: "{{ ec2_platform.volumes }}" + key_name: "{{ (ec2_platform.key_inject_method == 'ec2') | ternary(ec2_platform.key_name, omit) }}" + tags: "{{ __ec2_platform_tags }}" + user_data: "{{ __ec2_platform_user_data }}" + state: "running" + wait: true + vars: + __ec2_platform_security_groups: "{{ ec2_platform.security_groups or [ec2_platform.security_group_name] }}" + __ec2_platform_generated_image_id: "{{ (ami_info.results[0].images | sort(attribute='creation_date', reverse=True))[0].image_id }}" + __ec2_platform_image_id: "{{ ec2_platform.image or __ec2_platform_generated_image_id }}" + + __ec2_platform_generated_cloud_config: + users: + - name: "{{ ec2_platform.ssh_user }}" + ssh_authorized_keys: + - "{{ __ec2_platform_local_keypair.public_key }}" + sudo: "ALL=(ALL) NOPASSWD:ALL" + __ec2_platform_cloud_config: >- + {{ (ec2_platform.key_inject_method == 'cloud-init') + | ternary((ec2_platform.cloud_config | combine(__ec2_platform_generated_cloud_config)), ec2_platform.cloud_config) }} + __ec2_platform_user_data: |- + #cloud-config + {{ __ec2_platform_cloud_config | to_yaml }} + + __ec2_platform_generated_tags: + instance: "{{ ec2_platform.name }}" + "molecule-run-id": "{{ ec2_platform_run_config.run_id }}" + Name: molecule-{{ ec2_platform.name }} + __ec2_platform_tags: "{{ (ec2_platform.tags or {}) | combine(__ec2_platform_generated_tags) }}" + __ec2_platform_filter_keys: "{{ __ec2_platform_generated_tags.keys() | map('regex_replace', '^(.+)$', 'tag:\\1') }}" + __ec2_platform_filters: "{{ dict(__ec2_platform_filter_keys | zip(__ec2_platform_generated_tags.values())) }}" + register: __ec2_instance_creation + + +# NOTE: Var is used by the `platform` role to write Molecule instance configuration +- name: Collect instance configs + vars: + __ec2_platform_instance: "{{ __ec2_instance_creation.instances[0] }}" + ansible.builtin.set_fact: + ec2_platform_instance_config: + instance: "{{ ec2_platform.name }}" + address: "{{ ec2_platform.assign_public_ip | ternary(__ec2_platform_instance.public_ip_address, __ec2_platform_instance.private_ip_address) }}" + user: "{{ ec2_platform.ssh_user }}" + port: "{{ ec2_platform.ssh_port }}" + identity_file: "{{ ec2_platform.private_key_path }}" + instance_ids: + - "{{ __ec2_platform_instance.instance_id }}" + +- name: Wait for SSH connectivity + ansible.builtin.wait_for: + host: "{{ ec2_platform_instance_config.address }}" + port: "{{ ec2_platform_instance_config.port }}" + search_regex: SSH + delay: 10 + timeout: 320 + +# TODO: Add an actual check here instead of only waiting +- name: Wait for boot process to finish + ansible.builtin.pause: + seconds: "{{ ec2_platform.boot_wait_seconds }}" + diff --git a/roles/ec2_platform/vars/main.yml b/roles/ec2_platform/vars/main.yml new file mode 100644 index 0000000..11e401d --- /dev/null +++ b/roles/ec2_platform/vars/main.yml @@ -0,0 +1,2 @@ +--- +# vars file for ec2_platform diff --git a/roles/init/README.md b/roles/init/README.md index ae645c2..e1e8bdc 100644 --- a/roles/init/README.md +++ b/roles/init/README.md @@ -124,6 +124,17 @@ Role Variables # The type of project that this Molecule configuration will be integrated into init_project_type: auto +# The type of platform that this Molecule configuration will be testing on (docker, ec2) +# WARN: mixing platform types is not supported! +init_platform_type: docker + +# Version of this collection that should be used by the Molecule test +# - Set to "" to attempt to use the running version +init_collection_version: "" + +# Source of the collection that this role is part of (galaxy, git) +init_collection_source: git + # Filesystem location of the molecule scenario being initialized init_scenario_dir: "{{ molecule_scenario_directory | default(playbook_dir) }}" @@ -144,12 +155,9 @@ init_project_dir: "{{ init_scenario_dir.split('/')[:-2] | join('/') }}" # modify_image: (true/false) # modify_image_buildpath: (string) # path to directory containing Dockerfile # privileged: (true/false) -init_platforms: - - name: docker-rocklinux9 - type: docker - config: - image: "geerlingguy/docker-rockylinux9-ansible:latest" - systemd: true +# +# If not specified, the role will attempt to use the default platform configuration +init_platforms: [] # Create backups of any files that would be clobbered by running this role init_file_backup: true diff --git a/roles/init/defaults/main.yml b/roles/init/defaults/main.yml index 26f6ea9..9463cbd 100644 --- a/roles/init/defaults/main.yml +++ b/roles/init/defaults/main.yml @@ -4,6 +4,17 @@ # The type of project that this Molecule configuration will be integrated into init_project_type: auto +# The type of platform that this Molecule configuration will be testing on (docker, ec2) +# WARN: mixing platform types is not supported! +init_platform_type: docker + +# Version of this collection that should be used by the Molecule test +# - Set to "" to attempt to use the running version +init_collection_version: "" + +# Source of the collection that this role is part of (galaxy, git) +init_collection_source: git + # Filesystem location of the molecule scenario being initialized init_scenario_dir: "{{ molecule_scenario_directory | default(playbook_dir) }}" @@ -24,12 +35,9 @@ init_project_dir: "{{ init_scenario_dir.split('/')[:-2] | join('/') }}" # modify_image: (true/false) # modify_image_buildpath: (string) # path to directory containing Dockerfile # privileged: (true/false) -init_platforms: - - name: docker-rockylinux9 - type: docker - config: - image: "geerlingguy/docker-rockylinux9-ansible:latest" - systemd: true +# +# If not specified, the role will attempt to use the default platform configuration +init_platforms: [] # Create backups of any files that would be clobbered by running this role init_file_backup: true diff --git a/roles/init/files/init.yml b/roles/init/files/init.yml index 1da5128..59fcf5c 100644 --- a/roles/init/files/init.yml +++ b/roles/init/files/init.yml @@ -7,4 +7,9 @@ - name: Launch provisioner ansible.builtin.include_role: name: syndr.molecule.init + vars: + # Supported platform types are: docker, ec2 + init_platform_type: docker + # Supported collection sources are: git, galaxy + init_collection_source: git diff --git a/roles/init/tasks/asserts.yml b/roles/init/tasks/asserts.yml index 307793e..30dd9aa 100644 --- a/roles/init/tasks/asserts.yml +++ b/roles/init/tasks/asserts.yml @@ -8,6 +8,9 @@ - init_scenario_dir is string - init_project_dir is string - init_file_backup in [true, false] + - init_platform_type in ['docker', 'ec2'] + - init_collection_version is string + - init_collection_source in ['galaxy', 'git'] fail_msg: Global configuration option for init role is not sane success_msg: Sanity check passed diff --git a/roles/init/tasks/main.yml b/roles/init/tasks/main.yml index 778a914..f30da08 100644 --- a/roles/init/tasks/main.yml +++ b/roles/init/tasks/main.yml @@ -8,6 +8,76 @@ ansible.builtin.include_tasks: "{{ role_path }}/tasks/auto.yml" when: init_project_type == 'auto' +- name: Build base platform definition + when: init_platforms is not truthy + # TODO: Define these values in the role defaults + block: + - name: Build base docker platform definition + when: init_platform_type == 'docker' + ansible.builtin.set_fact: + init_platforms: + - name: docker-rockylinux9 + type: docker + config: + image: "geerlingguy/docker-rockylinux9-ansible:latest" + systemd: true + + - name: Build base ec2 platform definition + when: init_platform_type == 'ec2' + ansible.builtin.set_fact: + init_platforms: + - name: ec2-rockylinux9 + type: ec2 + config: + image: "ami-067daee80a6d36ac0" + instance_type: "t3.micro" + region: "us-east-2" + vpc_id: "vpc-12345678" + vpc_subnet_id: "subnet-12345678" + + - name: Platform definition is valid + ansible.builtin.assert: + that: + - init_platforms is defined + - init_platforms | length > 0 + - init_platforms[0].name is string + - init_platforms[0].type is string + - init_platforms[0].config is mapping + fail_msg: "Platform definition failed! Check the platform configuration." + success_msg: "Platform definition is valid" + +- name: Load collection meta information + block: + - name: Load collection meta data + ansible.builtin.slurp: + src: "{{ role_path }}/../../MANIFEST.json" + register: __init_collection_meta + ignore_errors: true + + - name: 🐜 Show collection meta data + ansible.builtin.debug: + var: __init_collection_meta.content | b64decode | from_json + verbosity: 1 + ignore_errors: true + + - name: Collection meta data is valid + ansible.builtin.assert: + that: + - __init_collection_meta is not failed + - __init_collection_meta.content is defined + - (__init_collection_meta.content | b64decode | from_json).collection_info.version is defined + fail_msg: "Collection meta data not found! Check the collection configuration." + success_msg: "Collection meta data found" + + - name: Extract collection meta info + ansible.builtin.set_fact: + __init_collection_meta: "{{ (__init_collection_meta.content | b64decode | from_json) }}" + +- name: Get collection version + when: init_collection_version is not truthy + ansible.builtin.set_fact: + init_collection_version: "{{ __init_collection_meta.collection_info.version }}" + - name: Deploy molecule configuration ansible.builtin.template: src: "{{ role_path }}/templates/molecule.yml.j2" diff --git a/roles/init/templates/collections.yml.j2 b/roles/init/templates/collections.yml.j2 index 83d66c4..bf6773c 100644 --- a/roles/init/templates/collections.yml.j2 +++ b/roles/init/templates/collections.yml.j2 @@ -2,7 +2,11 @@ collections: - name: community.docker - - name: git+https://github.com/syndr/ansible-collection-molecule.git +{% if init_collection_source == 'git' %} + - name: git+{{ __init_collection_meta.collection_info.repository }}.git type: git - version: latest +{% elif init_collection_source == 'galaxy' %} + - name: {{ __init_collection_meta.collection_info.namespace }}.{{ __init_collection_meta.collection_info.name }} +{% endif %} + version: {{ init_collection_version }} diff --git a/roles/init/templates/create.yml.j2 b/roles/init/templates/create.yml.j2 index c203a3f..fe9ce25 100644 --- a/roles/init/templates/create.yml.j2 +++ b/roles/init/templates/create.yml.j2 @@ -6,8 +6,7 @@ - name: Create docker platform(s) ansible.builtin.include_role: name: {{ ansible_collection_name }}.platform - vars: -{% raw %} + vars:{% raw %} platform_name: "{{ item.name }}" platform_state: present platform_type: "{{ item.type }}" diff --git a/roles/init/templates/destroy.yml.j2 b/roles/init/templates/destroy.yml.j2 index 4254478..2d49265 100644 --- a/roles/init/templates/destroy.yml.j2 +++ b/roles/init/templates/destroy.yml.j2 @@ -1,15 +1,20 @@ --- - name: Perform cleanup - hosts: molecule + hosts: localhost gather_facts: false tasks: - name: Remove platform ansible.builtin.include_role: - name: {{ ansible_collection_name }}.docker_platform + name: {{ ansible_collection_name }}.platform vars: {% raw %} - docker_platform_name: "{{ inventory_hostname }}" - docker_platform_state: absent + platform_name: "{{ item.name }}" + platform_state: absent + platform_type: "{{ item.type }}" + platform_molecule_cfg: "{{ item }}" + loop: "{{ molecule_yml.platforms }}" + loop_control: + label: item.name {% endraw %} diff --git a/roles/init/templates/molecule.yml.j2 b/roles/init/templates/molecule.yml.j2 index 9aca6f8..c9302b9 100644 --- a/roles/init/templates/molecule.yml.j2 +++ b/roles/init/templates/molecule.yml.j2 @@ -6,7 +6,9 @@ driver: name: default options: managed: true +{% if 'docker' in init_platforms | map(attribute='type') %} login_cmd_template: 'docker exec -ti {instance} bash' +{% endif %} platforms: {% for init_platform in init_platforms %} - name: {{ init_platform.name }} @@ -19,8 +21,14 @@ platforms: modify_image_buildpath: {{ init_platform.config.modify_image_buildpath }} {% endif %} privileged: {{ init_platform.config.privileged | default(false) }} - hostvars: {} {% endif %} +{% if init_platform.type == 'ec2' %} + image: {{ init_platform.config.image }} + region: {{ init_platform.config.region }} + vpc_id: {{ init_platform.config.vpc_id }} + vpc_subnet_id: {{ init_platform.config.vpc_subnet_id }} +{% endif %} + hostvars: {} {% endfor %} provisioner: name: ansible diff --git a/roles/init/templates/prepare.yml.j2 b/roles/init/templates/prepare.yml.j2 index 53fd233..e16f3fc 100644 --- a/roles/init/templates/prepare.yml.j2 +++ b/roles/init/templates/prepare.yml.j2 @@ -13,6 +13,7 @@ - name: Prepare target host for execution hosts: molecule tags: always + become: true tasks: ## # Creating an admin service account for Molecule/Ansible to use for testing diff --git a/roles/platform/README.md b/roles/platform/README.md index 225dd44..1a0269d 100644 --- a/roles/platform/README.md +++ b/roles/platform/README.md @@ -1,38 +1,87 @@ -Role Name +molecule.platform ========= -A brief description of the role goes here. +Deploy a Molecule platform for testing. + +This role handles both the creation and destruction of a Molecule platform, as well as configuration of internal Molecule inventory files necessary to utilize them. It is the recommended way to utilize this collection's platform roles. + +Supported platforms are: +- `docker` +- `ec2` Requirements ------------ -Any pre-requisites that may not be covered by Ansible itself or the role should be mentioned here. For instance, if the role uses the EC2 module, it may be a good idea to mention in this section that the boto package is required. +1. Molecule should be installed and executable from a location in the users PATH +1. Ansible should be installed, with `ansible-playbook` executable via the users PATH Role Variables -------------- -A description of the settable variables for this role should go here, including any variables that are in defaults/main.yml, vars/main.yml, and any variables that can/should be set via parameters to the role. Any variables that are read from other roles and/or the global scope (ie. hostvars, group vars, etc.) should be mentioned here as well. +```yaml +# Name of this Molecule platform +platform_name: instance + +# Whether this platform should be deployed on the current system (present/absent) +platform_state: present + +# What type of platform should be deployed +platform_type: docker + +# Molecule platform configuration +platform_molecule_cfg: {} +``` Dependencies ------------ -A list of other roles hosted on Galaxy should go here, plus any details in regards to parameters that may need to be set for other roles, or variables that are used from other roles. +**Roles included with this collection:** +- `molecule.docker_platform` +- `molecule.ec2_platform` Example Playbook ---------------- -Including an example of how to use your role (for instance, with variables passed in as parameters) is always nice for users too: +```yaml +- name: Create + hosts: localhost + gather_facts: false + tasks: + - name: Create platform(s) + ansible.builtin.include_role: + name: syndr.molecule.platform + vars: + platform_name: "{{ item.name }}" + platform_state: present + platform_type: "{{ item.type }}" + platform_molecule_cfg: "{{ item }}" + loop: "{{ molecule_yml.platforms }}" + loop_control: + label: item.name + +# We want to avoid errors like "Failed to create temporary directory" +- name: Validate that inventory was refreshed + hosts: molecule + gather_facts: false + tasks: + - name: Check uname + ansible.builtin.raw: uname -a + register: result + changed_when: false - - hosts: servers - roles: - - { role: username.rolename, x: 42 } + - name: Display uname info + ansible.builtin.debug: + msg: "{{ result.stdout }}" + +``` License ------- -BSD +MIT Author Information ------------------ -An optional section for the role authors to include contact information, or a website (HTML is not allowed). +- [@syndr](https://github.com/syndr/) + diff --git a/roles/platform/meta/main.yml b/roles/platform/meta/main.yml index c572acc..481d17f 100644 --- a/roles/platform/meta/main.yml +++ b/roles/platform/meta/main.yml @@ -1,7 +1,9 @@ galaxy_info: - author: your name - description: your role description - company: your company (optional) + role_name: platform + namespace: syndr + author: syndr + description: Create and destroy a platform for Molecule testing + company: UltronCORE # If the issue tracker for your role is not on github, uncomment the # next line and provide a value @@ -14,9 +16,9 @@ galaxy_info: # - GPL-3.0-only # - Apache-2.0 # - CC-BY-4.0 - license: license (GPL-2.0-or-later, MIT, etc) + license: MIT - min_ansible_version: 2.1 + min_ansible_version: 2.16 # If this a Container Enabled role, provide the minimum Ansible Container version. # min_ansible_container_version: diff --git a/roles/platform/tasks/deprovision.yml b/roles/platform/tasks/deprovision.yml new file mode 100644 index 0000000..e4dc627 --- /dev/null +++ b/roles/platform/tasks/deprovision.yml @@ -0,0 +1,25 @@ +--- +# Remove deployed resources + +- name: Initilze state + ansible.builtin.set_fact: + # Number of times this role has been included during this playbook run + __platform_run_count: "{{ __platform_run_count | default(0) | int + 1 }}" + +- name: Remove docker-type platform + when: platform_type == 'docker' + ansible.builtin.include_role: + name: "{{ ansible_collection_name }}.docker_platform" + vars: + docker_platform_name: "{{ platform_name }}" + docker_platform_state: absent + +- name: Remove ec2-type platform + when: platform_type == 'ec2' + ansible.builtin.include_role: + name: "{{ ansible_collection_name }}.ec2_platform" + vars: + ec2_platform_name: "{{ platform_name }}" + ec2_platform_state: absent + ec2_platform_definition: "{{ platform_molecule_cfg }}" + diff --git a/roles/platform/tasks/inventory.yml b/roles/platform/tasks/inventory.yml index 88692d3..3e5fcde 100644 --- a/roles/platform/tasks/inventory.yml +++ b/roles/platform/tasks/inventory.yml @@ -2,7 +2,6 @@ # Add a host to the Molecule inventory - name: Load existing instance configuration - when: __platform_run_count | int > 1 block: - name: Load existing instance configuration file ansible.builtin.slurp: @@ -14,21 +13,118 @@ ansible.builtin.set_fact: __platform_current_instance_config: "{{ __platform_current_instance_config_b64.content | default('') | b64decode | from_yaml }}" +- name: Generate new instance configuration + when: platform_state == 'present' + block: + - name: Generate {{ platform_name }} instance configuration (Docker) + when: platform_type == 'docker' + ansible.builtin.set_fact: + __platform_new_instance_config: "{{ { + 'instance': platform_name, + 'connection': 'docker' + } }}" + __platform_ansible_hostvars: + ansible_connection: "community.docker.docker" + + - name: Generate {{ platform_name }} instance configuration (EC2) + when: platform_type == 'ec2' + ansible.builtin.set_fact: + # NOTE: This depends on the ec2_platform_instance_config being set by the ec2_platform role + __platform_new_instance_config: "{{ { + 'instance': platform_name, + 'address': ec2_platform_instance_config.address, + 'user': ec2_platform_instance_config.user, + 'port': ec2_platform_instance_config.port, + 'identity_file': ec2_platform_instance_config.identity_file, + 'instance_ids': ec2_platform_instance_config.instance_ids + } }}" + __platform_ansible_connection: "ssh" + __platform_ansible_hostvars: + ansible_connection: "ssh" + ansible_host: "{{ ec2_platform_instance_config.address }}" + ansible_port: "{{ ec2_platform_instance_config.port }}" + ansible_user: "{{ ec2_platform_instance_config.user }}" + ansible_ssh_private_key_file: "{{ ec2_platform_instance_config.identity_file }}" + + - name: Instance configuration {{ platform_name }} is valid + ansible.builtin.assert: + that: + - __platform_new_instance_config is defined + - __platform_new_instance_config.instance is string + fail_msg: "Instance configuration for {{ platform_name }} failed! Check the platform configuration." + success_msg: "Instance configuration for {{ platform_name }} is defined" + + - name: 🪲 Current instance config + ansible.builtin.debug: + var: __platform_current_instance_config + verbosity: 1 + + - name: Instance name matching this already exists in configuration + when: + - __platform_current_instance_config is truthy + - platform_name in __platform_current_instance_config | map(attribute='instance') | list + block: + - name: Mark config update as unneeded + ansible.builtin.set_fact: + __platform_instance_config_update_needed: false + + - name: Existing configuration does not match desired + when: __platform_new_instance_config != (__platform_current_instance_config | selectattr('instance', 'equalto', platform_name) | list | first) + block: + - name: Remove existing {{ platform_name }} configuration (does not match) + ansible.builtin.set_fact: + __platform_current_instance_config: "{{ + __platform_current_instance_config | rejectattr('instance', 'equalto', platform_name) | list }}" + __platform_instance_config_update_needed: true + +- name: Remove instance configuration + when: platform_state == 'absent' + block: + - name: Remove existing {{ platform_name }} configuration + when: + - __platform_current_instance_config is truthy + - platform_name in __platform_current_instance_config | map(attribute='instance') | list + ansible.builtin.set_fact: + __platform_current_instance_config: "{{ + __platform_current_instance_config | rejectattr('instance', 'equalto', platform_name) | list }}" + __platform_instance_config_update_needed: true + +- name: dump new instance config + ansible.builtin.debug: + var: __platform_new_instance_config + ignore_errors: true + +- name: dump current instance config + ansible.builtin.debug: + var: __platform_current_instance_config + ignore_errors: true + - name: Write {{ platform_name }} instance config file + when: + - __platform_instance_config_update_needed + - __platform_current_instance_config | default(false, true)is truthy or __platform_new_instance_config | default(false, true) is truthy + vars: + __platform_instance_config: >- + {{ __platform_current_instance_config | default([], true) + [__platform_new_instance_config] + if __platform_new_instance_config | default(false, true) is truthy + else __platform_current_instance_config }} ansible.builtin.copy: - # This is very basic - just needs an item there to show as managed with docker config - content: | - {% if __platform_current_instance_config is defined %} - {{ __platform_current_instance_config | to_yaml }} - {% endif %} - # TODO: update this from just docker for connection - - instance: {{ platform_name }} - connection: docker + content: "{{ __platform_instance_config | to_nice_yaml(indent=2) }}" dest: "{{ platform_molecule_ephemeral_directory }}/instance_config.yml" mode: "0600" +# If the file would be empty, remove it +- name: Remove molecule instance config file + when: + - platform_state == 'absent' + - __platform_current_instance_config | default(false) is not truthy + - __platform_new_instance_config | default(false) is not truthy + ansible.builtin.file: + path: "{{ platform_molecule_ephemeral_directory }}/instance_config.yml" + state: absent + - name: Load existing molecule inventory - when: __platform_run_count | int > 1 + when: __platform_run_count | int > 1 or platform_state == 'absent' block: - name: Load existing molecule inventory file ansible.builtin.slurp: @@ -41,24 +137,37 @@ __platform_current_molecule_inventory: "{{ __platform_current_molecule_inventory_b64.content | default({}) | b64decode | from_yaml }}" - name: Add {{ platform_name }} to molecule_inventory + when: platform_state == 'present' vars: - __platform_inventory_partial_hostvars: "{{ { - 'ansible_connection': 'community.docker.docker' - } | combine(platform_molecule_cfg.hostvars, recursive=true) }}" - __platform_inventory_partial_yaml: | - all: - children: - molecule: - hosts: - "{{ platform_name }}": {{ __platform_inventory_partial_hostvars }} + __platform_inventory_partial: "{{ { + 'all': { + 'children': { + 'molecule': { + 'hosts': { + platform_name: __platform_ansible_hostvars + }}}}} }}" ansible.builtin.set_fact: __platform_molecule_inventory: > - {{ __platform_current_molecule_inventory | from_yaml | default({}) | combine(__platform_inventory_partial_yaml | from_yaml, recursive=true) }} + {{ __platform_current_molecule_inventory | from_yaml | default({}) | combine(__platform_inventory_partial, recursive=true) }} + +- name: Remove {{ platform_name }} from molecule_inventory + when: + - platform_state == 'absent' + - __platform_current_molecule_inventory is truthy + ansible.builtin.set_fact: + __platform_molecule_inventory: "{{ + __platform_current_molecule_inventory | combine({ + 'all': { + 'children': { + 'molecule': { + 'hosts': (__platform_current_molecule_inventory.all.children.molecule.hosts | default({}) | + dict2items | rejectattr('key', 'equalto', platform_name) | items2dict) + }}}}, recursive=true) }}" - name: Write {{ platform_name }} to molecule inventory file ansible.builtin.copy: content: | - {{ __platform_molecule_inventory | to_yaml }} + {{ __platform_molecule_inventory | default({}) | to_nice_yaml(indent=2) }} dest: "{{ platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" mode: "0600" @@ -66,6 +175,7 @@ ansible.builtin.meta: refresh_inventory - name: Fail if molecule group is missing + when: __platform_molecule_inventory is defined ansible.builtin.assert: that: "'molecule' in groups" fail_msg: | diff --git a/roles/platform/tasks/main.yml b/roles/platform/tasks/main.yml index 404984b..7b9ddec 100644 --- a/roles/platform/tasks/main.yml +++ b/roles/platform/tasks/main.yml @@ -6,6 +6,9 @@ when: platform_state == 'present' - name: Platform is destroyed - ansible.builtin.include_tasks: "{ role_path }}/tasks/destroy.yml" + ansible.builtin.include_tasks: "{{ role_path }}/tasks/deprovision.yml" when: platform_state == 'absent' +- name: Configure Molecule inventory + ansible.builtin.include_tasks: "{{ role_path }}/tasks/inventory.yml" + diff --git a/roles/platform/tasks/provision.yml b/roles/platform/tasks/provision.yml index aea6771..895fabd 100644 --- a/roles/platform/tasks/provision.yml +++ b/roles/platform/tasks/provision.yml @@ -18,7 +18,7 @@ name: "{{ ansible_collection_name }}.docker_platform" vars: docker_platform_name: "{{ platform_name }}" - docker_platform_state: "{{ platform_state }}" + docker_platform_state: present docker_platform_image: "{{ platform_molecule_cfg.image }}" docker_platform_systemd: "{{ platform_molecule_cfg.systemd | default(false) }}" docker_platform_modify_image: "{{ platform_molecule_cfg.modify_image | default(false) }}" @@ -26,6 +26,12 @@ docker_platform_privileged: "{{ platform_molecule_cfg.privileged | default (false) }}" docker_platform_hostvars: "{{ platform_molecule_cfg.hostvars | default({}) }}" -- name: Configure Molecule inventory - ansible.builtin.include_tasks: "{{ role_path }}/tasks/inventory.yml" +- name: Configure platform for ec2 type + when: platform_type == 'ec2' + ansible.builtin.include_role: + name: "{{ ansible_collection_name }}.ec2_platform" + vars: + ec2_platform_name: "{{ platform_name }}" + ec2_platform_state: present + ec2_platform_definition: "{{ platform_molecule_cfg }}" diff --git a/roles/platform/vars/main.yml b/roles/platform/vars/main.yml index 2cc9bf1..aa4a248 100644 --- a/roles/platform/vars/main.yml +++ b/roles/platform/vars/main.yml @@ -4,4 +4,9 @@ # Filesystem location of the Molecule ephemeral directory. Should not need to be updated by the user of this role! platform_molecule_ephemeral_directory: "{{ molecule_ephemeral_directory }}" +# Does the molecule instance configuration file need to be updated? (assume yes) +__platform_instance_config_update_needed: true + + # Default connection method for hosts -- update as needed in tasks/inventory.yml +__platform_ansible_connection: "ssh" From 1ea945b322a237b2d2598e5b64be94da88b84b1b Mon Sep 17 00:00:00 2001 From: Vincent Date: Fri, 12 Apr 2024 11:45:58 -0600 Subject: [PATCH 09/18] Add default values for case where collection manifest is not accessible (#9) Fixes issue primarily affecting CI, where the `init` role fails if it cannot read the collection manifest. --- roles/init/defaults/main.yml | 7 +++++++ roles/init/tasks/main.yml | 23 +++++++++++------------ roles/init/templates/collections.yml.j2 | 4 ++-- 3 files changed, 20 insertions(+), 14 deletions(-) diff --git a/roles/init/defaults/main.yml b/roles/init/defaults/main.yml index 9463cbd..be8a275 100644 --- a/roles/init/defaults/main.yml +++ b/roles/init/defaults/main.yml @@ -47,3 +47,10 @@ init_file_backup: true # - Set to "" to disable init_ansible_secret_path: "" +# Configuration defaults to be used if the collection manifest is not accessible +init_collection_defaults: + repository: https://github.com/syndr/ansible-collection-molecule + name: molecule + namespace: syndr + version: latest + diff --git a/roles/init/tasks/main.yml b/roles/init/tasks/main.yml index f30da08..85300ce 100644 --- a/roles/init/tasks/main.yml +++ b/roles/init/tasks/main.yml @@ -60,23 +60,22 @@ verbosity: 1 ignore_errors: true - - name: Collection meta data is valid - ansible.builtin.assert: - that: - - __init_collection_meta is not failed - - __init_collection_meta.content is defined - - (__init_collection_meta.content | b64decode | from_json).collection_info.version is defined - fail_msg: "Collection meta data not found! Check the collection configuration." - success_msg: "Collection meta data found" - - name: Extract collection meta info + when: + - __init_collection_meta is not failed + - __init_collection_meta.content is defined + - (__init_collection_meta.content | b64decode | from_json).collection_info.version is defined ansible.builtin.set_fact: __init_collection_meta: "{{ (__init_collection_meta.content | b64decode | from_json) }}" -- name: Get collection version - when: init_collection_version is not truthy +- name: Set collection information ansible.builtin.set_fact: - init_collection_version: "{{ __init_collection_meta.collection_info.version }}" + init_collection_version: >- + "{{ init_collection_version if init_collection_version is truthy + else __init_collection_meta.collection_info.version | default(init_collection_defaults.version) }}" + __init_collection_name: "{{ __init_collection_meta.collection_info.name | default(init_collection_defaults.name) }}" + __init_collection_namespace: "{{ __init_collection_meta.collection_info.namespace | default(init_collection_defaults.namespace) }}" + __init_collection_repository: "{{ __init_collection_meta.collection_info.repository | default(init_collection_defaults.repository) }}" - name: Deploy molecule configuration ansible.builtin.template: diff --git a/roles/init/templates/collections.yml.j2 b/roles/init/templates/collections.yml.j2 index bf6773c..6dde64e 100644 --- a/roles/init/templates/collections.yml.j2 +++ b/roles/init/templates/collections.yml.j2 @@ -3,10 +3,10 @@ collections: - name: community.docker {% if init_collection_source == 'git' %} - - name: git+{{ __init_collection_meta.collection_info.repository }}.git + - name: git+{{ __init_collection_repository }}.git type: git {% elif init_collection_source == 'galaxy' %} - - name: {{ __init_collection_meta.collection_info.namespace }}.{{ __init_collection_meta.collection_info.name }} + - name: {{ __init_collection_namespace }}.{{ __init_collection_name }} {% endif %} version: {{ init_collection_version }} From 6006b078691e4a12fda3ace4835460b4de9206de Mon Sep 17 00:00:00 2001 From: Vincent Date: Wed, 8 Jan 2025 17:08:34 -0700 Subject: [PATCH 10/18] Update architecture (#11) * Rename components for influxdata namespace * Fix collections template, add latest tag (#1) * Fix collections template * Add workflow for generating 'latest' tag * Update docs (#2) * Add testing for init role * Fix license * Disable release workflow * Fix running multiple platforms, support ubuntu for testing Fix the `docker_platform` role so that it works correctly when more than one platform is specified in molecule.yml. Add more useful output to container creation steps (`docker_platform`) Fix docker_platform to work with debian-based systemd images Support ubuntu for testing workflow * Replace chicken with egg Fixes in this devel branch are needed for it to run its own internal tests. Will need to create another PR after current changing collection version back to main after these fixes are merged. * Fix typo * Use 'latest' for version -- makes way more sense * Only push to Galaxy on mainline release * Fix namespace * Merge internal branch updates Amazon EC2 test scenario Refactoring of Molecule tests Lots of other things... * Fix build CI, docker_platform, bump version * Dynamically generate branch tag * Fix CI execution * Only deploy on successful build, allow 'degraded' systemd status * Refactor branch tag CI step * Bump version to 1.5 --- .github/workflows/build.yml | 79 +++++++ .github/workflows/latest.yml | 27 --- .github/workflows/main.yml | 36 --- .github/workflows/publish.yml | 11 +- .gitignore | 2 +- galaxy.yml | 2 +- molecule/default/collections.yml | 4 +- molecule/default/create.yml | 7 - molecule/default/molecule.yml | 23 +- molecule/default/requirements.yml | 5 +- molecule/default/verify.yml | 21 -- molecule/ec2_platform/collections.yml | 1 + .../ec2_platform}/converge.yml | 0 .../ec2_platform}/molecule.yml | 13 +- .../ec2_platform}/prepare.yml | 2 +- molecule/ec2_platform/requirements.yml | 1 + molecule/{default => resources}/cleanup.yml | 0 molecule/resources/collections.yml | 9 + molecule/resources/create.yml | 53 +++++ molecule/{default => resources}/destroy.yml | 0 molecule/{default => resources}/prepare.yml | 16 ++ .../resources}/requirements.yml | 0 .../{default => resources}/side_effect.yml | 0 molecule/resources/verify.yml | 13 ++ roles/docker_platform/tasks/create.yml | 117 ++++++++++ roles/docker_platform/tasks/present.yml | 147 ++++++------- roles/docker_platform/templates/Dockerfile.j2 | 2 +- roles/ec2_platform/README.md | 80 +++++++ .../molecule/role-ec2_platform/cleanup.yml | 13 -- .../role-ec2_platform/collections.yml | 10 - .../molecule/role-ec2_platform/create.yml | 58 ----- .../molecule/role-ec2_platform/destroy.yml | 18 -- .../molecule/role-ec2_platform/init.yml | 10 - .../role-ec2_platform/side_effect.yml | 10 - .../molecule/role-ec2_platform/verify.yml | 21 -- roles/ec2_platform/tasks/present.yml | 7 + roles/init/defaults/main.yml | 20 +- roles/init/files/init.yml | 3 + roles/init/tasks/asserts.yml | 1 + roles/init/tasks/main.yml | 24 +- roles/init/templates/collections.yml.j2 | 3 +- roles/init/templates/create.yml.j2 | 2 +- roles/init/templates/destroy.yml.j2 | 4 +- roles/init/templates/molecule.yml.j2 | 3 + roles/init/templates/prepare.yml.j2 | 2 +- roles/init/vars/main.yml | 1 + roles/platform/tasks/ansible_inventory.yml | 56 +++++ roles/platform/tasks/instance_config.yml | 85 +++++++ roles/platform/tasks/inventory.yml | 207 +++++++----------- roles/platform/vars/main.yml | 3 + 50 files changed, 736 insertions(+), 496 deletions(-) create mode 100644 .github/workflows/build.yml delete mode 100644 .github/workflows/latest.yml delete mode 100644 .github/workflows/main.yml mode change 100644 => 120000 molecule/default/requirements.yml delete mode 100644 molecule/default/verify.yml create mode 120000 molecule/ec2_platform/collections.yml rename {roles/ec2_platform/molecule/role-ec2_platform => molecule/ec2_platform}/converge.yml (100%) rename {roles/ec2_platform/molecule/role-ec2_platform => molecule/ec2_platform}/molecule.yml (80%) rename {roles/ec2_platform/molecule/role-ec2_platform => molecule/ec2_platform}/prepare.yml (97%) create mode 120000 molecule/ec2_platform/requirements.yml rename molecule/{default => resources}/cleanup.yml (100%) create mode 100644 molecule/resources/collections.yml create mode 100644 molecule/resources/create.yml rename molecule/{default => resources}/destroy.yml (100%) rename molecule/{default => resources}/prepare.yml (87%) rename {roles/ec2_platform/molecule/role-ec2_platform => molecule/resources}/requirements.yml (100%) rename molecule/{default => resources}/side_effect.yml (100%) create mode 100644 molecule/resources/verify.yml create mode 100644 roles/docker_platform/tasks/create.yml delete mode 100644 roles/ec2_platform/molecule/role-ec2_platform/cleanup.yml delete mode 100644 roles/ec2_platform/molecule/role-ec2_platform/collections.yml delete mode 100644 roles/ec2_platform/molecule/role-ec2_platform/create.yml delete mode 100644 roles/ec2_platform/molecule/role-ec2_platform/destroy.yml delete mode 100644 roles/ec2_platform/molecule/role-ec2_platform/init.yml delete mode 100644 roles/ec2_platform/molecule/role-ec2_platform/side_effect.yml delete mode 100644 roles/ec2_platform/molecule/role-ec2_platform/verify.yml create mode 100644 roles/platform/tasks/ansible_inventory.yml create mode 100644 roles/platform/tasks/instance_config.yml diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml new file mode 100644 index 0000000..9f9fb0c --- /dev/null +++ b/.github/workflows/build.yml @@ -0,0 +1,79 @@ +--- + +name: Test and release 'main' +on: + push: + branches: + - main + - devel + pull_request: + branches: + - main +jobs: + build: + runs-on: ubuntu-latest + strategy: + max-parallel: 4 + matrix: + python-version: ["3.12"] + molecule-scenario: + - default + - ec2_platform + + steps: + - uses: actions/checkout@v4 + + - name: Set up Python ${{ matrix.python-version }} + uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + + - name: Install dependencies + run: | + python3 -m pip install --upgrade pip + pip install molecule ansible-lint requests boto3[crt] + if [ -f requirements.txt ]; then pip install -r requirements.txt; fi + + - name: Build and install collection + run: | + ansible-galaxy collection build --force + ansible-galaxy collection install --force syndr-molecule-*.tar.gz + + - name: Log into AWS + if: ${{ matrix.molecule-scenario == 'ec2_platform' }} + uses: aws-actions/configure-aws-credentials@v4 + with: + aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }} + aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }} + aws-region: us-east-2 + + - name: Test with molecule + env: + # Make molecule output colored content (easier to read via GH UI) + PY_COLORS: 1 + ANSIBLE_FORCE_COLOR: 1 + run: | + molecule test -s ${{ matrix.molecule-scenario }} + + deploy: + runs-on: ubuntu-latest + needs: build + if: github.event_name == 'push' + steps: + - uses: actions/checkout@v4 + + - name: Create and Push Tag + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + run: | + set -e + TAG_NAME=${{ github.ref_name }} + echo "Creating tag $TAG_NAME" + # Configure Git user + git config user.name "github-actions[bot]" + git config user.email "github-actions[bot]@users.noreply.github.com" + # Create the tag + git tag "$TAG_NAME" + # Push the tag to the repository + git push origin "refs/tags/$TAG_NAME" --force + diff --git a/.github/workflows/latest.yml b/.github/workflows/latest.yml deleted file mode 100644 index 82397a0..0000000 --- a/.github/workflows/latest.yml +++ /dev/null @@ -1,27 +0,0 @@ ---- - -name: Update `latest` tag -on: - push: - branches: - - main - -jobs: - run: - runs-on: ubuntu-latest - permissions: - contents: write - packages: write - steps: - - name: Checkout repository - uses: actions/checkout@v4 - - - name: Run latest-tag - uses: EndBug/latest-tag@latest - with: - ref: latest - description: This tag is automatically generated on new releases. - env: - GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} - - diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml deleted file mode 100644 index ca85014..0000000 --- a/.github/workflows/main.yml +++ /dev/null @@ -1,36 +0,0 @@ ---- - -name: Molecule Test -on: - push: - branches: - - main - pull_request: - branches: - - main -jobs: - build: - runs-on: ubuntu-latest - strategy: - max-parallel: 4 - matrix: - python-version: ["3.12"] - - steps: - - uses: actions/checkout@v4 - - - name: Set up Python ${{ matrix.python-version }} - uses: actions/setup-python@v5 - with: - python-version: ${{ matrix.python-version }} - - - name: Install dependencies - run: | - python3 -m pip install --upgrade pip - pip install molecule ansible-lint requests - if [ -f requirements.txt ]; then pip install -r requirements.txt; fi - - - name: Test with molecule - run: | - molecule test - diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index fb2b080..87030f1 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -4,7 +4,8 @@ name: Deploy Collection on: release: types: - - created + - published + jobs: deploy: runs-on: ubuntu-latest @@ -13,6 +14,14 @@ jobs: steps: - uses: actions/checkout@v4 + - name: Tag latest release + uses: EndBug/latest-tag@latest + with: + ref: latest + description: This tag is automatically generated on new releases. + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + - name: Build and Deploy Collection uses: artis3n/ansible_galaxy_collection@v2 with: diff --git a/.gitignore b/.gitignore index 6fe9758..de67a10 100644 --- a/.gitignore +++ b/.gitignore @@ -1 +1 @@ -syndr-molecule-*.tar.gz +*-molecule-*.tar.gz diff --git a/galaxy.yml b/galaxy.yml index becb239..9b81e8e 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -8,7 +8,7 @@ namespace: syndr name: molecule # The version of the collection. Must be compatible with semantic versioning -version: 1.4.0 +version: 1.5.0 # The path to the Markdown (.md) readme file. This path is relative to the root of the collection readme: README.md diff --git a/molecule/default/collections.yml b/molecule/default/collections.yml index 6db7c73..b640240 100644 --- a/molecule/default/collections.yml +++ b/molecule/default/collections.yml @@ -3,7 +3,5 @@ collections: - name: community.general - name: community.docker - - name: git+https://github.com/syndr/ansible-collection-molecule.git - type: git - version: latest + - name: syndr.molecule diff --git a/molecule/default/create.yml b/molecule/default/create.yml index fda8be2..b05d21e 100644 --- a/molecule/default/create.yml +++ b/molecule/default/create.yml @@ -46,12 +46,5 @@ delay: 5 changed_when: false failed_when: systemctl_status.rc > 1 - - - name: Check systemd status - ansible.builtin.assert: - that: - - systemctl_status.stdout == 'running' - fail_msg: Systemd-enabled container does not have a healthy Systemd! - success_msg: Systemd is running when: ansible_service_mgr == 'systemd' diff --git a/molecule/default/molecule.yml b/molecule/default/molecule.yml index db2a5bd..7afbbd0 100644 --- a/molecule/default/molecule.yml +++ b/molecule/default/molecule.yml @@ -14,30 +14,35 @@ platforms: systemd: True modify_image: False privileged: False - hostvars: {} - - name: docker-fedora39 + hostvars: + test_hostvar: test + - name: docker-fedora41 type: docker - image: geerlingguy/docker-fedora39-ansible:latest + image: geerlingguy/docker-fedora41-ansible:latest systemd: True modify_image: False privileged: False - hostvars: {} + hostvars: + test_hostvar: test - name: docker-ubuntu2204 type: docker image: geerlingguy/docker-ubuntu2204-ansible:latest systemd: True modify_image: False privileged: False - hostvars: {} + hostvars: + test_hostvar: test provisioner: name: ansible log: True playbooks: - prepare: prepare.yml + create: ../resources/create.yml + prepare: ../resources/prepare.yml converge: converge.yml - side_effect: side_effect.yml - verify: verify.yml - cleanup: cleanup.yml + side_effect: ../resources/side_effect.yml + verify: ../resources/verify.yml + cleanup: ../resources/cleanup.yml + destroy: ../resources/destroy.yml config_options: defaults: gathering: explicit diff --git a/molecule/default/requirements.yml b/molecule/default/requirements.yml deleted file mode 100644 index 39b222d..0000000 --- a/molecule/default/requirements.yml +++ /dev/null @@ -1,4 +0,0 @@ ---- - -roles: [] - diff --git a/molecule/default/requirements.yml b/molecule/default/requirements.yml new file mode 120000 index 0000000..bffdc7c --- /dev/null +++ b/molecule/default/requirements.yml @@ -0,0 +1 @@ +../resources/requirements.yml \ No newline at end of file diff --git a/molecule/default/verify.yml b/molecule/default/verify.yml deleted file mode 100644 index 3237d2c..0000000 --- a/molecule/default/verify.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -# Verify that the role being tested has done what it's supposed to - -- name: Verify - hosts: molecule - tasks: - - name: Load local host facts - ansible.builtin.setup: - gather_subset: - - '!all' - - '!min' - - local - - - name: Load test data (example) - ansible.builtin.set_fact: - test_prepare_fact: "{{ ansible_local.molecule.test_prepare_fact }}" - - - name: Add your verification tasks here - ansible.builtin.debug: - msg: "IE: For a 'users' role, check that the test user exists" - diff --git a/molecule/ec2_platform/collections.yml b/molecule/ec2_platform/collections.yml new file mode 120000 index 0000000..fec6c24 --- /dev/null +++ b/molecule/ec2_platform/collections.yml @@ -0,0 +1 @@ +../resources/collections.yml \ No newline at end of file diff --git a/roles/ec2_platform/molecule/role-ec2_platform/converge.yml b/molecule/ec2_platform/converge.yml similarity index 100% rename from roles/ec2_platform/molecule/role-ec2_platform/converge.yml rename to molecule/ec2_platform/converge.yml diff --git a/roles/ec2_platform/molecule/role-ec2_platform/molecule.yml b/molecule/ec2_platform/molecule.yml similarity index 80% rename from roles/ec2_platform/molecule/role-ec2_platform/molecule.yml rename to molecule/ec2_platform/molecule.yml index fd0f925..8a106b8 100644 --- a/roles/ec2_platform/molecule/role-ec2_platform/molecule.yml +++ b/molecule/ec2_platform/molecule.yml @@ -9,21 +9,24 @@ driver: platforms: - name: ec2-rockylinux9 type: ec2 - image: ami-067daee80a6d36ac0 + image: ami-01bd836275f79352c region: us-east-2 vpc_id: vpc-0eb9fd1391f4207ec vpc_subnet_id: subnet-0aa189c0d6fc53923 instance_type: t3.micro - hostvars: {} + hostvars: + test_hostvar: test provisioner: name: ansible log: True playbooks: + create: ../resources/create.yml prepare: prepare.yml converge: converge.yml - side_effect: side_effect.yml - verify: verify.yml - cleanup: cleanup.yml + side_effect: ../resourcs/side_effect.yml + verify: ../resources/verify.yml + cleanup: ../resources/cleanup.yml + destroy: ../resources/destroy.yml config_options: defaults: gathering: explicit diff --git a/roles/ec2_platform/molecule/role-ec2_platform/prepare.yml b/molecule/ec2_platform/prepare.yml similarity index 97% rename from roles/ec2_platform/molecule/role-ec2_platform/prepare.yml rename to molecule/ec2_platform/prepare.yml index 6d66d37..a08e4df 100644 --- a/roles/ec2_platform/molecule/role-ec2_platform/prepare.yml +++ b/molecule/ec2_platform/prepare.yml @@ -8,7 +8,7 @@ ansible.builtin.include_role: name: syndr.molecule.prepare_controller vars: - prepare_controller_project_type: role + prepare_controller_project_type: collection - name: Prepare target host for execution hosts: molecule diff --git a/molecule/ec2_platform/requirements.yml b/molecule/ec2_platform/requirements.yml new file mode 120000 index 0000000..bffdc7c --- /dev/null +++ b/molecule/ec2_platform/requirements.yml @@ -0,0 +1 @@ +../resources/requirements.yml \ No newline at end of file diff --git a/molecule/default/cleanup.yml b/molecule/resources/cleanup.yml similarity index 100% rename from molecule/default/cleanup.yml rename to molecule/resources/cleanup.yml diff --git a/molecule/resources/collections.yml b/molecule/resources/collections.yml new file mode 100644 index 0000000..91773b1 --- /dev/null +++ b/molecule/resources/collections.yml @@ -0,0 +1,9 @@ +--- + +collections: + - name: community.general + - name: community.docker + - name: community.crypto + - name: syndr.molecule + - name: amazon.aws + diff --git a/molecule/resources/create.yml b/molecule/resources/create.yml new file mode 100644 index 0000000..56f8790 --- /dev/null +++ b/molecule/resources/create.yml @@ -0,0 +1,53 @@ +--- +- name: Create + hosts: localhost + gather_facts: false + tasks: + - name: Create test platform(s) + ansible.builtin.include_role: + name: syndr.molecule.platform + vars: + platform_name: "{{ item.name }}" + platform_state: present + platform_type: "{{ item.type }}" + platform_molecule_cfg: "{{ item }}" + loop: "{{ molecule_yml.platforms }}" + loop_control: + label: item.name + +# We want to avoid errors like "Failed to create temporary directory" +- name: Validate that inventory was refreshed + hosts: molecule + gather_facts: false + tasks: + - name: Check uname + ansible.builtin.raw: uname -a + register: result + changed_when: false + + - name: Display uname info + ansible.builtin.debug: + msg: "{{ result.stdout }}" + + - name: Systems likely to have python + when: not result.stdout | regex_search('coreos') + block: + - name: Load system facts + ansible.builtin.setup: + filter: + - ansible_service_mgr + + - name: Check on Systemd + when: ansible_service_mgr == 'systemd' + block: + - name: Wait for systemd to complete initialization. + ansible.builtin.command: systemctl is-system-running + register: systemctl_status + until: > + 'running' in systemctl_status.stdout or + 'degraded' in systemctl_status.stdout + retries: 30 + delay: 5 + changed_when: false + failed_when: systemctl_status.rc > 1 + diff --git a/molecule/default/destroy.yml b/molecule/resources/destroy.yml similarity index 100% rename from molecule/default/destroy.yml rename to molecule/resources/destroy.yml diff --git a/molecule/default/prepare.yml b/molecule/resources/prepare.yml similarity index 87% rename from molecule/default/prepare.yml rename to molecule/resources/prepare.yml index 229ae43..aca497f 100644 --- a/molecule/default/prepare.yml +++ b/molecule/resources/prepare.yml @@ -2,6 +2,9 @@ - name: Prepare controller for execution hosts: localhost + vars: + collection_namespace: "{{ (lookup('file', playbook_dir ~ '/../../galaxy.yml') | from_yaml).namespace }}" + collection_name: "{{ (lookup('file', playbook_dir ~ '/../../galaxy.yml') | from_yaml).name }}" tasks: - name: Configure for standalone role testing ansible.builtin.include_role: @@ -15,6 +18,19 @@ dest: "{{ molecule_ephemeral_directory }}/project.tar" format: tar + - name: Build collection + ansible.builtin.command: + chdir: "{{ playbook_dir }}/../../" + cmd: ansible-galaxy collection build --force + register: build_result + + - name: Install collection + vars: + collection_archive_path: "{{ build_result.stdout | regex_replace('.*\\s+(\\S+\\.tar\\.gz)', '\\1') }}" + ansible.builtin.command: + chdir: "{{ playbook_dir }}/../../" + cmd: ansible-galaxy collection install --force {{ collection_archive_path }} + - name: Prepare target host for execution hosts: molecule tags: always diff --git a/roles/ec2_platform/molecule/role-ec2_platform/requirements.yml b/molecule/resources/requirements.yml similarity index 100% rename from roles/ec2_platform/molecule/role-ec2_platform/requirements.yml rename to molecule/resources/requirements.yml diff --git a/molecule/default/side_effect.yml b/molecule/resources/side_effect.yml similarity index 100% rename from molecule/default/side_effect.yml rename to molecule/resources/side_effect.yml diff --git a/molecule/resources/verify.yml b/molecule/resources/verify.yml new file mode 100644 index 0000000..cd1e25e --- /dev/null +++ b/molecule/resources/verify.yml @@ -0,0 +1,13 @@ +--- +# Verify that the role being tested has done what it's supposed to + +- name: Verify + hosts: molecule + tasks: + - name: Test hostvar exists + ansible.builtin.assert: + that: + - test_hostvar == "test" + fail_msg: "Custom hostvar not found! Possible issue with storing user-defined hostvars!" + success_msg: "Custom hostvar found!" + diff --git a/roles/docker_platform/tasks/create.yml b/roles/docker_platform/tasks/create.yml new file mode 100644 index 0000000..7246db6 --- /dev/null +++ b/roles/docker_platform/tasks/create.yml @@ -0,0 +1,117 @@ +--- +# Create a docker container for use by molecule +# +# Expected to be called in a loop with `platform` defined as the loop var +# (loop off of `platforms` list in molecule.yml) + + +- name: Docker image needs to be customized + when: docker_platform_modify_image + block: + - name: Check build path + ansible.builtin.stat: + path: "{{ docker_platform_modify_image_buildpath }}" + register: __docker_platform_buildpath_stat + + - name: Build directory doesn't exist + when: __docker_platform_buildpath_stat.stat.exists is false + block: + - name: Create build directory + ansible.builtin.file: + path: "{{ docker_platform_modify_image_buildpath }}" + state: directory + mode: 0755 + + - name: Copy templates + loop: + - bash.service.j2 + - entrypoint.sh.j2 + - Dockerfile.j2 + loop_control: + loop_var: __docker_platform_item + ansible.builtin.template: + src: templates/{{ __docker_platform_item }} + dest: "{{ docker_platform_modify_image_buildpath}}/{{ __docker_platform_item | regex_replace('\\.j2$', '') }}" + + - name: Build local image name + ansible.builtin.set_fact: + __docker_platform_built_image_name: "molecule-local-build/{{ docker_platform_image | split(':') | first | split('/') | last }}-custom" + + - name: Docker image is built + community.docker.docker_image: + name: "{{ __docker_platform_built_image_name }}" + build: + path: "{{ docker_platform_modify_image_buildpath }}" + cache_from: "{{ docker_platform_image }}" + source: build + force_source: true # Always build a new image when this is run + tag: latest + register: image_build_output + + - name: Show image build details + ansible.builtin.debug: + var: image_build_output + verbosity: 1 + +- name: Build docker volume list + ansible.builtin.set_fact: + __docker_platform_volume_list: "{{ docker_platform_volumes + ['/sys/fs/cgroup:/sys/fs/cgroup:rw'] + if docker_platform_systemd + else docker_platform_volumes }}" + +- name: "{{ docker_platform_name }} docker container is present and running" + community.docker.docker_container: + name: "{{ docker_platform_name }}" + image: "{{ __docker_platform_built_image_name | default(docker_platform_image) }}" + state: started + command: "{{ docker_platform_command }}" + log_driver: json-file + hostname: molecule-ci-{{ docker_platform_name }} + init: false + cgroupns_mode: "{{ 'host' if docker_platform_systemd is true else 'private' }}" + privileged: "{{ docker_platform_privileged }}" + tmpfs: "{{ docker_platform_tmpfs + ['/run', '/run/lock'] if docker_platform_systemd else docker_platform_tmpfs }}" + volumes: "{{ __docker_platform_volume_list }}" + register: __docker_platform_create_result + +- name: Print creation output + ansible.builtin.debug: + msg: "{{ __docker_platform_create_result }}" + verbosity: 1 + +- name: Fail if is not running + when: > + __docker_platform_create_result.container.State.ExitCode != 0 or + not __docker_platform_create_result.container.State.Running + block: + - name: Retrieve {{ docker_platform_name }} container log + ansible.builtin.command: + cmd: docker logs {{ __docker_platform_create_result.container.Name }} + changed_when: false + register: __docker_platform_logfile_cmd + + - name: Container {{ docker_platform_name }} failed to start + ansible.builtin.fail: + msg: "{{ __docker_platform_logfile_cmd.stdout ~ __docker_platform_logfile_cmd.stderr }}" + +- name: "{{ docker_platform_name }} Systemd status is healthy" + when: docker_platform_systemd + block: + - name: System service manager is Systemd + ansible.builtin.assert: + that: + - "ansible_service_mgr == 'systemd'" + fail_msg: Systemd is enabled, but container service manager isn't Systemd! Is this a Systemd-enabled container? + + - name: Systemd has completed initialization - {{ docker_platform_name }} + ansible.builtin.command: + cmd: systemctl is-system-running + register: __docker_platform_systemctl_status + until: > + 'running' in __docker_platform_systemctl_status.stdout or + 'degraded' in __docker_platform_systemctl_status.stdout + retries: 30 + delay: 5 + changed_when: false + failed_when: __docker_platform_systemctl_status.rc > 1 + diff --git a/roles/docker_platform/tasks/present.yml b/roles/docker_platform/tasks/present.yml index c3d6d16..dd1a81c 100644 --- a/roles/docker_platform/tasks/present.yml +++ b/roles/docker_platform/tasks/present.yml @@ -1,101 +1,84 @@ --- -# Create a docker container for use by molecule -# -# Expected to be called in a loop with `platform` defined as the loop var -# (loop off of `platforms` list in molecule.yml) - - name: Initialize state ansible.builtin.set_fact: # Number of times this role has been included during this playbook run __docker_platform_run_count: "{{ __docker_platform_run_count | default(0) | int + 1 }}" -- name: Docker image needs to be customized - block: - - name: Check build path - ansible.builtin.stat: - path: "{{ docker_platform_modify_image_buildpath }}" - register: __docker_platform_buildpath_stat +- name: Load system facts + ansible.builtin.setup: + filter: + - ansible_service_mgr - - name: Build directory doesn't exist - block: - - name: Create build directory - ansible.builtin.file: - path: "{{ docker_platform_modify_image_buildpath }}" - state: directory - mode: 0755 +- name: Create {{ docker_platform_name }} docker container + ansible.builtin.include_tasks: "{{ role_path }}/tasks/create.yml" - - name: Copy templates - ansible.builtin.template: - src: templates/{{ __docker_platform_item }} - dest: "{{ docker_platform_modify_image_buildpath}}/{{ __docker_platform_item | regex_replace('\\.j2$', '') }}" - loop: - - bash.service.j2 - - entrypoint.sh.j2 - - Dockerfile.j2 - loop_control: - loop_var: __docker_platform_item - when: __docker_platform_buildpath_stat.stat.exists is false +- name: Load existing instance configuration + block: + - name: Load existing instance configuration file + ansible.builtin.slurp: + src: "{{ docker_platform_molecule_ephemeral_directory }}/instance_config.yml" + register: __docker_platform_current_instance_config_b64 + ignore_errors: true - - name: Build local image name + - name: Decode instance configuration data ansible.builtin.set_fact: - __docker_platform_built_image_name: "molecule-local-build/{{ docker_platform_image | split(':') | first | split('/') | last }}-custom" + __docker_platform_current_instance_config: "{{ __docker_platform_current_instance_config_b64.content | default('') | b64decode | from_yaml }}" + when: __docker_platform_run_count | int > 1 + +- name: Write {{ docker_platform_name }} instance config file + ansible.builtin.copy: + # This is very basic - just needs an item there to show as managed with docker config + content: | + {% if __docker_platform_current_instance_config is defined %} + {{ __docker_platform_current_instance_config | to_yaml }} + {% endif %} + - instance: {{ docker_platform_name }} + connection: docker + dest: "{{ docker_platform_molecule_ephemeral_directory }}/instance_config.yml" + mode: "0600" - - name: Docker image is built - community.docker.docker_image: - name: "{{ __docker_platform_built_image_name }}" - build: - path: "{{ docker_platform_modify_image_buildpath }}" - cache_from: "{{ docker_platform_image }}" - source: build - force_source: true # Always build a new image when this is run - tag: latest - register: image_build_output +- name: Load existing molecule inventory + block: + - name: Load existing molecule inventory file + ansible.builtin.slurp: + src: "{{ docker_platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" + register: __docker_platform_current_molecule_inventory_b64 + ignore_errors: true - - name: Show image build details - ansible.builtin.debug: - var: image_build_output - verbosity: 1 - when: docker_platform_modify_image + - name: Decode instance configuration data + ansible.builtin.set_fact: + __docker_platform_current_molecule_inventory: "{{ __docker_platform_current_molecule_inventory_b64.content | default({}) | b64decode | from_yaml }}" + when: __docker_platform_run_count | int > 1 -- name: Build docker volume list +- name: Add {{ docker_platform_name }} to molecule_inventory + vars: + __docker_platform_inventory_partial_hostvars: "{{ { + 'ansible_connection': 'community.docker.docker' + } | combine(docker_platform_hostvars, recursive=true) }}" + __docker_platform_inventory_partial_yaml: | + all: + children: + molecule: + hosts: + "{{ docker_platform_name }}": {{ __docker_platform_inventory_partial_hostvars }} ansible.builtin.set_fact: - __docker_platform_volume_list: "{{ docker_platform_volumes + ['/sys/fs/cgroup:/sys/fs/cgroup:rw'] - if docker_platform_systemd - else docker_platform_volumes }}" + __docker_platform_molecule_inventory: > + {{ __docker_platform_current_molecule_inventory | from_yaml | default({}) | combine(__docker_platform_inventory_partial_yaml | from_yaml, recursive=true) }} -- name: "{{ docker_platform_name }} docker container is present and running" - community.docker.docker_container: - name: "{{ docker_platform_name }}" - image: "{{ __docker_platform_built_image_name | default(docker_platform_image) }}" - state: started - command: "{{ docker_platform_command }}" - log_driver: json-file - hostname: molecule-ci-{{ docker_platform_name }} - init: false - cgroupns_mode: "{{ 'host' if docker_platform_systemd is true else 'private' }}" - privileged: "{{ docker_platform_privileged }}" - tmpfs: "{{ docker_platform_tmpfs + ['/run', '/run/lock'] if docker_platform_systemd else docker_platform_tmpfs }}" - volumes: "{{ __docker_platform_volume_list }}" - register: __docker_platform_create_result +- name: Write {{ docker_platform_name }} to molecule inventory file + ansible.builtin.copy: + content: | + {{ __docker_platform_molecule_inventory | to_yaml }} + dest: "{{ docker_platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" + mode: "0600" -- name: Print creation output - ansible.builtin.debug: - msg: "{{ __docker_platform_create_result }}" - verbosity: 1 - -- name: Fail if is not running - block: - - name: Retrieve {{ docker_platform_name }} container log - ansible.builtin.command: - cmd: docker logs {{ __docker_platform_create_result.container.Name }} - changed_when: false - register: __docker_platform_logfile_cmd +- name: Force inventory refresh + ansible.builtin.meta: refresh_inventory - - name: Container {{ docker_platform_name }} failed to start - ansible.builtin.fail: - msg: "{{ __docker_platform_logfile_cmd.stdout ~ __docker_platform_logfile_cmd.stderr }}" - when: > - __docker_platform_create_result.container.State.ExitCode != 0 or - not __docker_platform_create_result.container.State.Running +- name: Fail if molecule group is missing + ansible.builtin.assert: + that: "'molecule' in groups" + fail_msg: | + molecule group was not found inside inventory groups: {{ groups }} diff --git a/roles/docker_platform/templates/Dockerfile.j2 b/roles/docker_platform/templates/Dockerfile.j2 index 0214ed4..f1d3101 100644 --- a/roles/docker_platform/templates/Dockerfile.j2 +++ b/roles/docker_platform/templates/Dockerfile.j2 @@ -1,5 +1,5 @@ -FROM {{ platform.image }} +FROM {{ docker_platform_image }} COPY bash.service /etc/systemd/system/bash.service COPY entrypoint.sh /entrypoint.sh RUN chown root:root /entrypoint.sh \ diff --git a/roles/ec2_platform/README.md b/roles/ec2_platform/README.md index 706abf9..8f8a17f 100644 --- a/roles/ec2_platform/README.md +++ b/roles/ec2_platform/README.md @@ -45,6 +45,86 @@ Requirements **Python Modules** - `boto3` +## AWS Configuration + +These tests require an AWS account with the following or similar permissions: + +
+AWS AIM permissions policy + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "ec2:RunInstances", + "ec2:DescribeInstances", + "ec2:DescribeInstanceStatus", + "ec2:DescribeSubnets", + "ec2:CreateSecurityGroup", + "ec2:DescribeSecurityGroups", + "ec2:CreateTags", + "ec2:DescribeTags" + ], + "Resource": "*" + }, + { + "Effect": "Allow", + "Action": [ + "ec2:TerminateInstances", + "ec2:StopInstances", + "ec2:StartInstances", + "ec2:RebootInstances", + "ec2:DescribeInstanceAttribute", + "ec2:ModifyInstanceAttribute" + ], + "Resource": "arn:aws:ec2:*:*:instance/*", + "Condition": { + "StringLike": { + "ec2:ResourceTag/Name": "molecule*" + } + } + }, + { + "Effect": "Allow", + "Action": [ + "ec2:DeleteSecurityGroup", + "ec2:AuthorizeSecurityGroupIngress", + "ec2:AuthorizeSecurityGroupEgress", + "ec2:RevokeSecurityGroupIngress", + "ec2:RevokeSecurityGroupEgress", + "ec2:ModifySecurityGroupRules" + ], + "Resource": "arn:aws:ec2:*:*:security-group/*", + "Condition": { + "StringLike": { + "ec2:ResourceTag/Name": "molecule*" + } + } + }, + { + "Effect": "Allow", + "Action": [ + "ec2:DeleteTags" + ], + "Resource": [ + "arn:aws:ec2:*:*:instance/*", + "arn:aws:ec2:*:*:security-group/*" + ], + "Condition": { + "StringLike": { + "ec2:ResourceTag/Name": "molecule*" + } + } + } + ] +} +``` + +
+ Role Variables -------------- diff --git a/roles/ec2_platform/molecule/role-ec2_platform/cleanup.yml b/roles/ec2_platform/molecule/role-ec2_platform/cleanup.yml deleted file mode 100644 index 26e7fa5..0000000 --- a/roles/ec2_platform/molecule/role-ec2_platform/cleanup.yml +++ /dev/null @@ -1,13 +0,0 @@ ---- -# The cleanup.yml playbook should be used to remove any test infrastructure that was created by this test process -# and is not present within the instance itself (IE: the docker container created by Molecule). For example, it -# could be used to remove AWS infrastructure created as part of this test and that should not persist. - -- name: Remove external test infrastructure - hosts: molecule - tasks: - - name: Cleanup tasks not configured - delegate_to: localhost - ansible.builtin.debug: - msg: Add your cleanup tasks here as required! - diff --git a/roles/ec2_platform/molecule/role-ec2_platform/collections.yml b/roles/ec2_platform/molecule/role-ec2_platform/collections.yml deleted file mode 100644 index 4fdc1bc..0000000 --- a/roles/ec2_platform/molecule/role-ec2_platform/collections.yml +++ /dev/null @@ -1,10 +0,0 @@ ---- - -collections: - - name: community.docker -# - name: git+https://github.com/syndr/ansible-collection-molecule.git -# type: git -# version: latest - - name: syndr.molecule - version: 1.4.0-dev - diff --git a/roles/ec2_platform/molecule/role-ec2_platform/create.yml b/roles/ec2_platform/molecule/role-ec2_platform/create.yml deleted file mode 100644 index 2815558..0000000 --- a/roles/ec2_platform/molecule/role-ec2_platform/create.yml +++ /dev/null @@ -1,58 +0,0 @@ ---- -- name: Create - hosts: localhost - gather_facts: false - tasks: - - name: Create platform(s) - ansible.builtin.include_role: - name: syndr.molecule.platform - vars: - - platform_name: "{{ item.name }}" - platform_state: present - platform_type: "{{ item.type }}" - platform_molecule_cfg: "{{ item }}" - loop: "{{ molecule_yml.platforms }}" - loop_control: - label: item.name - -# We want to avoid errors like "Failed to create temporary directory" -- name: Validate that inventory was refreshed - hosts: molecule - gather_facts: false - tasks: - - name: Check uname - ansible.builtin.raw: uname -a - register: result - changed_when: false - - - name: Display uname info - ansible.builtin.debug: - msg: "{{ result.stdout }}" - - - name: Load system facts - ansible.builtin.setup: - filter: - - ansible_service_mgr - - - name: Check on Systemd - block: - - name: Wait for systemd to complete initialization. - ansible.builtin.command: systemctl is-system-running - register: systemctl_status - until: > - 'running' in systemctl_status.stdout or - 'degraded' in systemctl_status.stdout - retries: 30 - delay: 5 - changed_when: false - failed_when: systemctl_status.rc > 1 - - - name: Check systemd status - ansible.builtin.assert: - that: - - systemctl_status.stdout == 'running' - fail_msg: Systemd-enabled container does not have a healthy Systemd! - success_msg: Systemd is running - when: ansible_service_mgr == 'systemd' - diff --git a/roles/ec2_platform/molecule/role-ec2_platform/destroy.yml b/roles/ec2_platform/molecule/role-ec2_platform/destroy.yml deleted file mode 100644 index 6569c07..0000000 --- a/roles/ec2_platform/molecule/role-ec2_platform/destroy.yml +++ /dev/null @@ -1,18 +0,0 @@ ---- - -- name: Perform cleanup - hosts: localhost - gather_facts: false - tasks: - - name: Remove platform(s) - ansible.builtin.include_role: - name: syndr.molecule.platform - vars: - platform_name: "{{ item.name }}" - platform_state: absent - platform_type: "{{ item.type }}" - platform_molecule_cfg: "{{ item }}" - loop: "{{ molecule_yml.platforms }}" - loop_control: - label: item.name - diff --git a/roles/ec2_platform/molecule/role-ec2_platform/init.yml b/roles/ec2_platform/molecule/role-ec2_platform/init.yml deleted file mode 100644 index 1da5128..0000000 --- a/roles/ec2_platform/molecule/role-ec2_platform/init.yml +++ /dev/null @@ -1,10 +0,0 @@ ---- -# Initialize a Molecule scenario for use within a role - -- name: Provision file structure - hosts: localhost - tasks: - - name: Launch provisioner - ansible.builtin.include_role: - name: syndr.molecule.init - diff --git a/roles/ec2_platform/molecule/role-ec2_platform/side_effect.yml b/roles/ec2_platform/molecule/role-ec2_platform/side_effect.yml deleted file mode 100644 index 4d1d7af..0000000 --- a/roles/ec2_platform/molecule/role-ec2_platform/side_effect.yml +++ /dev/null @@ -1,10 +0,0 @@ ---- -# The side effect playbook executes actions which produce side effects to the instances(s). Intended to test HA failover scenarios or the like. - -- name: Test side effects - hosts: molecule - tasks: - - name: No side effect tests configured - ansible.builtin.debug: - msg: Add side-effect tests here! - diff --git a/roles/ec2_platform/molecule/role-ec2_platform/verify.yml b/roles/ec2_platform/molecule/role-ec2_platform/verify.yml deleted file mode 100644 index 3237d2c..0000000 --- a/roles/ec2_platform/molecule/role-ec2_platform/verify.yml +++ /dev/null @@ -1,21 +0,0 @@ ---- -# Verify that the role being tested has done what it's supposed to - -- name: Verify - hosts: molecule - tasks: - - name: Load local host facts - ansible.builtin.setup: - gather_subset: - - '!all' - - '!min' - - local - - - name: Load test data (example) - ansible.builtin.set_fact: - test_prepare_fact: "{{ ansible_local.molecule.test_prepare_fact }}" - - - name: Add your verification tasks here - ansible.builtin.debug: - msg: "IE: For a 'users' role, check that the test user exists" - diff --git a/roles/ec2_platform/tasks/present.yml b/roles/ec2_platform/tasks/present.yml index 02a0212..f533b0d 100644 --- a/roles/ec2_platform/tasks/present.yml +++ b/roles/ec2_platform/tasks/present.yml @@ -91,6 +91,8 @@ description: "{{ ec2_platform.security_group_description }}" rules: "{{ ec2_platform.security_group_rules }}" rules_egress: "{{ ec2_platform.security_group_rules_egress }}" + tags: + Name: "{{ ec2_platform.security_group_name }}" vars: vpc_subnet: "{{ __ec2_platform_subnet_info.results[0].subnets[0] }}" when: ec2_platform.security_groups | length == 0 @@ -161,6 +163,11 @@ search_regex: SSH delay: 10 timeout: 320 + msg: "SSH connectivity check to {{ ec2_platform_instance_config.address }} on port {{ ec2_platform_instance_config.port }} failed" + register: __ec2_platform_ssh_connectivity_check + until: "'Connection reset by peer' not in __ec2_platform_ssh_connectivity_check.module_stderr | default('')" + retries: 6 + delay: 10 # TODO: Add an actual check here instead of only waiting - name: Wait for boot process to finish diff --git a/roles/init/defaults/main.yml b/roles/init/defaults/main.yml index be8a275..d561e71 100644 --- a/roles/init/defaults/main.yml +++ b/roles/init/defaults/main.yml @@ -10,7 +10,7 @@ init_platform_type: docker # Version of this collection that should be used by the Molecule test # - Set to "" to attempt to use the running version -init_collection_version: "" +init_collection_version: latest # Source of the collection that this role is part of (galaxy, git) init_collection_source: git @@ -42,6 +42,24 @@ init_platforms: [] # Create backups of any files that would be clobbered by running this role init_file_backup: true +# Default configuration to be added to generated molecule.yml if init_platforms is not defined +init_platform_defaults: + docker: + - name: docker-rockylinux9 + type: docker + config: + image: "geerlingguy/docker-rockylinux9-ansible:latest" + systemd: true + ec2: + - name: ec2-rockylinux9 + type: ec2 + config: + image: "ami-01bd836275f79352c" + instance_type: "t3.micro" + region: "us-east-2" + vpc_id: "vpc-12345678" + vpc_subnet_id: "subnet-12345678" + # Path to the ansible secret file that should be used by the Molecule test # - Variable substitution can be used as described here: https://ansible.readthedocs.io/projects/molecule/configuration/#variable-substitution # - Set to "" to disable diff --git a/roles/init/files/init.yml b/roles/init/files/init.yml index 59fcf5c..a2bbd1e 100644 --- a/roles/init/files/init.yml +++ b/roles/init/files/init.yml @@ -12,4 +12,7 @@ init_platform_type: docker # Supported collection sources are: git, galaxy init_collection_source: git + # Version of this collection that should be used by the Molecule test + # - Set to "" to attempt to use the running version + init_collection_version: latest diff --git a/roles/init/tasks/asserts.yml b/roles/init/tasks/asserts.yml index 30dd9aa..b211f78 100644 --- a/roles/init/tasks/asserts.yml +++ b/roles/init/tasks/asserts.yml @@ -29,6 +29,7 @@ that: - __init_item.stat.exists is true fail_msg: Specified file path does not exist! + quiet: true loop: "{{ __init_filepath_stat.results }}" loop_control: label: "{{ __init_item.stat.path }}" diff --git a/roles/init/tasks/main.yml b/roles/init/tasks/main.yml index 85300ce..f58e1d2 100644 --- a/roles/init/tasks/main.yml +++ b/roles/init/tasks/main.yml @@ -10,30 +10,16 @@ - name: Build base platform definition when: init_platforms is not truthy - # TODO: Define these values in the role defaults block: - name: Build base docker platform definition when: init_platform_type == 'docker' ansible.builtin.set_fact: - init_platforms: - - name: docker-rockylinux9 - type: docker - config: - image: "geerlingguy/docker-rockylinux9-ansible:latest" - systemd: true + init_platforms: "{{ init_platforms | default(init_platform_defaults.docker, true) }}" - name: Build base ec2 platform definition when: init_platform_type == 'ec2' ansible.builtin.set_fact: - init_platforms: - - name: ec2-rockylinux9 - type: ec2 - config: - image: "ami-067daee80a6d36ac0" - instance_type: "t3.micro" - region: "us-east-2" - vpc_id: "vpc-12345678" - vpc_subnet_id: "subnet-12345678" + init_platforms: "{{ init_platforms | default(init_platform_defaults.ec2, true) }}" - name: Platform definition is valid ansible.builtin.assert: @@ -70,9 +56,9 @@ - name: Set collection information ansible.builtin.set_fact: - init_collection_version: >- - "{{ init_collection_version if init_collection_version is truthy - else __init_collection_meta.collection_info.version | default(init_collection_defaults.version) }}" + __init_collection_version: >- + {{ init_collection_version if init_collection_version is truthy + else __init_collection_meta.collection_info.version | default(init_collection_defaults.version) }} __init_collection_name: "{{ __init_collection_meta.collection_info.name | default(init_collection_defaults.name) }}" __init_collection_namespace: "{{ __init_collection_meta.collection_info.namespace | default(init_collection_defaults.namespace) }}" __init_collection_repository: "{{ __init_collection_meta.collection_info.repository | default(init_collection_defaults.repository) }}" diff --git a/roles/init/templates/collections.yml.j2 b/roles/init/templates/collections.yml.j2 index 6dde64e..5f634a3 100644 --- a/roles/init/templates/collections.yml.j2 +++ b/roles/init/templates/collections.yml.j2 @@ -5,8 +5,9 @@ collections: {% if init_collection_source == 'git' %} - name: git+{{ __init_collection_repository }}.git type: git + version: v{{ __init_collection_version }} {% elif init_collection_source == 'galaxy' %} - name: {{ __init_collection_namespace }}.{{ __init_collection_name }} + version: {{ __init_collection_version }} {% endif %} - version: {{ init_collection_version }} diff --git a/roles/init/templates/create.yml.j2 b/roles/init/templates/create.yml.j2 index fe9ce25..a73f31c 100644 --- a/roles/init/templates/create.yml.j2 +++ b/roles/init/templates/create.yml.j2 @@ -52,7 +52,7 @@ ansible.builtin.assert: that: - systemctl_status.stdout == 'running' - fail_msg: Systemd-enabled container does not have a healthy Systemd! + fail_msg: Systemd-enabled host does not have a healthy Systemd! success_msg: Systemd is running when: ansible_service_mgr == 'systemd' diff --git a/roles/init/templates/destroy.yml.j2 b/roles/init/templates/destroy.yml.j2 index 2d49265..a5aadf8 100644 --- a/roles/init/templates/destroy.yml.j2 +++ b/roles/init/templates/destroy.yml.j2 @@ -8,7 +8,7 @@ ansible.builtin.include_role: name: {{ ansible_collection_name }}.platform vars: -{% raw %} +{%- raw %} platform_name: "{{ item.name }}" platform_state: absent platform_type: "{{ item.type }}" @@ -16,5 +16,5 @@ loop: "{{ molecule_yml.platforms }}" loop_control: label: item.name -{% endraw %} +{%- endraw %} diff --git a/roles/init/templates/molecule.yml.j2 b/roles/init/templates/molecule.yml.j2 index c9302b9..5367117 100644 --- a/roles/init/templates/molecule.yml.j2 +++ b/roles/init/templates/molecule.yml.j2 @@ -21,6 +21,7 @@ platforms: modify_image_buildpath: {{ init_platform.config.modify_image_buildpath }} {% endif %} privileged: {{ init_platform.config.privileged | default(false) }} + hostvars: {} {% endif %} {% if init_platform.type == 'ec2' %} image: {{ init_platform.config.image }} @@ -34,11 +35,13 @@ provisioner: name: ansible log: True playbooks: + create: create.yml prepare: prepare.yml converge: converge.yml side_effect: side_effect.yml verify: verify.yml cleanup: cleanup.yml + destroy: destroy.yml config_options: defaults: gathering: explicit diff --git a/roles/init/templates/prepare.yml.j2 b/roles/init/templates/prepare.yml.j2 index e16f3fc..0d588fd 100644 --- a/roles/init/templates/prepare.yml.j2 +++ b/roles/init/templates/prepare.yml.j2 @@ -13,7 +13,6 @@ - name: Prepare target host for execution hosts: molecule tags: always - become: true tasks: ## # Creating an admin service account for Molecule/Ansible to use for testing @@ -29,6 +28,7 @@ ## {% raw %} - name: Create ansible service account + become: true vars: molecule_user: molecule_runner block: diff --git a/roles/init/vars/main.yml b/roles/init/vars/main.yml index ab1d901..59e7c45 100644 --- a/roles/init/vars/main.yml +++ b/roles/init/vars/main.yml @@ -13,4 +13,5 @@ __init_supported_project_types: # The platform types supported by this role __init_supported_platform_types: - docker + - ec2 diff --git a/roles/platform/tasks/ansible_inventory.yml b/roles/platform/tasks/ansible_inventory.yml new file mode 100644 index 0000000..77878f3 --- /dev/null +++ b/roles/platform/tasks/ansible_inventory.yml @@ -0,0 +1,56 @@ +--- +# Build an ansible inventory data structure +# +# This is a dictionary such as: +# +# all: +# children: +# molecule: +# hosts: +# instance_host_1: +# instance_host_2: +# vars: +# ansible_user: molecule_runner +# ansible_ssh_private_key_file: /path/to/private_key +# +# Expected variables: +# - __platform_target_instance_config: A dictionary containing the new instance configuration +# - __platform_ansible_hostvars: A dictionary containing additional hostvars to add to the instance configuration +# - __platform_current_molecule_inventory: The current molecule inventory contents +# +# Output +# - __platform_molecule_inventory: The updated molecule inventory contents +# + +# TODO: Omit the `ec2_tag_vars` key itself so that it doesn't get written to the inventory file (duplicating values) +- name: Host exists in inventory file + when: platform_state == 'present' + vars: + # NOTE: Has to be inside jinja so that we can set key name + __platform_molecule_inventory_partial: "{{ { + 'all': { + 'children': { + 'molecule': { + 'hosts': { + __platform_target_instance_config.instance: __platform_ansible_hostvars[__platform_target_instance_config.instance] + | combine(platform_molecule_cfg.hostvars | default({}) + | combine(__platform_ansible_hostvars[__platform_target_instance_config.instance].ec2_tag_vars | default({}))) + }}}}} }}" + ansible.builtin.set_fact: + __platform_molecule_inventory: "{{ __platform_molecule_inventory | from_yaml | default({}) + | combine(__platform_molecule_inventory_partial, recursive=true) }}" + +- name: Host does not exist in inventory file + when: + - platform_state == 'absent' + - __platform_molecule_inventory is truthy + ansible.builtin.set_fact: + __platform_molecule_inventory: "{{ + __platform_molecule_inventory | combine({ + 'all': { + 'children': { + 'molecule': { + 'hosts': (__platform_molecule_inventory.all.children.molecule.hosts | default({}) | + dict2items | rejectattr('key', 'equalto', __platform_target_instance_config.instance) | items2dict) + }}}}, recursive=true) }}" + diff --git a/roles/platform/tasks/instance_config.yml b/roles/platform/tasks/instance_config.yml new file mode 100644 index 0000000..22e069c --- /dev/null +++ b/roles/platform/tasks/instance_config.yml @@ -0,0 +1,85 @@ +--- +# Add an instance to the list of Molecule hosts +# +# Expected variables: +# - __platform_instance_config: The current list of instance configurations +# - __platform_target_instance_config: The configuration for the new instance +# - __platform_ansible_hostvars: Additional hostvars to add to the instance configuration +# +# Outputs: +# - __platform_instance_config: The updated list of instance configurations +# - __platform_instance_config_update_needed: Whether the instance configuration file needs to be updated +# + + +- name: We are adding/updating an instance + when: platform_state == 'present' + block: + - name: Instance configuration {{ __platform_target_instance_config.instance }} is valid + ansible.builtin.assert: + that: + - __platform_target_instance_config is defined + - __platform_target_instance_config.instance is string + fail_msg: "Instance configuration for {{ __platform_target_instance_config.instance }} failed! Check the platform configuration." + success_msg: "Instance configuration for {{ __platform_target_instance_config.instance }} is defined" + + - name: Add user-specified hostvars to instance configuration + when: + - platform_molecule_cfg.hostvars is defined + - platform_molecule_cfg.hostvars is truthy + ansible.builtin.set_fact: + __platform_ansible_hostvars: "{{ __platform_ansible_hostvars | combine(platform_molecule_cfg.hostvars, recursive=true) }}" + + - name: 🪲 Current instance config + ansible.builtin.debug: + var: __platform_instance_config + verbosity: 1 + + - name: 🪲 New instance config + ansible.builtin.debug: + var: __platform_target_instance_config + verbosity: 1 + + - name: 🪲 Ansible hostvars + ansible.builtin.debug: + var: __platform_ansible_hostvars + verbosity: 1 + + - name: Instance name matching this already exists in configuration + when: + - __platform_instance_config is truthy + - __platform_target_instance_config.instance in __platform_instance_config | map(attribute='instance') | list + block: + - name: Mark config update as unneeded + ansible.builtin.set_fact: + __platform_instance_config_update_needed: false + + - name: Existing configuration does not match desired + when: __platform_target_instance_config != (__platform_instance_config | + selectattr('instance', 'equalto', __platform_target_instance_config.instance) | list | first) + block: + - name: Remove existing {{ __platform_target_instance_config.instance }} configuration (does not match) + ansible.builtin.set_fact: + __platform_instance_config: "{{ + __platform_instance_config | rejectattr('instance', 'equalto', __platform_target_instance_config.instance) | list }}" + __platform_instance_config_update_needed: true + + - name: Append new instance to config + ansible.builtin.set_fact: + __platform_instance_config: >- + {{ __platform_instance_config | default([], true) + [__platform_target_instance_config] + if __platform_target_instance_config | default(false, true) is truthy + else __platform_instance_config }} + +- name: Remove instance configuration + when: platform_state == 'absent' + block: + - name: Remove existing {{ __platform_target_instance_config.instance }} configuration + when: + - __platform_instance_config is truthy + - __platform_target_instance_config.instance in __platform_instance_config | map(attribute='instance') | list + ansible.builtin.set_fact: + __platform_instance_config: "{{ + __platform_instance_config | rejectattr('instance', 'equalto', __platform_target_instance_config.instance) | list }}" + __platform_instance_config_update_needed: true + diff --git a/roles/platform/tasks/inventory.yml b/roles/platform/tasks/inventory.yml index 3e5fcde..4e81113 100644 --- a/roles/platform/tasks/inventory.yml +++ b/roles/platform/tasks/inventory.yml @@ -1,127 +1,82 @@ --- # Add a host to the Molecule inventory -- name: Load existing instance configuration - block: - - name: Load existing instance configuration file - ansible.builtin.slurp: - src: "{{ platform_molecule_ephemeral_directory }}/instance_config.yml" - register: __platform_current_instance_config_b64 - ignore_errors: true - - - name: Decode instance configuration data - ansible.builtin.set_fact: - __platform_current_instance_config: "{{ __platform_current_instance_config_b64.content | default('') | b64decode | from_yaml }}" - - name: Generate new instance configuration when: platform_state == 'present' block: - name: Generate {{ platform_name }} instance configuration (Docker) when: platform_type == 'docker' ansible.builtin.set_fact: - __platform_new_instance_config: "{{ { + __platform_new_instance_configs: "{{ [{ 'instance': platform_name, 'connection': 'docker' - } }}" - __platform_ansible_hostvars: - ansible_connection: "community.docker.docker" + }] }}" + __platform_ansible_hostvars: "{{ { + platform_name: { + 'ansible_connection': 'community.docker.docker' + } } }}" - name: Generate {{ platform_name }} instance configuration (EC2) when: platform_type == 'ec2' ansible.builtin.set_fact: # NOTE: This depends on the ec2_platform_instance_config being set by the ec2_platform role - __platform_new_instance_config: "{{ { - 'instance': platform_name, - 'address': ec2_platform_instance_config.address, - 'user': ec2_platform_instance_config.user, - 'port': ec2_platform_instance_config.port, - 'identity_file': ec2_platform_instance_config.identity_file, - 'instance_ids': ec2_platform_instance_config.instance_ids - } }}" + __platform_new_instance_configs: "{{ [ec2_platform_instance_config] }}" __platform_ansible_connection: "ssh" - __platform_ansible_hostvars: - ansible_connection: "ssh" - ansible_host: "{{ ec2_platform_instance_config.address }}" - ansible_port: "{{ ec2_platform_instance_config.port }}" - ansible_user: "{{ ec2_platform_instance_config.user }}" - ansible_ssh_private_key_file: "{{ ec2_platform_instance_config.identity_file }}" - - - name: Instance configuration {{ platform_name }} is valid - ansible.builtin.assert: - that: - - __platform_new_instance_config is defined - - __platform_new_instance_config.instance is string - fail_msg: "Instance configuration for {{ platform_name }} failed! Check the platform configuration." - success_msg: "Instance configuration for {{ platform_name }} is defined" - - - name: 🪲 Current instance config + __platform_ansible_hostvars: "{{ { + ec2_platform_instance_config.instance: { + 'ansible_connection': 'ssh', + 'ansible_host': ec2_platform_instance_config.address, + 'ansible_port': ec2_platform_instance_config.port, + 'ansible_user': ec2_platform_instance_config.user, + 'ansible_ssh_private_key_file': ec2_platform_instance_config.identity_file + } } }}" + + - name: Show hostvars configuration ansible.builtin.debug: - var: __platform_current_instance_config + var: __platform_ansible_hostvars verbosity: 1 - - name: Instance name matching this already exists in configuration - when: - - __platform_current_instance_config is truthy - - platform_name in __platform_current_instance_config | map(attribute='instance') | list - block: - - name: Mark config update as unneeded - ansible.builtin.set_fact: - __platform_instance_config_update_needed: false - - - name: Existing configuration does not match desired - when: __platform_new_instance_config != (__platform_current_instance_config | selectattr('instance', 'equalto', platform_name) | list | first) - block: - - name: Remove existing {{ platform_name }} configuration (does not match) - ansible.builtin.set_fact: - __platform_current_instance_config: "{{ - __platform_current_instance_config | rejectattr('instance', 'equalto', platform_name) | list }}" - __platform_instance_config_update_needed: true - -- name: Remove instance configuration - when: platform_state == 'absent' +- name: Load existing instance configuration block: - - name: Remove existing {{ platform_name }} configuration - when: - - __platform_current_instance_config is truthy - - platform_name in __platform_current_instance_config | map(attribute='instance') | list - ansible.builtin.set_fact: - __platform_current_instance_config: "{{ - __platform_current_instance_config | rejectattr('instance', 'equalto', platform_name) | list }}" - __platform_instance_config_update_needed: true - -- name: dump new instance config - ansible.builtin.debug: - var: __platform_new_instance_config - ignore_errors: true - -- name: dump current instance config - ansible.builtin.debug: - var: __platform_current_instance_config - ignore_errors: true - -- name: Write {{ platform_name }} instance config file - when: - - __platform_instance_config_update_needed - - __platform_current_instance_config | default(false, true)is truthy or __platform_new_instance_config | default(false, true) is truthy - vars: - __platform_instance_config: >- - {{ __platform_current_instance_config | default([], true) + [__platform_new_instance_config] - if __platform_new_instance_config | default(false, true) is truthy - else __platform_current_instance_config }} - ansible.builtin.copy: - content: "{{ __platform_instance_config | to_nice_yaml(indent=2) }}" - dest: "{{ platform_molecule_ephemeral_directory }}/instance_config.yml" - mode: "0600" + - name: Load existing instance configuration file + ansible.builtin.slurp: + src: "{{ platform_molecule_ephemeral_directory }}/instance_config.yml" + register: __platform_current_instance_config_b64 + failed_when: false -# If the file would be empty, remove it -- name: Remove molecule instance config file - when: - - platform_state == 'absent' - - __platform_current_instance_config | default(false) is not truthy - - __platform_new_instance_config | default(false) is not truthy - ansible.builtin.file: - path: "{{ platform_molecule_ephemeral_directory }}/instance_config.yml" - state: absent + - name: Decode instance configuration data + ansible.builtin.set_fact: + __platform_instance_config: "{{ __platform_current_instance_config_b64.content | default('') | b64decode | from_yaml }}" + +# Build the Molecule instance configuration object +- name: Process instance configuration + when: __platform_instance_config is truthy or __platform_new_instance_configs is defined + ansible.builtin.include_tasks: "{{ role_path }}/tasks/instance_config.yml" + loop: "{{ __platform_new_instance_configs if __platform_new_instance_configs is defined else __platform_instance_config | default([], true) }}" + loop_control: + loop_var: __platform_target_instance_config + label: "{{ __platform_target_instance_config.instance }}" + +- name: Manage molecule instance config file + block: + - name: Write {{ platform_name }} instance config file + when: + - __platform_instance_config_update_needed + - __platform_instance_config | default(false, true)is truthy or __platform_new_instance_config | default(false, true) is truthy + ansible.builtin.copy: + content: "{{ __platform_instance_config | to_nice_yaml(indent=2) }}" + dest: "{{ platform_molecule_ephemeral_directory }}/instance_config.yml" + mode: "0600" + + # If the file would be empty, remove it + - name: Remove molecule instance config file + when: + - platform_state == 'absent' + - __platform_current_instance_config | default(false) is not truthy + - __platform_new_instance_config | default(false) is not truthy + ansible.builtin.file: + path: "{{ molecule_instance_config }}" + state: absent - name: Load existing molecule inventory when: __platform_run_count | int > 1 or platform_state == 'absent' @@ -130,41 +85,27 @@ ansible.builtin.slurp: src: "{{ platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" register: __platform_current_molecule_inventory_b64 - ignore_errors: true + failed_when: false - name: Decode instance configuration data ansible.builtin.set_fact: - __platform_current_molecule_inventory: "{{ __platform_current_molecule_inventory_b64.content | default({}) | b64decode | from_yaml }}" + __platform_molecule_inventory: "{{ __platform_current_molecule_inventory_b64.content | default({}) | b64decode | from_yaml }}" -- name: Add {{ platform_name }} to molecule_inventory - when: platform_state == 'present' - vars: - __platform_inventory_partial: "{{ { - 'all': { - 'children': { - 'molecule': { - 'hosts': { - platform_name: __platform_ansible_hostvars - }}}}} }}" - ansible.builtin.set_fact: - __platform_molecule_inventory: > - {{ __platform_current_molecule_inventory | from_yaml | default({}) | combine(__platform_inventory_partial, recursive=true) }} - -- name: Remove {{ platform_name }} from molecule_inventory - when: - - platform_state == 'absent' - - __platform_current_molecule_inventory is truthy - ansible.builtin.set_fact: - __platform_molecule_inventory: "{{ - __platform_current_molecule_inventory | combine({ - 'all': { - 'children': { - 'molecule': { - 'hosts': (__platform_current_molecule_inventory.all.children.molecule.hosts | default({}) | - dict2items | rejectattr('key', 'equalto', platform_name) | items2dict) - }}}}, recursive=true) }}" +- name: Show instance configuration + ansible.builtin.debug: + var: __platform_instance_config + verbosity: 1 + +- name: Process instance inventory membership + when: __platform_instance_config | default(false) is truthy or __platform_new_instance_configs | default(false) is truthy + ansible.builtin.include_tasks: "{{ role_path }}/tasks/ansible_inventory.yml" + loop: "{{ __platform_new_instance_configs if __platform_new_instance_configs is defined else __platform_instance_config | default([], true) }}" + loop_control: + loop_var: __platform_target_instance_config + label: "{{ __platform_target_instance_config.instance }}" - name: Write {{ platform_name }} to molecule inventory file + when: __platform_molecule_inventory is truthy ansible.builtin.copy: content: | {{ __platform_molecule_inventory | default({}) | to_nice_yaml(indent=2) }} @@ -175,7 +116,9 @@ ansible.builtin.meta: refresh_inventory - name: Fail if molecule group is missing - when: __platform_molecule_inventory is defined + when: + - __platform_molecule_inventory is defined + - platform_state == 'present' ansible.builtin.assert: that: "'molecule' in groups" fail_msg: | diff --git a/roles/platform/vars/main.yml b/roles/platform/vars/main.yml index aa4a248..917f1d5 100644 --- a/roles/platform/vars/main.yml +++ b/roles/platform/vars/main.yml @@ -10,3 +10,6 @@ __platform_instance_config_update_needed: true # Default connection method for hosts -- update as needed in tasks/inventory.yml __platform_ansible_connection: "ssh" +# The current molecule inventory contents +__platform_current_molecule_inventory: {} + From c4ec981bc644d508863a6e3879cd201cdf3a1ac6 Mon Sep 17 00:00:00 2001 From: syndr Date: Thu, 9 Jan 2025 01:39:29 -0700 Subject: [PATCH 11/18] Refactor docker_platform role logic, begin updating platform role logic Async container management - process full platform list if provided Updates to 'platform' role to support this --- galaxy.yml | 2 +- molecule/default/create.yml | 8 - molecule/default/molecule.yml | 13 +- molecule/resources/create.yml | 8 - molecule/resources/destroy.yml | 1 - roles/docker_platform/README.md | 153 ++++++++++-- roles/docker_platform/defaults/main.yml | 131 ++++++++-- roles/docker_platform/tasks/absent.yml | 58 +++-- roles/docker_platform/tasks/create.yml | 117 --------- roles/docker_platform/tasks/custom_image.yml | 67 ++++++ roles/docker_platform/tasks/main.yml | 3 + roles/docker_platform/tasks/present.yml | 238 +++++++++++++------ roles/docker_platform/tasks/validate_cfg.yml | 23 ++ roles/docker_platform/vars/main.yml | 16 ++ roles/platform/defaults/main.yml | 9 +- roles/platform/tasks/deprovision.yml | 48 ++-- roles/platform/tasks/inventory.yml | 82 ++++--- roles/platform/tasks/main.yml | 31 ++- roles/platform/tasks/provision.yml | 61 +++-- roles/platform/tasks/validate_cfg.yml | 31 +++ roles/platform/vars/main.yml | 8 +- 21 files changed, 740 insertions(+), 368 deletions(-) delete mode 100644 roles/docker_platform/tasks/create.yml create mode 100644 roles/docker_platform/tasks/custom_image.yml create mode 100644 roles/docker_platform/tasks/validate_cfg.yml create mode 100644 roles/platform/tasks/validate_cfg.yml diff --git a/galaxy.yml b/galaxy.yml index 9b81e8e..4590aad 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -8,7 +8,7 @@ namespace: syndr name: molecule # The version of the collection. Must be compatible with semantic versioning -version: 1.5.0 +version: 1.6.0 # The path to the Markdown (.md) readme file. This path is relative to the root of the collection readme: README.md diff --git a/molecule/default/create.yml b/molecule/default/create.yml index b05d21e..d1707c6 100644 --- a/molecule/default/create.yml +++ b/molecule/default/create.yml @@ -6,14 +6,6 @@ - name: Create platform ansible.builtin.include_role: name: syndr.molecule.platform - vars: - platform_name: "{{ item.name }}" - platform_state: present - platform_type: "{{ item.type }}" - platform_molecule_cfg: "{{ item }}" - loop: "{{ molecule_yml.platforms }}" - loop_control: - label: item.name # We want to avoid errors like "Failed to create temporary directory" - name: Validate that inventory was refreshed diff --git a/molecule/default/molecule.yml b/molecule/default/molecule.yml index 7afbbd0..8d2e453 100644 --- a/molecule/default/molecule.yml +++ b/molecule/default/molecule.yml @@ -8,7 +8,7 @@ driver: managed: true login_cmd_template: 'docker exec -ti {instance} bash' platforms: - - name: docker-rockylinux9 + - name: ansible-collection-molecule-docker-rockylinux9 type: docker image: geerlingguy/docker-rockylinux9-ansible:latest systemd: True @@ -16,7 +16,7 @@ platforms: privileged: False hostvars: test_hostvar: test - - name: docker-fedora41 + - name: ansible-collection-molecule-docker-fedora41 type: docker image: geerlingguy/docker-fedora41-ansible:latest systemd: True @@ -24,7 +24,7 @@ platforms: privileged: False hostvars: test_hostvar: test - - name: docker-ubuntu2204 + - name: ansible-collection-molecule-docker-ubuntu2204 type: docker image: geerlingguy/docker-ubuntu2204-ansible:latest systemd: True @@ -47,6 +47,13 @@ provisioner: defaults: gathering: explicit verbosity: ${ANSIBLE_VERBOSITY:-0} + env: + ARA_API_CLIENT: ${ARA_API_CLIENT:-'http'} + ARA_API_SERVER: ${ARA_API_SERVER:-'http://localhost:8000'} + ARA_DEFAULT_LABELS: ${ARA_DEFAULT_LABELS:-'testing,molecule'} + # To use Ara with molecule: + # export the ANSIBLE_CALLBACK_PLUGINS env var with the output of 'python3 -m ara.setup.callback_plugins' + ANSIBLE_CALLBACK_PLUGINS: ${ANSIBLE_CALLBACK_PLUGINS} scenario: create_sequence: - dependency diff --git a/molecule/resources/create.yml b/molecule/resources/create.yml index 56f8790..db09aa7 100644 --- a/molecule/resources/create.yml +++ b/molecule/resources/create.yml @@ -6,14 +6,6 @@ - name: Create test platform(s) ansible.builtin.include_role: name: syndr.molecule.platform - vars: - platform_name: "{{ item.name }}" - platform_state: present - platform_type: "{{ item.type }}" - platform_molecule_cfg: "{{ item }}" - loop: "{{ molecule_yml.platforms }}" - loop_control: - label: item.name # We want to avoid errors like "Failed to create temporary directory" - name: Validate that inventory was refreshed diff --git a/molecule/resources/destroy.yml b/molecule/resources/destroy.yml index 6569c07..2157a1d 100644 --- a/molecule/resources/destroy.yml +++ b/molecule/resources/destroy.yml @@ -8,7 +8,6 @@ ansible.builtin.include_role: name: syndr.molecule.platform vars: - platform_name: "{{ item.name }}" platform_state: absent platform_type: "{{ item.type }}" platform_molecule_cfg: "{{ item }}" diff --git a/roles/docker_platform/README.md b/roles/docker_platform/README.md index 750eddc..3ea4029 100644 --- a/roles/docker_platform/README.md +++ b/roles/docker_platform/README.md @@ -11,13 +11,16 @@ Required configuration options are: - `name`: Name of the platform (string) - `type`: `docker` -- `image`: Docker image to use for the platform (string) -Optional configuration options are: +Optional configuration options of note are: +- `image`: Docker image to use for the platform (string) - `systemd`: Whether the container should be started with SystemD enabled (boolean) -- `modify_image`: Whether the provided image should be modified at runtime (boolean) -- `modify_image_buildpath`: Path to Docker build files that should be used to modify the image (string) +- `privileged`: Whether the container should be started in privileged mode (boolean) +- `cpus`: Number of CPUs to allocate to the container (integer) +- `memory`: Amount of memory to allocate to the container (string) +- `pull`: Pull policy to use for the image (string) +- `volumes`: List of volumes to mount in the container (list) Requirements ------------ @@ -28,6 +31,126 @@ Requirements Role Variables -------------- +Configuration that can be set in the `platforms` section of the `molecule.yml` file in your Molecule scenario directory: + +```yaml +# Name of the platform in the Molecule configuration +name: Your platform name + +# Image to use for the container +image: "geerlingguy/docker-rockylinux9-ansible:latest" + +# Run the container in privileged mode (WARNING: Less secure!) +privileged: false + +# List of capabilities to add to the container +capabilities: [] + +# List of capabilities to drop from the container +cap_drop: [] + +# Command to run in the container +command: "" + +# Number of CPUs to allocate to the container +cpus: 2 + +# List of device path and read rate (bytes per second) from device. +# Each entry should be a dictionary with keys 'path' and 'rate'. +device_read_bps: [] + +# List of device path and write rate (bytes per second) to device. +# Each entry should be a dictionary with keys 'path' and 'rate'. +device_write_bps: [] + +# List of device path and read rate (IO per second) from device. +# Each entry should be a dictionary with keys 'path' and 'rate'. +device_read_iops: [] + +# List of device path and write rate (IO per second) to device. +# Each entry should be a dictionary with keys 'path' and 'rate'. +device_write_iops: [] + +# List of host device bindings to add to the container. +# Each binding is a mapping expressed in the format: +# :: +devices: [] + +# List of DNS options +dns_opts: [] + +# List of DNS search domains +dns_search_domains: [] + +# List of DNS servers +dns_servers: [] + +# The URL or Unix socket path used to connect to the Docker API +docker_host: "{{ lookup('env', 'DOCKER_HOST') | default('unix:///var/run/docker.sock') }}" + +# Path to a file, present on the controller, containing environment variables FOO=BAR +env_file: "" + +# Dict of host-to-IP mappings, where each host name is a key in the dictionary. Each host name will be added to the container’s /etc/hosts file. +# Instead of an IP address, the special value host-gateway can also be used, which resolves to the host’s gateway +# IP and allows containers to connect to services running on the host. +etc_hosts: {} + +# Dict of labels to apply to the container +labels: {} + +# Memory limit in format []. +# Number is a positive integer. +# Unit can be B (byte), K (kibibyte, 1024B), M (mebibyte), G (gibibyte), T (tebibyte), or P (pebibyte). +memory: "0" + +# Connect the container to a network. +# Choices are bridge, host, none, container:, or default. +network_mode: default + +# List of ports to publish from the container to the host. +# Use docker CLI syntax: 8000, 9000:8000, or 0.0.0.0:9000:8000, +# where 8000 is a container port, 9000 is a host port, and 0.0.0.0 is a host interface. +published_ports: [] + +# Pull setting for the container image (never,missing,always) +pull: always + +# List of security options in the form of "label:user:User" +security_opts: [] + +# State of the container +state: started + +# Whether the container runs systemd +systemd: false + +# Attempt to execute systemd in the container on start +# WARNING: +# - This can cause issues with some containers +# - Not required if the container is already built with systemd running as PID 1 +# - Expects the container to have systemd installed +exec_systemd: false + +# Path to the systemd binary in the container +exec_systemd_path: /usr/lib/systemd/systemd + +# Tmpfs mounts to add to the container +tmpfs: [] + +# Volumes to mount in the container +# Use docker CLI-style syntax: /host:/container[:mode] +# +# Mount modes can be a comma-separated list of various modes such as +# ro, rw, consistent, delegated, cached, rprivate, private, rshared, shared, rslave, slave, and nocopy. +# +# Note that the docker daemon might not support all modes and combinations of such modes. +# +# SELinux hosts can additionally use z or Z to use a shared or private label for the volume. +volumes: [] + +``` + This role should not be used directly in a playbook, and should instead be used via the `molecule.platform` role. Detailed information on configuration variables for this role can be found in [defaults/main.yml](defaults/main.yml). @@ -66,28 +189,6 @@ To utilize this role, use the `platform` role that is included with this collect - name: Create platform(s) ansible.builtin.include_role: name: syndr.molecule.platform - vars: - platform_name: "{{ item.name }}" - platform_state: present - platform_type: "{{ item.type }}" - platform_molecule_cfg: "{{ item }}" - loop: "{{ molecule_yml.platforms }}" - loop_control: - label: item.name - -# We want to avoid errors like "Failed to create temporary directory" -- name: Validate that inventory was refreshed - hosts: molecule - gather_facts: false - tasks: - - name: Check uname - ansible.builtin.raw: uname -a - register: result - changed_when: false - - - name: Display uname info - ansible.builtin.debug: - msg: "{{ result.stdout }}" ``` diff --git a/roles/docker_platform/defaults/main.yml b/roles/docker_platform/defaults/main.yml index bb5812d..b061b7a 100644 --- a/roles/docker_platform/defaults/main.yml +++ b/roles/docker_platform/defaults/main.yml @@ -1,39 +1,120 @@ --- # defaults file for docker_platform -# Name of this Molecule platform -docker_platform_name: instance - # Whether this platform should be deployed on the current system (present/absent) -docker_platform_state: present +docker_platform_state: "{{ platform_target_state | default('present') }}" + +# Molecule platform configuration (list of dictionaries) +docker_platform_config: "{{ platform_target_config | default(molecule_yml.platforms | default([])) }}" + +# Default values for Docker containers managed by this role +docker_platform_container_defaults: + # Image to use for the container + image: "geerlingguy/docker-rockylinux9-ansible:latest" + + # Run the container in privileged mode (WARNING: Less secure!) + privileged: false + + # List of capabilities to add to the container + capabilities: [] + + # List of capabilities to drop from the container + cap_drop: [] + + # Command to run in the container + command: "" + + # Number of CPUs to allocate to the container + cpus: 2 + + # List of device path and read rate (bytes per second) from device. + # Each entry should be a dictionary with keys 'path' and 'rate'. + device_read_bps: [] + + # List of device path and write rate (bytes per second) to device. + # Each entry should be a dictionary with keys 'path' and 'rate'. + device_write_bps: [] + + # List of device path and read rate (IO per second) from device. + # Each entry should be a dictionary with keys 'path' and 'rate'. + device_read_iops: [] + + # List of device path and write rate (IO per second) to device. + # Each entry should be a dictionary with keys 'path' and 'rate'. + device_write_iops: [] + + # List of host device bindings to add to the container. + # Each binding is a mapping expressed in the format: + # :: + devices: [] + + # List of DNS options + dns_opts: [] + + # List of DNS search domains + dns_search_domains: [] + + # List of DNS servers + dns_servers: [] + + # The URL or Unix socket path used to connect to the Docker API + docker_host: "{{ lookup('env', 'DOCKER_HOST') | default('unix:///var/run/docker.sock') }}" + + # Dict of host-to-IP mappings, where each host name is a key in the dictionary. Each host name will be added to the container’s /etc/hosts file. + # Instead of an IP address, the special value host-gateway can also be used, which resolves to the host’s gateway + # IP and allows containers to connect to services running on the host. + etc_hosts: {} + + # Dict of labels to apply to the container + labels: {} + + # Memory limit in format []. + # Number is a positive integer. + # Unit can be B (byte), K (kibibyte, 1024B), M (mebibyte), G (gibibyte), T (tebibyte), or P (pebibyte). + memory: "0" + + # Connect the container to a network. + # Choices are bridge, host, none, container:, or default. + network_mode: default -# Docker image that this platform runs -docker_platform_image: "geerlingguy/docker-rockylinux9-ansible:latest" + # List of ports to publish from the container to the host. + # Use docker CLI syntax: 8000, 9000:8000, or 0.0.0.0:9000:8000, + # where 8000 is a container port, 9000 is a host port, and 0.0.0.0 is a host interface. + published_ports: [] -# Should the provided image be modified at runtime -docker_platform_modify_image: false + # Pull setting for the container image (never,missing,always) + pull: always -# Path to docker build files that should be used to modify the image. Files are treated as templates -# and can contain jinja2 templating language ("{{ my_var }}" etc.) -docker_platform_modify_image_buildpath: "{{ molecule_ephemeral_directory }}/build" + # List of security options in the form of "label:user:User" + security_opts: [] -# Command to be executed at runtime on the container -# Leave as "" to use container default -docker_platform_command: "" + # State of the container + state: started -# Is this a SystemD enabled container? -docker_platform_systemd: true + # Whether the container runs systemd + systemd: false -# A list of Docker volumes that should be attached to the container -docker_platform_volumes: [] + # Attempt to execute systemd in the container on start + # WARNING: + # - This can cause issues with some containers + # - Not required if the container is already built with systemd running as PID 1 + # - Expects the container to have systemd installed + exec_systemd: false -# Run the container in Privileged mode (greater host access, less security!) -docker_platform_privileged: false + # Path to the systemd binary in the container + exec_systemd_path: /usr/lib/systemd/systemd -# A list of tmpfs filesystem paths to be passed to the container -docker_platform_tmpfs: [] + # Tmpfs mounts to add to the container + tmpfs: [] -# Ansible hostvars that should be associated with the ansible test "host" created by this role -# stored in the format `name: value` -docker_platform_hostvars: {} + # Volumes to mount in the container + # Use docker CLI-style syntax: /host:/container[:mode] + # + # Mount modes can be a comma-separated list of various modes such as + # ro, rw, consistent, delegated, cached, rprivate, private, rshared, shared, rslave, slave, and nocopy. + # + # Note that the docker daemon might not support all modes and combinations of such modes. + # + # SELinux hosts can additionally use z or Z to use a shared or private label for the volume. + volumes: [] diff --git a/roles/docker_platform/tasks/absent.yml b/roles/docker_platform/tasks/absent.yml index d54f927..eaf18ed 100644 --- a/roles/docker_platform/tasks/absent.yml +++ b/roles/docker_platform/tasks/absent.yml @@ -1,28 +1,52 @@ --- -- name: Removing infrastructure for platform +- name: Destroy Platform | Removing docker infrastructure for platform(s) ansible.builtin.debug: - var: docker_platform_name + msg: "{{ docker_platform_config | map(attribute='name') | list | join(', ') }}" - name: Stop and remove container - delegate_to: localhost + loop: "{{ docker_platform_config }}" + loop_control: + loop_var: __docker_platform_instance + label: "{{ __docker_platform_instance.name }}" community.docker.docker_container: - name: "{{ docker_platform_name }}" + name: "{{ __docker_platform_instance.name }}" state: absent auto_remove: true + register: __docker_platform_remove_jobs + async: 300 + poll: 0 -# TODO: Remove just this host, not the whole inventory -#- name: Remove dynamic molecule inventory -# delegate_to: localhost -# block: -# - name: Remove dynamic inventory file -# ansible.builtin.file: -# path: "{{ docker_platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" -# state: absent -# -# - name: Remove instance config file -# ansible.builtin.file: -# path: "{{ docker_platform_molecule_ephemeral_directory }}/instance_config.yml" -# state: absent +- name: Destroy Platform | Print removal job status + ansible.builtin.debug: + var: __docker_platform_remove_jobs + verbosity: 1 + +- name: Destroy Platform | Wait for container removal to complete + loop: "{{ __docker_platform_remove_jobs.results }}" + loop_control: + loop_var: __docker_platform_remove_result + label: "{{ __docker_platform_remove_result.__docker_platform_instance.name }}" + ansible.builtin.async_status: + jid: "{{ __docker_platform_remove_result.ansible_job_id }}" + register: __docker_platform_remove_results + until: __docker_platform_remove_results.finished + retries: 12 + delay: 5 + +- name: Destroy Platform | Print removal job results + ansible.builtin.debug: + var: __docker_platform_remove_results + verbosity: 1 + +- name: Destroy Platform | Clean up async cache + loop: "{{ __docker_platform_remove_results.results }}" + loop_control: + loop_var: __docker_platform_cache_cleanup + label: "{{ __docker_platform_cache_cleanup.__docker_platform_remove_result.__docker_platform_instance.name }}" + ansible.builtin.async_status: + jid: "{{ __docker_platform_cache_cleanup.__docker_platform_remove_result.ansible_job_id }}" + mode: cleanup +# TODO: Remove instances from molecule config diff --git a/roles/docker_platform/tasks/create.yml b/roles/docker_platform/tasks/create.yml deleted file mode 100644 index 7246db6..0000000 --- a/roles/docker_platform/tasks/create.yml +++ /dev/null @@ -1,117 +0,0 @@ ---- -# Create a docker container for use by molecule -# -# Expected to be called in a loop with `platform` defined as the loop var -# (loop off of `platforms` list in molecule.yml) - - -- name: Docker image needs to be customized - when: docker_platform_modify_image - block: - - name: Check build path - ansible.builtin.stat: - path: "{{ docker_platform_modify_image_buildpath }}" - register: __docker_platform_buildpath_stat - - - name: Build directory doesn't exist - when: __docker_platform_buildpath_stat.stat.exists is false - block: - - name: Create build directory - ansible.builtin.file: - path: "{{ docker_platform_modify_image_buildpath }}" - state: directory - mode: 0755 - - - name: Copy templates - loop: - - bash.service.j2 - - entrypoint.sh.j2 - - Dockerfile.j2 - loop_control: - loop_var: __docker_platform_item - ansible.builtin.template: - src: templates/{{ __docker_platform_item }} - dest: "{{ docker_platform_modify_image_buildpath}}/{{ __docker_platform_item | regex_replace('\\.j2$', '') }}" - - - name: Build local image name - ansible.builtin.set_fact: - __docker_platform_built_image_name: "molecule-local-build/{{ docker_platform_image | split(':') | first | split('/') | last }}-custom" - - - name: Docker image is built - community.docker.docker_image: - name: "{{ __docker_platform_built_image_name }}" - build: - path: "{{ docker_platform_modify_image_buildpath }}" - cache_from: "{{ docker_platform_image }}" - source: build - force_source: true # Always build a new image when this is run - tag: latest - register: image_build_output - - - name: Show image build details - ansible.builtin.debug: - var: image_build_output - verbosity: 1 - -- name: Build docker volume list - ansible.builtin.set_fact: - __docker_platform_volume_list: "{{ docker_platform_volumes + ['/sys/fs/cgroup:/sys/fs/cgroup:rw'] - if docker_platform_systemd - else docker_platform_volumes }}" - -- name: "{{ docker_platform_name }} docker container is present and running" - community.docker.docker_container: - name: "{{ docker_platform_name }}" - image: "{{ __docker_platform_built_image_name | default(docker_platform_image) }}" - state: started - command: "{{ docker_platform_command }}" - log_driver: json-file - hostname: molecule-ci-{{ docker_platform_name }} - init: false - cgroupns_mode: "{{ 'host' if docker_platform_systemd is true else 'private' }}" - privileged: "{{ docker_platform_privileged }}" - tmpfs: "{{ docker_platform_tmpfs + ['/run', '/run/lock'] if docker_platform_systemd else docker_platform_tmpfs }}" - volumes: "{{ __docker_platform_volume_list }}" - register: __docker_platform_create_result - -- name: Print creation output - ansible.builtin.debug: - msg: "{{ __docker_platform_create_result }}" - verbosity: 1 - -- name: Fail if is not running - when: > - __docker_platform_create_result.container.State.ExitCode != 0 or - not __docker_platform_create_result.container.State.Running - block: - - name: Retrieve {{ docker_platform_name }} container log - ansible.builtin.command: - cmd: docker logs {{ __docker_platform_create_result.container.Name }} - changed_when: false - register: __docker_platform_logfile_cmd - - - name: Container {{ docker_platform_name }} failed to start - ansible.builtin.fail: - msg: "{{ __docker_platform_logfile_cmd.stdout ~ __docker_platform_logfile_cmd.stderr }}" - -- name: "{{ docker_platform_name }} Systemd status is healthy" - when: docker_platform_systemd - block: - - name: System service manager is Systemd - ansible.builtin.assert: - that: - - "ansible_service_mgr == 'systemd'" - fail_msg: Systemd is enabled, but container service manager isn't Systemd! Is this a Systemd-enabled container? - - - name: Systemd has completed initialization - {{ docker_platform_name }} - ansible.builtin.command: - cmd: systemctl is-system-running - register: __docker_platform_systemctl_status - until: > - 'running' in __docker_platform_systemctl_status.stdout or - 'degraded' in __docker_platform_systemctl_status.stdout - retries: 30 - delay: 5 - changed_when: false - failed_when: __docker_platform_systemctl_status.rc > 1 - diff --git a/roles/docker_platform/tasks/custom_image.yml b/roles/docker_platform/tasks/custom_image.yml new file mode 100644 index 0000000..1a2152b --- /dev/null +++ b/roles/docker_platform/tasks/custom_image.yml @@ -0,0 +1,67 @@ +--- +# Build a custom image based upon the provided image, running Systemd as PID 1. +# +# Expected variables: +# - __docker_platform_config_custom: The desired Molecule platform configurations requiring custom entrypoint configuration for systemd +# - docker_platform_modify_image_buildpath: The path to the build directory for the custom image +# + +- name: Exec Systemd | Build path exists + ansible.builtin.file: + path: "{{ docker_platform_modify_image_buildpath }}" + state: directory + mode: 0755 + +- name: Exec Systemd | Build file exists + loop: + - bash.service.j2 + - entrypoint.sh.j2 + - Dockerfile.j2 + loop_control: + loop_var: __docker_platform_item + ansible.builtin.template: + src: templates/{{ __docker_platform_item }} + dest: "{{ docker_platform_modify_image_buildpath}}/{{ __docker_platform_item | regex_replace('\\.j2$', '') }}" + +- name: Exec Systemd | Launch image build + loop: "{{ __docker_platform_config_custom }}" + loop_control: + loop_var: __docker_platform_definition + label: "{{ __docker_platform_definition.name }}" + vars: + __docker_platform_built_image_name: "molecule-local-build/{{ __docker_platform_definition.image | split(':') | first | split('/') | last }}-custom" + community.docker.docker_image: + name: "{{ __docker_platform_built_image_name }}" + build: + path: "{{ docker_platform_modify_image_buildpath }}" + cache_from: "{{ __docker_platform_definition.image }}" + source: build + force_source: true # Always build a new image when this is run + tag: latest + register: __docker_platform_image_build_tasks + async: 600 + poll: 0 + +- name: 🐜 Exec Systemd | Show image build job tasks + ansible.builtin.debug: + var: __docker_platform_image_build_tasks + verbosity: 1 + +- name: Exec Systemd | Wait for image build to complete + ansible.builtin.async_status: + jid: "{{ __docker_platform_image_build_tasks.ansible_job_id }}" + register: __docker_platform_image_build_result + until: __docker_platform_image_build_result.finished + retries: 60 + delay: 10 + +- name: Exec Systemd | Clean up async cache + ansible.builtin.async_status: + jid: "{{ __docker_platform_image_build_tasks.ansible_job_id }}" + mode: cleanup + +- name: Exec Systemd | Show image build details + ansible.builtin.debug: + var: __docker_platform_image_build_result + verbosity: 1 + diff --git a/roles/docker_platform/tasks/main.yml b/roles/docker_platform/tasks/main.yml index bcbd555..57c41a2 100644 --- a/roles/docker_platform/tasks/main.yml +++ b/roles/docker_platform/tasks/main.yml @@ -1,6 +1,9 @@ --- # tasks file for docker_platform +- name: Validate configuration + ansible.builtin.include_tasks: "{{ role_path }}/tasks/validate_cfg.yml" + - name: Platform is deployed ansible.builtin.include_tasks: "{{ role_path }}/tasks/present.yml" when: docker_platform_state == 'present' diff --git a/roles/docker_platform/tasks/present.yml b/roles/docker_platform/tasks/present.yml index dd1a81c..c7f4dde 100644 --- a/roles/docker_platform/tasks/present.yml +++ b/roles/docker_platform/tasks/present.yml @@ -1,84 +1,178 @@ --- +# Create a docker container for use by molecule +# +# Expected input: +# - docker_platform_config: The Molecule Docker platform configuration (list) +# - docker_platform_container_defaults: Default values for Docker containers (dict) +# -- name: Initialize state - ansible.builtin.set_fact: - # Number of times this role has been included during this playbook run - __docker_platform_run_count: "{{ __docker_platform_run_count | default(0) | int + 1 }}" - -- name: Load system facts - ansible.builtin.setup: - filter: - - ansible_service_mgr - -- name: Create {{ docker_platform_name }} docker container - ansible.builtin.include_tasks: "{{ role_path }}/tasks/create.yml" +- name: Create Platform | Build custom images + vars: + __docker_platform_config_custom: >- + {{ + docker_platform_config + | selectattr('exec_systemd', 'defined') + | selectattr('exec_systemd', 'equalto', true) + | list + }} + when: __docker_platform_config_custom | length > 0 + ansible.builtin.include_tasks: "{{ role_path }}/tasks/custom_image.yml" -- name: Load existing instance configuration - block: - - name: Load existing instance configuration file - ansible.builtin.slurp: - src: "{{ docker_platform_molecule_ephemeral_directory }}/instance_config.yml" - register: __docker_platform_current_instance_config_b64 - ignore_errors: true +- name: Create Platform | Docker container is provisioned + loop: "{{ docker_platform_config }}" + loop_control: + loop_var: __docker_platform_instance + label: "{{ __docker_platform_instance.name }}" + vars: + __docker_platform_container_cfg: + name: "{{ __docker_platform_instance.name}}" + image: "{{ __docker_platform_instance.image | default(docker_platform_container_defaults.image) }}" + command: "{{ __docker_platform_instance.command | default(docker_platform_container_defaults.command) }}" + volumes: "{{ __docker_platform_instance.volumes | default(docker_platform_container_defaults.volumes) }}" + privileged: "{{ __docker_platform_instance.privileged | default(docker_platform_container_defaults.privileged) }}" + state: "{{ __docker_platform_instance.state | default(docker_platform_container_defaults.state) }}" + systemd: "{{ __docker_platform_instance.systemd | default(docker_platform_container_defaults.systemd) }}" + tmpfs: "{{ __docker_platform_instance.tmpfs | default(docker_platform_container_defaults.tmpfs) }}" + capabilities: "{{ __docker_platform_instance.capabilities | default(docker_platform_container_defaults.capabilities) }}" + cap_drop: "{{ __docker_platform_instance.cap_drop | default(docker_platform_container_defaults.cap_drop) }}" + cpus: "{{ __docker_platform_instance.cpus | default(docker_platform_container_defaults.cpus) }}" + device_read_bps: "{{ __docker_platform_instance.device_read_bps | default(docker_platform_container_defaults.device_read_bps) }}" + device_write_bps: "{{ __docker_platform_instance.device_write_bps | default(docker_platform_container_defaults.device_write_bps) }}" + device_read_iops: "{{ __docker_platform_instance.device_read_iops | default(docker_platform_container_defaults.device_read_iops) }}" + device_write_iops: "{{ __docker_platform_instance.device_write_iops | default(docker_platform_container_defaults.device_write_iops) }}" + devices: "{{ __docker_platform_instance.devices | default(docker_platform_container_defaults.devices) }}" + dns_opts: "{{ __docker_platform_instance.dns_opts | default(docker_platform_container_defaults.dns_opts) }}" + dns_search_domains: "{{ __docker_platform_instance.dns_search_domains | default(docker_platform_container_defaults.dns_search_domains) }}" + dns_servers: "{{ __docker_platform_instance.dns_servers | default(docker_platform_container_defaults.dns_servers) }}" + docker_host: "{{ __docker_platform_instance.docker_host | default(docker_platform_container_defaults.docker_host) }}" + etc_hosts: "{{ __docker_platform_instance.etc_hosts | default(docker_platform_container_defaults.etc_hosts) }}" + labels: "{{ __docker_platform_instance.labels | default(docker_platform_container_defaults.labels) }}" + memory: "{{ __docker_platform_instance.memory | default(docker_platform_container_defaults.memory) }}" + network_mode: "{{ __docker_platform_instance.network_mode | default(docker_platform_container_defaults.network_mode) }}" + published_ports: "{{ __docker_platform_instance.published_ports | default(docker_platform_container_defaults.published_ports) }}" + pull: "{{ __docker_platform_instance.pull | default(docker_platform_container_defaults.pull) }}" + security_opts: "{{ __docker_platform_instance.security_opts | default(docker_platform_container_defaults.security_opts) }}" + community.docker.docker_container: + name: "{{ __docker_platform_container_cfg.name }}" + image: "{{ __docker_platform_container_cfg.image }}" + state: "{{ __docker_platform_container_cfg.state }}" + command: "{{ __docker_platform_container_cfg.command }}" + log_driver: json-file + hostname: molecule-ci-{{ __docker_platform_container_cfg.name }} + init: false + cgroupns_mode: "{{ 'host' if __docker_platform_container_cfg.systemd is true else 'private' }}" + privileged: "{{ __docker_platform_container_cfg.privileged }}" + tmpfs: >- + {{ + __docker_platform_container_cfg.tmpfs + docker_platform_systemd_tmpfs + if __docker_platform_container_cfg.systemd + else + __docker_platform_container_cfg.tmpfs + }} + volumes: >- + {{ + __docker_platform_container_cfg.volumes + (docker_platform_systemd_volumes | map('regex_replace', '(.+)', '\1:\1:rw') | list) + if __docker_platform_container_cfg.systemd + else + __docker_platform_container_cfg.volumes + }} + capabilities: "{{ __docker_platform_container_cfg.capabilities }}" + cap_drop: "{{ __docker_platform_container_cfg.cap_drop }}" + cpus: "{{ __docker_platform_container_cfg.cpus }}" + device_read_bps: "{{ __docker_platform_container_cfg.device_read_bps }}" + device_write_bps: "{{ __docker_platform_container_cfg.device_write_bps }}" + device_read_iops: "{{ __docker_platform_container_cfg.device_read_iops }}" + device_write_iops: "{{ __docker_platform_container_cfg.device_write_iops }}" + devices: "{{ __docker_platform_container_cfg.devices }}" + dns_opts: "{{ __docker_platform_container_cfg.dns_opts }}" + dns_search_domains: "{{ __docker_platform_container_cfg.dns_search_domains }}" + dns_servers: "{{ __docker_platform_container_cfg.dns_servers }}" + docker_host: "{{ __docker_platform_container_cfg.docker_host }}" + etc_hosts: "{{ __docker_platform_container_cfg.etc_hosts }}" + labels: "{{ __docker_platform_container_cfg.labels }}" + memory: "{{ __docker_platform_container_cfg.memory }}" + network_mode: "{{ __docker_platform_container_cfg.network_mode }}" + published_ports: "{{ __docker_platform_container_cfg.published_ports }}" + pull: "{{ __docker_platform_container_cfg.pull }}" + security_opts: "{{ __docker_platform_container_cfg.security_opts }}" + register: __docker_platform_create_jobs + async: 300 + poll: 0 - - name: Decode instance configuration data - ansible.builtin.set_fact: - __docker_platform_current_instance_config: "{{ __docker_platform_current_instance_config_b64.content | default('') | b64decode | from_yaml }}" - when: __docker_platform_run_count | int > 1 +- name: Create Platform | Print creation job status + ansible.builtin.debug: + var: __docker_platform_create_jobs + verbosity: 1 -- name: Write {{ docker_platform_name }} instance config file - ansible.builtin.copy: - # This is very basic - just needs an item there to show as managed with docker config - content: | - {% if __docker_platform_current_instance_config is defined %} - {{ __docker_platform_current_instance_config | to_yaml }} - {% endif %} - - instance: {{ docker_platform_name }} - connection: docker - dest: "{{ docker_platform_molecule_ephemeral_directory }}/instance_config.yml" - mode: "0600" +- name: Create Platform | Wait for container creation to complete + loop: "{{ __docker_platform_create_jobs.results }}" + loop_control: + loop_var: __docker_platform_create_result + label: "{{ __docker_platform_create_result.__docker_platform_instance.name }}" + ansible.builtin.async_status: + jid: "{{ __docker_platform_create_result.ansible_job_id }}" + register: __docker_platform_create_results + until: __docker_platform_create_results.finished + retries: 60 + delay: 1 + changed_when: false -- name: Load existing molecule inventory - block: - - name: Load existing molecule inventory file - ansible.builtin.slurp: - src: "{{ docker_platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" - register: __docker_platform_current_molecule_inventory_b64 - ignore_errors: true +- name: Create Platform | Print container creation result + ansible.builtin.debug: + var: __docker_platform_create_results + verbosity: 1 - - name: Decode instance configuration data - ansible.builtin.set_fact: - __docker_platform_current_molecule_inventory: "{{ __docker_platform_current_molecule_inventory_b64.content | default({}) | b64decode | from_yaml }}" - when: __docker_platform_run_count | int > 1 +- name: Create Platform | Clean up async cache + loop: "{{ __docker_platform_create_results.results }}" + loop_control: + loop_var: __docker_platform_cache_cleanup + label: "{{ __docker_platform_cache_cleanup.__docker_platform_create_result.__docker_platform_instance.name }}" + ansible.builtin.async_status: + jid: "{{ __docker_platform_cache_cleanup.__docker_platform_create_result.ansible_job_id }}" + mode: cleanup -- name: Add {{ docker_platform_name }} to molecule_inventory +- name: Create Platform | Collect failed container logs + loop: "{{ __docker_platform_create_results.results | selectattr('container.State.Running', 'equalto', False) }}" + loop_control: + loop_var: __docker_platform_failed_container + label: "{{ __docker_platform_failed_container_name }}" vars: - __docker_platform_inventory_partial_hostvars: "{{ { - 'ansible_connection': 'community.docker.docker' - } | combine(docker_platform_hostvars, recursive=true) }}" - __docker_platform_inventory_partial_yaml: | - all: - children: - molecule: - hosts: - "{{ docker_platform_name }}": {{ __docker_platform_inventory_partial_hostvars }} - ansible.builtin.set_fact: - __docker_platform_molecule_inventory: > - {{ __docker_platform_current_molecule_inventory | from_yaml | default({}) | combine(__docker_platform_inventory_partial_yaml | from_yaml, recursive=true) }} - -- name: Write {{ docker_platform_name }} to molecule inventory file - ansible.builtin.copy: - content: | - {{ __docker_platform_molecule_inventory | to_yaml }} - dest: "{{ docker_platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" - mode: "0600" + __docker_platform_failed_container_name: "{{ __docker_platform_failed_container.__docker_platform_create_result.__docker_platform_instance.name }}" + ansible.builtin.command: + cmd: docker logs {{ __docker_platform_failed_container_name }} + register: __docker_platform_container_faillogs + changed_when: false -- name: Force inventory refresh - ansible.builtin.meta: refresh_inventory +- name: Create Platform | Container creation failed + loop: "{{ __docker_platform_container_faillogs.results }}" + loop_control: + loop_var: __docker_platform_logfile_cmd + label: "{{ __docker_platform_failed_container_name }}" + vars: + __docker_platform_failed_container_name: >- + {{ + __docker_platform_logfile_cmd.__docker_platform_failed_container.__docker_platform_create_result.__docker_platform_instance.name + }} + ansible.builtin.fail: + msg: | + Return Code: {{ __docker_platform_logfile_cmd.__docker_platform_failed_container.container.State.ExitCode }} + Stdout: {{ __docker_platform_logfile_cmd.stdout }} + Stderr: {{ __docker_platform_logfile_cmd.stderr }} -- name: Fail if molecule group is missing - ansible.builtin.assert: - that: "'molecule' in groups" - fail_msg: | - molecule group was not found inside inventory groups: {{ groups }} +- name: Create Platform | Export platform configuration + ansible.builtin.set_fact: + platform_exported_config: >- + {%- set __platform_instances = {} -%} + {%- for __platform in __platform_config -%} + {%- set _ = __platform_instances.update({ + __platform.name: { + 'hostvars': __platform.hostvars | combine(docker_platform_hostvars_base), + 'instance_config': { + 'name': __platform.name, + 'connection': 'docker' + } + } + }) -%} + {%- endfor -%} + {{ __platform_instances }} diff --git a/roles/docker_platform/tasks/validate_cfg.yml b/roles/docker_platform/tasks/validate_cfg.yml new file mode 100644 index 0000000..4bdecb1 --- /dev/null +++ b/roles/docker_platform/tasks/validate_cfg.yml @@ -0,0 +1,23 @@ +--- +# Validate that configuration provided to the role is valid +# +# Expected variables: +# - docker_platform_config: The current Molecule platform configurations (list) +# + +- name: Validate | Docker platform configuration + loop: "{{ docker_platform_config }}" + loop_control: + loop_var: __docker_platform_item + label: "{{ __docker_platform_item.name | default('undefined') }}" + ansible.builtin.assert: + that: + - __docker_platform_item is mapping + - __docker_platform_item.name is string + - __docker_platform_item.name | length > 0 + - __docker_platform_item.type is string + - __docker_platform_item.type | length > 0 + - __docker_platform_item.type == 'docker' + fail_msg: "Invalid Docker platform configuration provided!" + success_msg: "Docker platform configuration validated." + diff --git a/roles/docker_platform/vars/main.yml b/roles/docker_platform/vars/main.yml index 7f2db29..1e1795e 100644 --- a/roles/docker_platform/vars/main.yml +++ b/roles/docker_platform/vars/main.yml @@ -4,3 +4,19 @@ # Filesystem location of the Molecule ephemeral directory. Should not need to be updated by the user of this role! docker_platform_molecule_ephemeral_directory: "{{ molecule_ephemeral_directory }}" +# Working directory for docker image build process +docker_platform_modify_image_buildpath: "{{ molecule_ephemeral_directory }}/build" + +# Volume mounts required for systemd enabled containers (will be mounted as read-write) +docker_platform_systemd_volumes: + - /sys/fs/cgroup + +# Tempfs mounts required for systemd enabled containers +docker_platform_systemd_tmpfs: + - /run + - /run/lock + +# Base hostvars for docker containers that should always be present +docker_platform_hostvars_base: + ansible_connection: community.docker.docker + diff --git a/roles/platform/defaults/main.yml b/roles/platform/defaults/main.yml index 70657a0..7067a05 100644 --- a/roles/platform/defaults/main.yml +++ b/roles/platform/defaults/main.yml @@ -1,15 +1,12 @@ --- # defaults file for platform -# Name of this Molecule platform -platform_name: instance - # Whether this platform should be deployed on the current system (present/absent) platform_state: present -# What type of platform should be deployed -platform_type: docker +# Molecule platform configuration (single platform) +platform_molecule_cfg: {} # Molecule platform configuration -platform_molecule_cfg: {} +platform_config: "{{ molecule_yml.platforms }}" diff --git a/roles/platform/tasks/deprovision.yml b/roles/platform/tasks/deprovision.yml index e4dc627..2f2a025 100644 --- a/roles/platform/tasks/deprovision.yml +++ b/roles/platform/tasks/deprovision.yml @@ -1,25 +1,43 @@ --- # Remove deployed resources -- name: Initilze state +- name: Show platform configuration + ansible.builtin.debug: + var: platform_config + verbosity: 1 + +- name: Show runtime platform configuration + ansible.builtin.debug: + var: __platform_config + verbosity: 1 + +- name: Initialize state ansible.builtin.set_fact: # Number of times this role has been included during this playbook run __platform_run_count: "{{ __platform_run_count | default(0) | int + 1 }}" -- name: Remove docker-type platform - when: platform_type == 'docker' - ansible.builtin.include_role: - name: "{{ ansible_collection_name }}.docker_platform" +- name: Remove {{ __platform_config | length }} Molecule platform(s) vars: - docker_platform_name: "{{ platform_name }}" - docker_platform_state: absent - -- name: Remove ec2-type platform - when: platform_type == 'ec2' + __platform_type: "{{ __platform_config | map(attribute='type') | first }}" + platform_target_config: "{{ __platform_config }}" + platform_target_state: absent ansible.builtin.include_role: - name: "{{ ansible_collection_name }}.ec2_platform" - vars: - ec2_platform_name: "{{ platform_name }}" - ec2_platform_state: absent - ec2_platform_definition: "{{ platform_molecule_cfg }}" + name: "{{ ansible_collection_name }}.{{ __platform_type }}_platform" + +#- name: Remove docker-type platform +# when: platform_type == 'docker' +# ansible.builtin.include_role: +# name: "{{ ansible_collection_name }}.docker_platform" +# vars: +# docker_platform_name: "{{ platform_name }}" +# docker_platform_state: absent +# +#- name: Remove ec2-type platform +# when: platform_type == 'ec2' +# ansible.builtin.include_role: +# name: "{{ ansible_collection_name }}.ec2_platform" +# vars: +# ec2_platform_name: "{{ platform_name }}" +# ec2_platform_state: absent +# ec2_platform_definition: "{{ platform_molecule_cfg }}" diff --git a/roles/platform/tasks/inventory.yml b/roles/platform/tasks/inventory.yml index 4e81113..c0bf64d 100644 --- a/roles/platform/tasks/inventory.yml +++ b/roles/platform/tasks/inventory.yml @@ -1,41 +1,52 @@ --- # Add a host to the Molecule inventory +# +# Expected variables: +# - platform_exported_config: The exported platform configuration from a xxx_platform role +# Expected format: +# : +# hostvars: +# hostvar1: value1 +# instance_config: +# name: +# connection: +# +# - name: Generate new instance configuration when: platform_state == 'present' block: - - name: Generate {{ platform_name }} instance configuration (Docker) - when: platform_type == 'docker' - ansible.builtin.set_fact: - __platform_new_instance_configs: "{{ [{ - 'instance': platform_name, - 'connection': 'docker' - }] }}" - __platform_ansible_hostvars: "{{ { - platform_name: { - 'ansible_connection': 'community.docker.docker' - } } }}" - - - name: Generate {{ platform_name }} instance configuration (EC2) - when: platform_type == 'ec2' - ansible.builtin.set_fact: - # NOTE: This depends on the ec2_platform_instance_config being set by the ec2_platform role - __platform_new_instance_configs: "{{ [ec2_platform_instance_config] }}" - __platform_ansible_connection: "ssh" - __platform_ansible_hostvars: "{{ { - ec2_platform_instance_config.instance: { - 'ansible_connection': 'ssh', - 'ansible_host': ec2_platform_instance_config.address, - 'ansible_port': ec2_platform_instance_config.port, - 'ansible_user': ec2_platform_instance_config.user, - 'ansible_ssh_private_key_file': ec2_platform_instance_config.identity_file - } } }}" - - - name: Show hostvars configuration + - name: 🐜 Show platform config ansible.builtin.debug: - var: __platform_ansible_hostvars + var: platform_exported_config verbosity: 1 + #- name: Generate {{ platform_name }} instance configuration (Docker) + # when: platform_type == 'docker' + # ansible.builtin.set_fact: + # __platform_new_instance_configs: "{{ [{ + # 'instance': platform_name, + # 'connection': 'docker' + # }] }}" + # __platform_ansible_hostvars: "{{ { + # platform_name: { + # 'ansible_connection': 'community.docker.docker' + # } } }}" + + #- name: Generate {{ platform_name }} instance configuration (EC2) + # when: platform_type == 'ec2' + # ansible.builtin.set_fact: + # # NOTE: This depends on the ec2_platform_instance_config being set by the ec2_platform role + # __platform_new_instance_configs: "{{ [ec2_platform_instance_config] }}" + # __platform_ansible_hostvars: "{{ { + # ec2_platform_instance_config.instance: { + # 'ansible_connection': 'ssh', + # 'ansible_host': ec2_platform_instance_config.address, + # 'ansible_port': ec2_platform_instance_config.port, + # 'ansible_user': ec2_platform_instance_config.user, + # 'ansible_ssh_private_key_file': ec2_platform_instance_config.identity_file + # } } }}" + - name: Load existing instance configuration block: - name: Load existing instance configuration file @@ -50,12 +61,18 @@ # Build the Molecule instance configuration object - name: Process instance configuration - when: __platform_instance_config is truthy or __platform_new_instance_configs is defined - ansible.builtin.include_tasks: "{{ role_path }}/tasks/instance_config.yml" - loop: "{{ __platform_new_instance_configs if __platform_new_instance_configs is defined else __platform_instance_config | default([], true) }}" + when: __platform_instance_config is truthy or platform_exported_config is defined + loop: >- + {{ + platform_exported_config | dict2items + if platform_exported_config is defined + else + __platform_instance_config | default([], true) + }} loop_control: loop_var: __platform_target_instance_config label: "{{ __platform_target_instance_config.instance }}" + ansible.builtin.include_tasks: "{{ role_path }}/tasks/instance_config.yml" - name: Manage molecule instance config file block: @@ -124,4 +141,3 @@ fail_msg: | molecule group was not found inside inventory groups: {{ groups }} - diff --git a/roles/platform/tasks/main.yml b/roles/platform/tasks/main.yml index 7b9ddec..7731887 100644 --- a/roles/platform/tasks/main.yml +++ b/roles/platform/tasks/main.yml @@ -1,14 +1,29 @@ --- # tasks file for platform -- name: Platform is provisioned - ansible.builtin.include_tasks: "{{ role_path }}/tasks/provision.yml" - when: platform_state == 'present' +- name: Dump hostvars + ansible.builtin.debug: + var: vars -- name: Platform is destroyed - ansible.builtin.include_tasks: "{{ role_path }}/tasks/deprovision.yml" - when: platform_state == 'absent' +- name: Show platform configuration + ansible.builtin.debug: + var: platform_config -- name: Configure Molecule inventory - ansible.builtin.include_tasks: "{{ role_path }}/tasks/inventory.yml" +- name: Process provided platform configuration + vars: + __platform_config: "{{ platform_molecule_cfg if platform_molecule_cfg is truthy else platform_config }}" + block: + - name: Validate platform configuration + ansible.builtin.include_tasks: "{{ role_path }}/tasks/validate_cfg.yml" + + - name: Platform is provisioned + when: platform_state == 'present' + ansible.builtin.include_tasks: "{{ role_path }}/tasks/provision.yml" + + - name: Platform is destroyed + when: platform_state == 'absent' + ansible.builtin.include_tasks: "{{ role_path }}/tasks/deprovision.yml" + + - name: Configure Molecule inventory + ansible.builtin.include_tasks: "{{ role_path }}/tasks/inventory.yml" diff --git a/roles/platform/tasks/provision.yml b/roles/platform/tasks/provision.yml index 895fabd..3724f36 100644 --- a/roles/platform/tasks/provision.yml +++ b/roles/platform/tasks/provision.yml @@ -1,37 +1,48 @@ --- # Create a host and requisite configuration for use by molecule # +# Expected input: +# - __platform_config: The molecule platform configuration (list) +# +# + +- name: Show runtime platform configuration + ansible.builtin.debug: + var: __platform_config + verbosity: 1 - name: Initialize state ansible.builtin.set_fact: # Number of times this role has been included during this playbook run __platform_run_count: "{{ __platform_run_count | default(0) | int + 1 }}" -- name: Load system facts - ansible.builtin.setup: - filter: - - ansible_service_mgr - -- name: Configure platform for docker type - when: platform_type == 'docker' - ansible.builtin.include_role: - name: "{{ ansible_collection_name }}.docker_platform" +- name: Configure {{ __platform_config | length }} Molecule platform(s) vars: - docker_platform_name: "{{ platform_name }}" - docker_platform_state: present - docker_platform_image: "{{ platform_molecule_cfg.image }}" - docker_platform_systemd: "{{ platform_molecule_cfg.systemd | default(false) }}" - docker_platform_modify_image: "{{ platform_molecule_cfg.modify_image | default(false) }}" - docker_platform_modify_image_buildpath: "{{ platform_molecule_cfg.modify_image_buildpath | default(molecule_ephemeral_directory + '/build') }}" - docker_platform_privileged: "{{ platform_molecule_cfg.privileged | default (false) }}" - docker_platform_hostvars: "{{ platform_molecule_cfg.hostvars | default({}) }}" - -- name: Configure platform for ec2 type - when: platform_type == 'ec2' + __platform_type: "{{ __platform_config | map(attribute='type') | first }}" + platform_target_config: "{{ __platform_config }}" ansible.builtin.include_role: - name: "{{ ansible_collection_name }}.ec2_platform" - vars: - ec2_platform_name: "{{ platform_name }}" - ec2_platform_state: present - ec2_platform_definition: "{{ platform_molecule_cfg }}" + name: "{{ ansible_collection_name }}.{{ __platform_type }}_platform" + +#- name: Configure platform for docker type +# when: platform_type == 'docker' +# ansible.builtin.include_role: +# name: "{{ ansible_collection_name }}.docker_platform" +# vars: +# docker_platform_name: "{{ platform_name }}" +# docker_platform_state: present +# docker_platform_image: "{{ platform_molecule_cfg.image }}" +# docker_platform_systemd: "{{ platform_molecule_cfg.systemd | default(false) }}" +# docker_platform_modify_image: "{{ platform_molecule_cfg.modify_image | default(false) }}" +# docker_platform_modify_image_buildpath: "{{ platform_molecule_cfg.modify_image_buildpath | default(molecule_ephemeral_directory + '/build') }}" +# docker_platform_privileged: "{{ platform_molecule_cfg.privileged | default (false) }}" +# docker_platform_hostvars: "{{ platform_molecule_cfg.hostvars | default({}) }}" +# +#- name: Configure platform for ec2 type +# when: platform_type == 'ec2' +# ansible.builtin.include_role: +# name: "{{ ansible_collection_name }}.ec2_platform" +# vars: +# ec2_platform_name: "{{ platform_name }}" +# ec2_platform_state: present +# ec2_platform_definition: "{{ platform_molecule_cfg }}" diff --git a/roles/platform/tasks/validate_cfg.yml b/roles/platform/tasks/validate_cfg.yml new file mode 100644 index 0000000..92150de --- /dev/null +++ b/roles/platform/tasks/validate_cfg.yml @@ -0,0 +1,31 @@ +--- +# Make sure provided Molecule platform configuration is valid +# +# Expected variables: +# - __platform_config: The current list of platform configurations +# + +- name: Validate | Platform configuration + ansible.builtin.assert: + that: + - __platform_config is truthy + - __platform_config | length > 0 + fail_msg: "No platform configuration provided, or platform configuration invalid!" + success_msg: "Platform configuration provided." + +- name: Validate | Unique attributes + ansible.builtin.assert: + that: + - __platform_config | map(attribute='name') | length == __platform_config | length + - __platform_config | map(attribute='name') | list | length == __platform_config | map(attribute='name') | list | unique | list | length + fail_msg: "No platforms defined or duplicate/missing platform names provided!" + success_msg: "Platform unique attributes validated." + +- name: Validate | Platform types + ansible.builtin.assert: + that: + - __platform_config | map(attribute='type') | length == __platform_config | length + - __platform_config | map(attribute='type') | unique | length == 1 + fail_msg: "Types must be the same for all platforms provided!" + success_msg: "Platform types validated." + diff --git a/roles/platform/vars/main.yml b/roles/platform/vars/main.yml index 917f1d5..be77150 100644 --- a/roles/platform/vars/main.yml +++ b/roles/platform/vars/main.yml @@ -4,12 +4,14 @@ # Filesystem location of the Molecule ephemeral directory. Should not need to be updated by the user of this role! platform_molecule_ephemeral_directory: "{{ molecule_ephemeral_directory }}" + +## INTERNAL VARIABLES -- DO NOT MODIFY ## + # Does the molecule instance configuration file need to be updated? (assume yes) __platform_instance_config_update_needed: true - # Default connection method for hosts -- update as needed in tasks/inventory.yml -__platform_ansible_connection: "ssh" - # The current molecule inventory contents __platform_current_molecule_inventory: {} +__platform_inventory_hostvars: {} + From d551ca9b1d314fc80c241c7888113a5928ad1e60 Mon Sep 17 00:00:00 2001 From: syndr Date: Thu, 9 Jan 2025 14:59:04 -0700 Subject: [PATCH 12/18] Molecule inventory configuration, instance management Link collection to molecule ephemeral dir instead of copying during prepare step Refactor Molecule inventory generation * simplify host addition/removal logic * support combined execution (list of multiple platforms) --- molecule/default/collections.yml | 8 +- molecule/default/create.yml | 42 ------- molecule/default/molecule.yml | 1 - molecule/resources/destroy.yml | 9 +- molecule/resources/prepare.yml | 20 +++- roles/docker_platform/tasks/main.yml | 17 +++ roles/docker_platform/tasks/present.yml | 17 --- roles/ec2_platform/vars/main.yml | 9 ++ roles/platform/tasks/ansible_inventory.yml | 112 +++++++++++++----- roles/platform/tasks/deprovision.yml | 17 --- roles/platform/tasks/instance_config.yml | 130 +++++++++------------ roles/platform/tasks/inventory.yml | 126 +------------------- roles/platform/tasks/main.yml | 10 +- roles/platform/tasks/provision.yml | 23 ---- roles/platform/tasks/validate_cfg.yml | 17 +++ roles/platform/vars/main.yml | 8 +- 16 files changed, 209 insertions(+), 357 deletions(-) mode change 100644 => 120000 molecule/default/collections.yml delete mode 100644 molecule/default/create.yml diff --git a/molecule/default/collections.yml b/molecule/default/collections.yml deleted file mode 100644 index b640240..0000000 --- a/molecule/default/collections.yml +++ /dev/null @@ -1,7 +0,0 @@ ---- - -collections: - - name: community.general - - name: community.docker - - name: syndr.molecule - diff --git a/molecule/default/collections.yml b/molecule/default/collections.yml new file mode 120000 index 0000000..fec6c24 --- /dev/null +++ b/molecule/default/collections.yml @@ -0,0 +1 @@ +../resources/collections.yml \ No newline at end of file diff --git a/molecule/default/create.yml b/molecule/default/create.yml deleted file mode 100644 index d1707c6..0000000 --- a/molecule/default/create.yml +++ /dev/null @@ -1,42 +0,0 @@ ---- -- name: Create - hosts: localhost - gather_facts: false - tasks: - - name: Create platform - ansible.builtin.include_role: - name: syndr.molecule.platform - -# We want to avoid errors like "Failed to create temporary directory" -- name: Validate that inventory was refreshed - hosts: molecule - gather_facts: false - tasks: - - name: Check uname - ansible.builtin.raw: uname -a - register: result - changed_when: false - - - name: Display uname info - ansible.builtin.debug: - msg: "{{ result.stdout }}" - - - name: Load system facts - ansible.builtin.setup: - filter: - - ansible_service_mgr - - - name: Check on Systemd - block: - - name: Wait for systemd to complete initialization. - ansible.builtin.command: systemctl is-system-running - register: systemctl_status - until: > - 'running' in systemctl_status.stdout or - 'degraded' in systemctl_status.stdout - retries: 30 - delay: 5 - changed_when: false - failed_when: systemctl_status.rc > 1 - when: ansible_service_mgr == 'systemd' - diff --git a/molecule/default/molecule.yml b/molecule/default/molecule.yml index 8d2e453..e0cfb06 100644 --- a/molecule/default/molecule.yml +++ b/molecule/default/molecule.yml @@ -58,7 +58,6 @@ scenario: create_sequence: - dependency - create - - prepare check_sequence: - dependency - cleanup diff --git a/molecule/resources/destroy.yml b/molecule/resources/destroy.yml index 2157a1d..f8669b3 100644 --- a/molecule/resources/destroy.yml +++ b/molecule/resources/destroy.yml @@ -5,13 +5,8 @@ gather_facts: false tasks: - name: Remove platform(s) - ansible.builtin.include_role: - name: syndr.molecule.platform vars: platform_state: absent - platform_type: "{{ item.type }}" - platform_molecule_cfg: "{{ item }}" - loop: "{{ molecule_yml.platforms }}" - loop_control: - label: item.name + ansible.builtin.include_role: + name: syndr.molecule.platform diff --git a/molecule/resources/prepare.yml b/molecule/resources/prepare.yml index aca497f..f8ba73f 100644 --- a/molecule/resources/prepare.yml +++ b/molecule/resources/prepare.yml @@ -24,12 +24,20 @@ cmd: ansible-galaxy collection build --force register: build_result - - name: Install collection - vars: - collection_archive_path: "{{ build_result.stdout | regex_replace('.*\\s+(\\S+\\.tar\\.gz)', '\\1') }}" - ansible.builtin.command: - chdir: "{{ playbook_dir }}/../../" - cmd: ansible-galaxy collection install --force {{ collection_archive_path }} + # WARNING: This can cause problems if the current state of the collection in this project is borked! + - name: Link current collection version for controller + block: + - name: Molecule scenario collection path exists + ansible.builtin.file: + path: "{{ molecule_ephemeral_directory }}/collections/ansible_collections/{{ collection_namespace }}" + state: directory + mode: 0755 + + - name: Molecule scenario collection link exists + ansible.builtin.file: + path: "{{ molecule_ephemeral_directory }}/collections/ansible_collections/{{ collection_namespace }}/{{ collection_name }}" + src: "{{ playbook_dir }}/../../" + state: link - name: Prepare target host for execution hosts: molecule diff --git a/roles/docker_platform/tasks/main.yml b/roles/docker_platform/tasks/main.yml index 57c41a2..733b7a3 100644 --- a/roles/docker_platform/tasks/main.yml +++ b/roles/docker_platform/tasks/main.yml @@ -12,3 +12,20 @@ ansible.builtin.include_tasks: "{{ role_path }}/tasks/absent.yml" when: docker_platform_state == 'absent' +- name: Export platform configuration + ansible.builtin.set_fact: + platform_exported_config: >- + {%- set __platform_instances = {} -%} + {%- for __platform in __platform_config -%} + {%- set _ = __platform_instances.update({ + __platform.name: { + 'hostvars': __platform.hostvars | combine(docker_platform_hostvars_base), + 'instance_config': { + 'instance': __platform.name, + 'connection': 'docker' + } + } + }) -%} + {%- endfor -%} + {{ __platform_instances }} + diff --git a/roles/docker_platform/tasks/present.yml b/roles/docker_platform/tasks/present.yml index c7f4dde..ef74698 100644 --- a/roles/docker_platform/tasks/present.yml +++ b/roles/docker_platform/tasks/present.yml @@ -159,20 +159,3 @@ Stdout: {{ __docker_platform_logfile_cmd.stdout }} Stderr: {{ __docker_platform_logfile_cmd.stderr }} -- name: Create Platform | Export platform configuration - ansible.builtin.set_fact: - platform_exported_config: >- - {%- set __platform_instances = {} -%} - {%- for __platform in __platform_config -%} - {%- set _ = __platform_instances.update({ - __platform.name: { - 'hostvars': __platform.hostvars | combine(docker_platform_hostvars_base), - 'instance_config': { - 'name': __platform.name, - 'connection': 'docker' - } - } - }) -%} - {%- endfor -%} - {{ __platform_instances }} - diff --git a/roles/ec2_platform/vars/main.yml b/roles/ec2_platform/vars/main.yml index 11e401d..d156fac 100644 --- a/roles/ec2_platform/vars/main.yml +++ b/roles/ec2_platform/vars/main.yml @@ -1,2 +1,11 @@ --- # vars file for ec2_platform + +# Base hostvars for EC2 instances that should always be present +ec2_platform_hostvars_base: + ansible_connection: ssh + ansible_host: "{{ ec2_platform_instance_config.address }}" + ansible_port: "{{ ec2_platform_instance_config.port }}" + ansible_user: "{{ ec2_platform_instance_config.user }}" + ansible_ssh_private_key_file: "{{ ec2_platform_instance_config.identity_file }}" + diff --git a/roles/platform/tasks/ansible_inventory.yml b/roles/platform/tasks/ansible_inventory.yml index 77878f3..3edf087 100644 --- a/roles/platform/tasks/ansible_inventory.yml +++ b/roles/platform/tasks/ansible_inventory.yml @@ -22,35 +22,93 @@ # - __platform_molecule_inventory: The updated molecule inventory contents # -# TODO: Omit the `ec2_tag_vars` key itself so that it doesn't get written to the inventory file (duplicating values) -- name: Host exists in inventory file - when: platform_state == 'present' +- name: 🐜 Show platform config + ansible.builtin.debug: + var: platform_exported_config + verbosity: 1 + +- name: 🐜 Show ansible inventory config from file + ansible.builtin.debug: + var: platform_inventory_config_from_file + verbosity: 1 + +- name: Platform Molecule Inventory | Build host configuration + ansible.builtin.set_fact: + __platform_ansible_hosts: >- + {%- set __platform_host_definitions = {} -%} + {%- for __platform_host_name, __platform_host_data in platform_exported_config.items() -%} + {%- set _ = __platform_host_definitions.update({ + __platform_host_name: __platform_host_data.hostvars + }) -%} + {%- endfor -%} + {{ __platform_host_definitions }} + +- name: Platform Molecule Inventory | Load existing unreferenced hosts + ansible.builtin.set_fact: + # Existing host configuration, minus those that are being updated + __platform_ansible_existing_unreferenced_hosts: >- + {%- set __platform_unreferenced_hosts = {} -%} + {%- for __platform_host_name, __platform_host_data in (platform_inventory_config_from_file.all.children.molecule.hosts | default({})).items() -%} + {%- if __platform_host_name not in __platform_ansible_hosts.keys() -%} + {%- set _ = __platform_unreferenced_hosts.update({ + __platform_host_name: __platform_host_data + }) -%} + {%- endif -%} + {%- endfor -%} + {{ __platform_unreferenced_hosts }} + +- name: 🐜 Show platform host definitions + ansible.builtin.debug: + var: __platform_ansible_hosts + verbosity: 1 + +- name: Show existing unreferenced hosts + ansible.builtin.debug: + var: __platform_ansible_existing_unreferenced_hosts + verbosity: 1 + +- name: Platform Molecule Inventory | Build inventory object vars: - # NOTE: Has to be inside jinja so that we can set key name - __platform_molecule_inventory_partial: "{{ { - 'all': { - 'children': { - 'molecule': { - 'hosts': { - __platform_target_instance_config.instance: __platform_ansible_hostvars[__platform_target_instance_config.instance] - | combine(platform_molecule_cfg.hostvars | default({}) - | combine(__platform_ansible_hostvars[__platform_target_instance_config.instance].ec2_tag_vars | default({}))) - }}}}} }}" + __platform_ansible_hosts_present: >- + {{ + __platform_ansible_hosts + | combine(__platform_ansible_existing_unreferenced_hosts) + if platform_state == 'present' + else __platform_ansible_existing_unreferenced_hosts + }} ansible.builtin.set_fact: - __platform_molecule_inventory: "{{ __platform_molecule_inventory | from_yaml | default({}) - | combine(__platform_molecule_inventory_partial, recursive=true) }}" + __platform_molecule_inventory: >- + {{ + platform_inventory_config_from_file + | combine({ + 'all': { + 'children': { + 'molecule': { + 'hosts': __platform_ansible_hosts_present + } + } + } + }) + }} + -- name: Host does not exist in inventory file +- name: Platform Molecule Inventory | Write inventory file + ansible.builtin.copy: + content: | + {{ __platform_molecule_inventory | to_nice_yaml(indent=2) }} + dest: "{{ platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" + mode: '0644' + +- name: Platform Molecule Inventory | Force inventory refresh + ansible.builtin.meta: refresh_inventory + +- name: Platform Molecule Inventory | Validate molecule group when: - - platform_state == 'absent' - - __platform_molecule_inventory is truthy - ansible.builtin.set_fact: - __platform_molecule_inventory: "{{ - __platform_molecule_inventory | combine({ - 'all': { - 'children': { - 'molecule': { - 'hosts': (__platform_molecule_inventory.all.children.molecule.hosts | default({}) | - dict2items | rejectattr('key', 'equalto', __platform_target_instance_config.instance) | items2dict) - }}}}, recursive=true) }}" + - __platform_molecule_inventory is defined + - platform_state == 'present' + ansible.builtin.assert: + that: "'molecule' in groups" + fail_msg: | + Molecule group was not found inside inventory groups: {{ groups }} + success_msg: "Molecule group found inside inventory groups: {{ groups }}" diff --git a/roles/platform/tasks/deprovision.yml b/roles/platform/tasks/deprovision.yml index 2f2a025..6cfb308 100644 --- a/roles/platform/tasks/deprovision.yml +++ b/roles/platform/tasks/deprovision.yml @@ -24,20 +24,3 @@ ansible.builtin.include_role: name: "{{ ansible_collection_name }}.{{ __platform_type }}_platform" -#- name: Remove docker-type platform -# when: platform_type == 'docker' -# ansible.builtin.include_role: -# name: "{{ ansible_collection_name }}.docker_platform" -# vars: -# docker_platform_name: "{{ platform_name }}" -# docker_platform_state: absent -# -#- name: Remove ec2-type platform -# when: platform_type == 'ec2' -# ansible.builtin.include_role: -# name: "{{ ansible_collection_name }}.ec2_platform" -# vars: -# ec2_platform_name: "{{ platform_name }}" -# ec2_platform_state: absent -# ec2_platform_definition: "{{ platform_molecule_cfg }}" - diff --git a/roles/platform/tasks/instance_config.yml b/roles/platform/tasks/instance_config.yml index 22e069c..7f7eca1 100644 --- a/roles/platform/tasks/instance_config.yml +++ b/roles/platform/tasks/instance_config.yml @@ -2,84 +2,62 @@ # Add an instance to the list of Molecule hosts # # Expected variables: -# - __platform_instance_config: The current list of instance configurations -# - __platform_target_instance_config: The configuration for the new instance -# - __platform_ansible_hostvars: Additional hostvars to add to the instance configuration +# - platform_state: The desired state of the platform instance (present, absent) +# - platform_instance_config_from_file: The current instance configuration from the platform configuration file +# - platform_molecule_ephemeral_directory: The directory where the instance configuration file should be written +# - platform_exported_config: The exported platform configuration from a xxx_platform role # -# Outputs: -# - __platform_instance_config: The updated list of instance configurations -# - __platform_instance_config_update_needed: Whether the instance configuration file needs to be updated -# - -- name: We are adding/updating an instance +- name: Platform Instance Config | Validate configuration + ansible.builtin.assert: + that: + - platform_state in ['present', 'absent'] + - platform_instance_config_from_file | type_debug == 'list' + - platform_exported_config is defined + - platform_exported_config is mapping + fail_msg: "Invalid platform instance configuration provided!" + success_msg: "Platform instance configuration validated." + +- name: 🐜 Show platform config + ansible.builtin.debug: + var: platform_exported_config + verbosity: 1 + +- name: 🐜 Show platform instance config from file + ansible.builtin.debug: + var: platform_instance_config_from_file + verbosity: 1 + +- name: Platform Instance Config | Create | Build new instance configuration when: platform_state == 'present' - block: - - name: Instance configuration {{ __platform_target_instance_config.instance }} is valid - ansible.builtin.assert: - that: - - __platform_target_instance_config is defined - - __platform_target_instance_config.instance is string - fail_msg: "Instance configuration for {{ __platform_target_instance_config.instance }} failed! Check the platform configuration." - success_msg: "Instance configuration for {{ __platform_target_instance_config.instance }} is defined" - - - name: Add user-specified hostvars to instance configuration - when: - - platform_molecule_cfg.hostvars is defined - - platform_molecule_cfg.hostvars is truthy - ansible.builtin.set_fact: - __platform_ansible_hostvars: "{{ __platform_ansible_hostvars | combine(platform_molecule_cfg.hostvars, recursive=true) }}" - - - name: 🪲 Current instance config - ansible.builtin.debug: - var: __platform_instance_config - verbosity: 1 - - - name: 🪲 New instance config - ansible.builtin.debug: - var: __platform_target_instance_config - verbosity: 1 - - - name: 🪲 Ansible hostvars - ansible.builtin.debug: - var: __platform_ansible_hostvars - verbosity: 1 - - - name: Instance name matching this already exists in configuration - when: - - __platform_instance_config is truthy - - __platform_target_instance_config.instance in __platform_instance_config | map(attribute='instance') | list - block: - - name: Mark config update as unneeded - ansible.builtin.set_fact: - __platform_instance_config_update_needed: false - - - name: Existing configuration does not match desired - when: __platform_target_instance_config != (__platform_instance_config | - selectattr('instance', 'equalto', __platform_target_instance_config.instance) | list | first) - block: - - name: Remove existing {{ __platform_target_instance_config.instance }} configuration (does not match) - ansible.builtin.set_fact: - __platform_instance_config: "{{ - __platform_instance_config | rejectattr('instance', 'equalto', __platform_target_instance_config.instance) | list }}" - __platform_instance_config_update_needed: true - - - name: Append new instance to config - ansible.builtin.set_fact: - __platform_instance_config: >- - {{ __platform_instance_config | default([], true) + [__platform_target_instance_config] - if __platform_target_instance_config | default(false, true) is truthy - else __platform_instance_config }} - -- name: Remove instance configuration + vars: + __platform_new_instance_configs: "{{ platform_exported_config.values() | map(attribute='instance_config') | list }}" + ansible.builtin.set_fact: + __platform_merged_instance_config: >- + {{ + platform_instance_config_from_file + | rejectattr('instance', 'in', __platform_new_instance_configs | map(attribute='instance') | list) + | list + + __platform_new_instance_configs + }} + +- name: Platform Instance Config | Destroy | Build new instance configuration when: platform_state == 'absent' - block: - - name: Remove existing {{ __platform_target_instance_config.instance }} configuration - when: - - __platform_instance_config is truthy - - __platform_target_instance_config.instance in __platform_instance_config | map(attribute='instance') | list - ansible.builtin.set_fact: - __platform_instance_config: "{{ - __platform_instance_config | rejectattr('instance', 'equalto', __platform_target_instance_config.instance) | list }}" - __platform_instance_config_update_needed: true + vars: + __platform_old_instance_configs: "{{ platform_exported_config.values() | map(attribute='instance_config') | list }}" + ansible.builtin.set_fact: + __platform_merged_instance_config: >- + {{ + platform_instance_config_from_file + | rejectattr('instance', 'in', __platform_old_instance_configs | map(attribute='instance') | list) + | list + }} + +- name: Platform Instance Config | Write new instance configuration + when: __platform_merged_instance_config is defined + ansible.builtin.copy: + content: | + {{ __platform_merged_instance_config | to_nice_yaml(indent=2) }} + dest: "{{ platform_molecule_ephemeral_directory }}/instance_config.yml" + mode: "0644" diff --git a/roles/platform/tasks/inventory.yml b/roles/platform/tasks/inventory.yml index c0bf64d..4d760d8 100644 --- a/roles/platform/tasks/inventory.yml +++ b/roles/platform/tasks/inventory.yml @@ -13,131 +13,9 @@ # # -- name: Generate new instance configuration - when: platform_state == 'present' - block: - - name: 🐜 Show platform config - ansible.builtin.debug: - var: platform_exported_config - verbosity: 1 - - #- name: Generate {{ platform_name }} instance configuration (Docker) - # when: platform_type == 'docker' - # ansible.builtin.set_fact: - # __platform_new_instance_configs: "{{ [{ - # 'instance': platform_name, - # 'connection': 'docker' - # }] }}" - # __platform_ansible_hostvars: "{{ { - # platform_name: { - # 'ansible_connection': 'community.docker.docker' - # } } }}" - - #- name: Generate {{ platform_name }} instance configuration (EC2) - # when: platform_type == 'ec2' - # ansible.builtin.set_fact: - # # NOTE: This depends on the ec2_platform_instance_config being set by the ec2_platform role - # __platform_new_instance_configs: "{{ [ec2_platform_instance_config] }}" - # __platform_ansible_hostvars: "{{ { - # ec2_platform_instance_config.instance: { - # 'ansible_connection': 'ssh', - # 'ansible_host': ec2_platform_instance_config.address, - # 'ansible_port': ec2_platform_instance_config.port, - # 'ansible_user': ec2_platform_instance_config.user, - # 'ansible_ssh_private_key_file': ec2_platform_instance_config.identity_file - # } } }}" - -- name: Load existing instance configuration - block: - - name: Load existing instance configuration file - ansible.builtin.slurp: - src: "{{ platform_molecule_ephemeral_directory }}/instance_config.yml" - register: __platform_current_instance_config_b64 - failed_when: false - - - name: Decode instance configuration data - ansible.builtin.set_fact: - __platform_instance_config: "{{ __platform_current_instance_config_b64.content | default('') | b64decode | from_yaml }}" - -# Build the Molecule instance configuration object -- name: Process instance configuration - when: __platform_instance_config is truthy or platform_exported_config is defined - loop: >- - {{ - platform_exported_config | dict2items - if platform_exported_config is defined - else - __platform_instance_config | default([], true) - }} - loop_control: - loop_var: __platform_target_instance_config - label: "{{ __platform_target_instance_config.instance }}" +- name: Configure Molecule instances ansible.builtin.include_tasks: "{{ role_path }}/tasks/instance_config.yml" -- name: Manage molecule instance config file - block: - - name: Write {{ platform_name }} instance config file - when: - - __platform_instance_config_update_needed - - __platform_instance_config | default(false, true)is truthy or __platform_new_instance_config | default(false, true) is truthy - ansible.builtin.copy: - content: "{{ __platform_instance_config | to_nice_yaml(indent=2) }}" - dest: "{{ platform_molecule_ephemeral_directory }}/instance_config.yml" - mode: "0600" - - # If the file would be empty, remove it - - name: Remove molecule instance config file - when: - - platform_state == 'absent' - - __platform_current_instance_config | default(false) is not truthy - - __platform_new_instance_config | default(false) is not truthy - ansible.builtin.file: - path: "{{ molecule_instance_config }}" - state: absent - -- name: Load existing molecule inventory - when: __platform_run_count | int > 1 or platform_state == 'absent' - block: - - name: Load existing molecule inventory file - ansible.builtin.slurp: - src: "{{ platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" - register: __platform_current_molecule_inventory_b64 - failed_when: false - - - name: Decode instance configuration data - ansible.builtin.set_fact: - __platform_molecule_inventory: "{{ __platform_current_molecule_inventory_b64.content | default({}) | b64decode | from_yaml }}" - -- name: Show instance configuration - ansible.builtin.debug: - var: __platform_instance_config - verbosity: 1 - -- name: Process instance inventory membership - when: __platform_instance_config | default(false) is truthy or __platform_new_instance_configs | default(false) is truthy +- name: Configure Molecule Ansible inventory ansible.builtin.include_tasks: "{{ role_path }}/tasks/ansible_inventory.yml" - loop: "{{ __platform_new_instance_configs if __platform_new_instance_configs is defined else __platform_instance_config | default([], true) }}" - loop_control: - loop_var: __platform_target_instance_config - label: "{{ __platform_target_instance_config.instance }}" - -- name: Write {{ platform_name }} to molecule inventory file - when: __platform_molecule_inventory is truthy - ansible.builtin.copy: - content: | - {{ __platform_molecule_inventory | default({}) | to_nice_yaml(indent=2) }} - dest: "{{ platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" - mode: "0600" - -- name: Force inventory refresh - ansible.builtin.meta: refresh_inventory - -- name: Fail if molecule group is missing - when: - - __platform_molecule_inventory is defined - - platform_state == 'present' - ansible.builtin.assert: - that: "'molecule' in groups" - fail_msg: | - molecule group was not found inside inventory groups: {{ groups }} diff --git a/roles/platform/tasks/main.yml b/roles/platform/tasks/main.yml index 7731887..4c21d90 100644 --- a/roles/platform/tasks/main.yml +++ b/roles/platform/tasks/main.yml @@ -1,17 +1,9 @@ --- # tasks file for platform -- name: Dump hostvars - ansible.builtin.debug: - var: vars - -- name: Show platform configuration - ansible.builtin.debug: - var: platform_config - - name: Process provided platform configuration vars: - __platform_config: "{{ platform_molecule_cfg if platform_molecule_cfg is truthy else platform_config }}" + __platform_config: "{{ [platform_molecule_cfg] if platform_molecule_cfg is truthy else platform_config }}" block: - name: Validate platform configuration ansible.builtin.include_tasks: "{{ role_path }}/tasks/validate_cfg.yml" diff --git a/roles/platform/tasks/provision.yml b/roles/platform/tasks/provision.yml index 3724f36..ea7df9c 100644 --- a/roles/platform/tasks/provision.yml +++ b/roles/platform/tasks/provision.yml @@ -23,26 +23,3 @@ ansible.builtin.include_role: name: "{{ ansible_collection_name }}.{{ __platform_type }}_platform" -#- name: Configure platform for docker type -# when: platform_type == 'docker' -# ansible.builtin.include_role: -# name: "{{ ansible_collection_name }}.docker_platform" -# vars: -# docker_platform_name: "{{ platform_name }}" -# docker_platform_state: present -# docker_platform_image: "{{ platform_molecule_cfg.image }}" -# docker_platform_systemd: "{{ platform_molecule_cfg.systemd | default(false) }}" -# docker_platform_modify_image: "{{ platform_molecule_cfg.modify_image | default(false) }}" -# docker_platform_modify_image_buildpath: "{{ platform_molecule_cfg.modify_image_buildpath | default(molecule_ephemeral_directory + '/build') }}" -# docker_platform_privileged: "{{ platform_molecule_cfg.privileged | default (false) }}" -# docker_platform_hostvars: "{{ platform_molecule_cfg.hostvars | default({}) }}" -# -#- name: Configure platform for ec2 type -# when: platform_type == 'ec2' -# ansible.builtin.include_role: -# name: "{{ ansible_collection_name }}.ec2_platform" -# vars: -# ec2_platform_name: "{{ platform_name }}" -# ec2_platform_state: present -# ec2_platform_definition: "{{ platform_molecule_cfg }}" - diff --git a/roles/platform/tasks/validate_cfg.yml b/roles/platform/tasks/validate_cfg.yml index 92150de..a0b4b83 100644 --- a/roles/platform/tasks/validate_cfg.yml +++ b/roles/platform/tasks/validate_cfg.yml @@ -5,6 +5,11 @@ # - __platform_config: The current list of platform configurations # +- name: Show platform config + ansible.builtin.debug: + var: __platform_config + verbosity: 1 + - name: Validate | Platform configuration ansible.builtin.assert: that: @@ -13,9 +18,20 @@ fail_msg: "No platform configuration provided, or platform configuration invalid!" success_msg: "Platform configuration provided." +- name: Show first item + ansible.builtin.debug: + var: __platform_config[0] + verbosity: 1 + +- name: Show first item name + ansible.builtin.debug: + var: __platform_config[0].name + verbosity: 1 + - name: Validate | Unique attributes ansible.builtin.assert: that: + - __platform_config | map(attribute='name') | length > 0 - __platform_config | map(attribute='name') | length == __platform_config | length - __platform_config | map(attribute='name') | list | length == __platform_config | map(attribute='name') | list | unique | list | length fail_msg: "No platforms defined or duplicate/missing platform names provided!" @@ -24,6 +40,7 @@ - name: Validate | Platform types ansible.builtin.assert: that: + - __platform_config | map(attribute='type') | length > 0 - __platform_config | map(attribute='type') | length == __platform_config | length - __platform_config | map(attribute='type') | unique | length == 1 fail_msg: "Types must be the same for all platforms provided!" diff --git a/roles/platform/vars/main.yml b/roles/platform/vars/main.yml index be77150..0c94435 100644 --- a/roles/platform/vars/main.yml +++ b/roles/platform/vars/main.yml @@ -4,8 +4,14 @@ # Filesystem location of the Molecule ephemeral directory. Should not need to be updated by the user of this role! platform_molecule_ephemeral_directory: "{{ molecule_ephemeral_directory }}" +platform_instance_config_path: "{{ molecule_ephemeral_directory }}/instance_config.yml" +platform_instance_config_from_file: "{{ (lookup('file', platform_instance_config_path, errors='ignore') or '[]') | from_yaml }}" -## INTERNAL VARIABLES -- DO NOT MODIFY ## +platform_inventory_config_path: "{{ molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" +platform_inventory_config_from_file: "{{ (lookup('file', platform_inventory_config_path, errors='ignore') or '{}') | from_yaml }}" + + +##⬇⬇ INTERNAL VARIABLES -- DO NOT MODIFY ⬇⬇## # Does the molecule instance configuration file need to be updated? (assume yes) __platform_instance_config_update_needed: true From f6d62bee25f07679e9a8cffd53d84667dca8944e Mon Sep 17 00:00:00 2001 From: syndr Date: Thu, 9 Jan 2025 19:28:48 -0700 Subject: [PATCH 13/18] Hostname, custom systemd containers Allow defining hostname Successfully create custom images --- molecule/default/molecule.yml | 9 +- roles/docker_platform/defaults/main.yml | 3 + roles/docker_platform/tasks/custom_image.yml | 67 -------------- .../tasks/custom_image/buildfiles.yml | 25 +++++ .../tasks/custom_image/main.yml | 92 +++++++++++++++++++ roles/docker_platform/tasks/main.yml | 2 +- roles/docker_platform/tasks/present.yml | 5 +- roles/docker_platform/templates/Dockerfile.j2 | 2 +- 8 files changed, 131 insertions(+), 74 deletions(-) delete mode 100644 roles/docker_platform/tasks/custom_image.yml create mode 100644 roles/docker_platform/tasks/custom_image/buildfiles.yml create mode 100644 roles/docker_platform/tasks/custom_image/main.yml diff --git a/molecule/default/molecule.yml b/molecule/default/molecule.yml index e0cfb06..423f79a 100644 --- a/molecule/default/molecule.yml +++ b/molecule/default/molecule.yml @@ -12,15 +12,19 @@ platforms: type: docker image: geerlingguy/docker-rockylinux9-ansible:latest systemd: True - modify_image: False privileged: False hostvars: test_hostvar: test + - name: ansible-collection-molecule-docker-rockylinux9-customized + type: docker + image: rockylinux/rockylinux:9-ubi + systemd: True + exec_systemd: True + privileged: False - name: ansible-collection-molecule-docker-fedora41 type: docker image: geerlingguy/docker-fedora41-ansible:latest systemd: True - modify_image: False privileged: False hostvars: test_hostvar: test @@ -28,7 +32,6 @@ platforms: type: docker image: geerlingguy/docker-ubuntu2204-ansible:latest systemd: True - modify_image: False privileged: False hostvars: test_hostvar: test diff --git a/roles/docker_platform/defaults/main.yml b/roles/docker_platform/defaults/main.yml index b061b7a..5a2cd1a 100644 --- a/roles/docker_platform/defaults/main.yml +++ b/roles/docker_platform/defaults/main.yml @@ -24,6 +24,9 @@ docker_platform_container_defaults: # Command to run in the container command: "" + # Container hostname + hostname: molecule-ci-{{ __docker_platform_instance.name }} + # Number of CPUs to allocate to the container cpus: 2 diff --git a/roles/docker_platform/tasks/custom_image.yml b/roles/docker_platform/tasks/custom_image.yml deleted file mode 100644 index 1a2152b..0000000 --- a/roles/docker_platform/tasks/custom_image.yml +++ /dev/null @@ -1,67 +0,0 @@ ---- -# Build a custom image based upon the provided image, running Systemd as PID 1. -# -# Expected variables: -# - __docker_platform_config_custom: The desired Molecule platform configurations requiring custom entrypoint configuration for systemd -# - docker_platform_modify_image_buildpath: The path to the build directory for the custom image -# - -- name: Exec Systemd | Build path exists - ansible.builtin.file: - path: "{{ docker_platform_modify_image_buildpath }}" - state: directory - mode: 0755 - -- name: Exec Systemd | Build file exists - loop: - - bash.service.j2 - - entrypoint.sh.j2 - - Dockerfile.j2 - loop_control: - loop_var: __docker_platform_item - ansible.builtin.template: - src: templates/{{ __docker_platform_item }} - dest: "{{ docker_platform_modify_image_buildpath}}/{{ __docker_platform_item | regex_replace('\\.j2$', '') }}" - -- name: Exec Systemd | Launch image build - loop: "{{ __docker_platform_config_custom }}" - loop_control: - loop_var: __docker_platform_definition - label: "{{ __docker_platform_definition.name }}" - vars: - __docker_platform_built_image_name: "molecule-local-build/{{ __docker_platform_definition.image | split(':') | first | split('/') | last }}-custom" - community.docker.docker_image: - name: "{{ __docker_platform_built_image_name }}" - build: - path: "{{ docker_platform_modify_image_buildpath }}" - cache_from: "{{ __docker_platform_definition.image }}" - source: build - force_source: true # Always build a new image when this is run - tag: latest - register: __docker_platform_image_build_tasks - async: 600 - poll: 0 - -- name: 🐜 Exec Systemd | Show image build job tasks - ansible.builtin.debug: - var: __docker_platform_image_build_tasks - verbosity: 1 - -- name: Exec Systemd | Wait for image build to complete - ansible.builtin.async_status: - jid: "{{ __docker_platform_image_build_tasks.ansible_job_id }}" - register: __docker_platform_image_build_result - until: __docker_platform_image_build_result.finished - retries: 60 - delay: 10 - -- name: Exec Systemd | Clean up async cache - ansible.builtin.async_status: - jid: "{{ __docker_platform_image_build_tasks.ansible_job_id }}" - mode: cleanup - -- name: Exec Systemd | Show image build details - ansible.builtin.debug: - var: __docker_platform_image_build_result - verbosity: 1 - diff --git a/roles/docker_platform/tasks/custom_image/buildfiles.yml b/roles/docker_platform/tasks/custom_image/buildfiles.yml new file mode 100644 index 0000000..8ca043e --- /dev/null +++ b/roles/docker_platform/tasks/custom_image/buildfiles.yml @@ -0,0 +1,25 @@ +--- +# Deploy buildfiles for a custom docker image +# +# Expected input: +# __docker_platform_definition: The Molecule Docker platform configuration (dict) +# + +- name: Exec Systemd | Build path exists + ansible.builtin.file: + path: "{{ docker_platform_modify_image_buildpath }}/{{ __docker_platform_definition.name }}" + state: directory + mode: "0755" + +- name: Exec Systemd | Build file exists + loop: + - bash.service.j2 + - entrypoint.sh.j2 + - Dockerfile.j2 + loop_control: + loop_var: __docker_platform_item + ansible.builtin.template: + src: templates/{{ __docker_platform_item }} + dest: "{{ docker_platform_modify_image_buildpath}}/{{ __docker_platform_definition.name }}/{{ __docker_platform_item | regex_replace('\\.j2$', '') }}" + mode: "0644" + diff --git a/roles/docker_platform/tasks/custom_image/main.yml b/roles/docker_platform/tasks/custom_image/main.yml new file mode 100644 index 0000000..66eab31 --- /dev/null +++ b/roles/docker_platform/tasks/custom_image/main.yml @@ -0,0 +1,92 @@ +--- +# Build a custom image based upon the provided image, running Systemd as PID 1. +# +# Expected variables: +# - __docker_platform_config_custom: The desired Molecule platform configurations requiring custom entrypoint configuration for systemd +# - docker_platform_modify_image_buildpath: The path to the build directory for the custom image +# + +- name: Build a custom docker image to launch systemd with PID 1 + block: + - name: Exec Systemd | Prepare build directory + loop: "{{ __docker_platform_config_custom }}" + loop_control: + loop_var: __docker_platform_definition + label: "{{ __docker_platform_definition.name }}" + ansible.builtin.include_tasks: buildfiles.yml + + - name: Exec Systemd | Launch image build + loop: "{{ __docker_platform_config_custom }}" + loop_control: + loop_var: __docker_platform_definition + label: "{{ __docker_platform_definition.name }}" + vars: + __docker_platform_built_image_name: molecule-local-build/{{ __docker_platform_definition.image | split(':') | first | split('/') | last }}-custom + community.docker.docker_image: + name: "{{ __docker_platform_built_image_name }}" + build: + path: "{{ docker_platform_modify_image_buildpath }}/{{ __docker_platform_definition.name }}" + cache_from: "{{ __docker_platform_definition.image }}" + source: build + force_source: true # Always build a new image when this is run + tag: "{{ __docker_platform_definition.image | split(':') | last }}" + register: __docker_platform_image_build_tasks + async: 600 + poll: 0 + + - name: 🐜 Exec Systemd | Show image build job tasks + ansible.builtin.debug: + var: __docker_platform_image_build_tasks + verbosity: 1 + + - name: Exec Systemd | Wait for image build to complete + loop: "{{ __docker_platform_image_build_tasks.results }}" + loop_control: + loop_var: __docker_platform_image_build_task + label: "{{ __docker_platform_image_build_task.__docker_platform_definition.name }}" + ansible.builtin.async_status: + jid: "{{ __docker_platform_image_build_task.ansible_job_id }}" + register: __docker_platform_image_build_result + until: __docker_platform_image_build_result.finished + retries: 60 + delay: 10 + + - name: Exec Systemd | Clean up async cache + loop: "{{ __docker_platform_image_build_tasks.results }}" + loop_control: + loop_var: __docker_platform_image_build_task_cleanup + label: "{{ __docker_platform_image_build_task_cleanup.__docker_platform_definition.name }}" + ansible.builtin.async_status: + jid: "{{ __docker_platform_image_build_task_cleanup.ansible_job_id }}" + mode: cleanup + + - name: Exec Systemd | Show image build details + ansible.builtin.debug: + var: __docker_platform_image_build_result + verbosity: 1 + + - name: Exec Systemd | Generate new platform configuration + # Replace the image in the platform configuration with the custom image + ansible.builtin.set_fact: + __docker_platform_config_runtime: >- + {%- set __docker_platform_config_custom_build = [] -%} + {%- for __docker_platform_definition in __docker_platform_config_custom -%} + {%- set __docker_platform_entry_update = __docker_platform_definition | combine({ + 'image': (__docker_platform_image_build_result.results + | json_query('[?__docker_platform_image_build_task.__docker_platform_definition.name==`' + __docker_platform_definition.name + '`]') + | first).image.RepoTags | first, + 'pull': 'never', + 'hostname': 'custom-' + __docker_platform_definition.name + }) -%} + {%- set _ = __docker_platform_config_custom_build.append(__docker_platform_entry_update) -%} + {%- endfor -%} + {{ __docker_platform_config_custom_build }} + + - name: Exec Systemd | Merge new platform configuration + ansible.builtin.set_fact: + docker_platform_config: >- + {{ + docker_platform_config | rejectattr('name', 'in', __docker_platform_config_runtime | map(attribute='name') | list) + | list + __docker_platform_config_runtime + }} + diff --git a/roles/docker_platform/tasks/main.yml b/roles/docker_platform/tasks/main.yml index 733b7a3..01e4300 100644 --- a/roles/docker_platform/tasks/main.yml +++ b/roles/docker_platform/tasks/main.yml @@ -19,7 +19,7 @@ {%- for __platform in __platform_config -%} {%- set _ = __platform_instances.update({ __platform.name: { - 'hostvars': __platform.hostvars | combine(docker_platform_hostvars_base), + 'hostvars': __platform.hostvars | default({}) | combine(docker_platform_hostvars_base), 'instance_config': { 'instance': __platform.name, 'connection': 'docker' diff --git a/roles/docker_platform/tasks/present.yml b/roles/docker_platform/tasks/present.yml index ef74698..4dd786f 100644 --- a/roles/docker_platform/tasks/present.yml +++ b/roles/docker_platform/tasks/present.yml @@ -16,7 +16,7 @@ | list }} when: __docker_platform_config_custom | length > 0 - ansible.builtin.include_tasks: "{{ role_path }}/tasks/custom_image.yml" + ansible.builtin.include_tasks: "{{ role_path }}/tasks/custom_image/main.yml" - name: Create Platform | Docker container is provisioned loop: "{{ docker_platform_config }}" @@ -27,6 +27,7 @@ __docker_platform_container_cfg: name: "{{ __docker_platform_instance.name}}" image: "{{ __docker_platform_instance.image | default(docker_platform_container_defaults.image) }}" + hostname: "{{ __docker_platform_instance.hostname | default(docker_platform_container_defaults.hostname) }}" command: "{{ __docker_platform_instance.command | default(docker_platform_container_defaults.command) }}" volumes: "{{ __docker_platform_instance.volumes | default(docker_platform_container_defaults.volumes) }}" privileged: "{{ __docker_platform_instance.privileged | default(docker_platform_container_defaults.privileged) }}" @@ -58,7 +59,7 @@ state: "{{ __docker_platform_container_cfg.state }}" command: "{{ __docker_platform_container_cfg.command }}" log_driver: json-file - hostname: molecule-ci-{{ __docker_platform_container_cfg.name }} + hostname: "{{ __docker_platform_container_cfg.hostname }}" init: false cgroupns_mode: "{{ 'host' if __docker_platform_container_cfg.systemd is true else 'private' }}" privileged: "{{ __docker_platform_container_cfg.privileged }}" diff --git a/roles/docker_platform/templates/Dockerfile.j2 b/roles/docker_platform/templates/Dockerfile.j2 index f1d3101..1ce8727 100644 --- a/roles/docker_platform/templates/Dockerfile.j2 +++ b/roles/docker_platform/templates/Dockerfile.j2 @@ -1,5 +1,5 @@ -FROM {{ docker_platform_image }} +FROM {{ __docker_platform_definition.image }} COPY bash.service /etc/systemd/system/bash.service COPY entrypoint.sh /entrypoint.sh RUN chown root:root /entrypoint.sh \ From 9c5c5874b09f5f66b1cbc08d6a0951ce556669fb Mon Sep 17 00:00:00 2001 From: syndr Date: Fri, 10 Jan 2025 16:56:03 -0700 Subject: [PATCH 14/18] Fix custom docker image build Systemd path is lauched as PID 1 Commands can be run pre-entrypoint in dockerfile (IE: dnf install -y sudo) --- galaxy.yml | 2 +- molecule/default/molecule.yml | 4 ++++ roles/docker_platform/defaults/main.yml | 12 ++++++++++-- .../tasks/custom_image/buildfiles.yml | 3 +-- roles/docker_platform/templates/Dockerfile.j2 | 14 ++++++-------- roles/docker_platform/templates/bash.service.j2 | 12 ------------ .../docker_platform/templates/entrypoint.sh.j2 | 17 ----------------- 7 files changed, 22 insertions(+), 42 deletions(-) delete mode 100644 roles/docker_platform/templates/bash.service.j2 delete mode 100644 roles/docker_platform/templates/entrypoint.sh.j2 diff --git a/galaxy.yml b/galaxy.yml index 4590aad..0c48788 100644 --- a/galaxy.yml +++ b/galaxy.yml @@ -8,7 +8,7 @@ namespace: syndr name: molecule # The version of the collection. Must be compatible with semantic versioning -version: 1.6.0 +version: 2.0.0 # The path to the Markdown (.md) readme file. This path is relative to the root of the collection readme: README.md diff --git a/molecule/default/molecule.yml b/molecule/default/molecule.yml index 423f79a..72c4999 100644 --- a/molecule/default/molecule.yml +++ b/molecule/default/molecule.yml @@ -20,7 +20,11 @@ platforms: image: rockylinux/rockylinux:9-ubi systemd: True exec_systemd: True + exec_systemd_build_commands: + - dnf install -y sudo privileged: False + hostvars: + test_hostvar: test - name: ansible-collection-molecule-docker-fedora41 type: docker image: geerlingguy/docker-fedora41-ansible:latest diff --git a/roles/docker_platform/defaults/main.yml b/roles/docker_platform/defaults/main.yml index 5a2cd1a..756a0a3 100644 --- a/roles/docker_platform/defaults/main.yml +++ b/roles/docker_platform/defaults/main.yml @@ -25,7 +25,7 @@ docker_platform_container_defaults: command: "" # Container hostname - hostname: molecule-ci-{{ __docker_platform_instance.name }} + hostname: molecule-ci-{{ __docker_platform_instance.name | default('instance') }} # Number of CPUs to allocate to the container cpus: 2 @@ -101,12 +101,20 @@ docker_platform_container_defaults: # WARNING: # - This can cause issues with some containers # - Not required if the container is already built with systemd running as PID 1 - # - Expects the container to have systemd installed + # - Expects the container to have systemd packages present + # - Rebuilds the container with a custom entrypoint, provided by 'exec_systemd_path' exec_systemd: false # Path to the systemd binary in the container + # - This is only used if 'exec_systemd' is true exec_systemd_path: /usr/lib/systemd/systemd + # List of commands to run as part of the docker build process to enable systemd + # - This is only used if 'exec_systemd' is true + # - Each command should be a string + # - Commands are run in the order they are defined, using the docker RUN directive + exec_systemd_build_commands: [] + # Tmpfs mounts to add to the container tmpfs: [] diff --git a/roles/docker_platform/tasks/custom_image/buildfiles.yml b/roles/docker_platform/tasks/custom_image/buildfiles.yml index 8ca043e..3882dad 100644 --- a/roles/docker_platform/tasks/custom_image/buildfiles.yml +++ b/roles/docker_platform/tasks/custom_image/buildfiles.yml @@ -12,9 +12,8 @@ mode: "0755" - name: Exec Systemd | Build file exists + # Add additional build files to this list as needed loop: - - bash.service.j2 - - entrypoint.sh.j2 - Dockerfile.j2 loop_control: loop_var: __docker_platform_item diff --git a/roles/docker_platform/templates/Dockerfile.j2 b/roles/docker_platform/templates/Dockerfile.j2 index 1ce8727..e54d6d7 100644 --- a/roles/docker_platform/templates/Dockerfile.j2 +++ b/roles/docker_platform/templates/Dockerfile.j2 @@ -1,11 +1,9 @@ FROM {{ __docker_platform_definition.image }} -COPY bash.service /etc/systemd/system/bash.service -COPY entrypoint.sh /entrypoint.sh -RUN chown root:root /entrypoint.sh \ - && chmod 755 /entrypoint.sh \ - && chown root:root /etc/systemd/system/bash.service \ - && chmod 644 /etc/systemd/system/bash.service \ - && systemctl enable bash.service -ENTRYPOINT ["/entrypoint.sh"] + +{% for __run_command in __docker_platform_definition.exec_systemd_build_commands %} +RUN {{ __run_command }} +{% endfor %} + +ENTRYPOINT ["{{ __docker_platform_definition.exec_systemd_path | default(docker_platform_container_defaults.exec_systemd_path) }}"] diff --git a/roles/docker_platform/templates/bash.service.j2 b/roles/docker_platform/templates/bash.service.j2 deleted file mode 100644 index b2f8133..0000000 --- a/roles/docker_platform/templates/bash.service.j2 +++ /dev/null @@ -1,12 +0,0 @@ -[Unit] -Description=Start bash shell attached to container STDIN/STDOUT - -[Service] -Type=simple -PassEnvironment=PATH LD_LIBRARY_PATH -ExecStart=/bin/bash -c "echo Attaching to pipes of PID `cat container-pipes-pid` && exec /bin/bash < /proc/`cat container-pipes-pid`/fd/0 > /proc/`cat container-pipes-pid`/fd/1 2>/proc/`cat container-pipes-pid`/fd/2" -ExecStopPost=/usr/bin/systemctl exit $EXIT_STATUS - -[Install] -WantedBy=multi-user.target rescue.target - diff --git a/roles/docker_platform/templates/entrypoint.sh.j2 b/roles/docker_platform/templates/entrypoint.sh.j2 deleted file mode 100644 index a38822e..0000000 --- a/roles/docker_platform/templates/entrypoint.sh.j2 +++ /dev/null @@ -1,17 +0,0 @@ -#!/bin/bash - -echo Start a long-running process to keep the container pipes open -sleep infinity < /proc/1/fd/0 > /proc/1/fd/1 2>&1 & - -echo Wait a bit before retrieving the PID -sleep 1 - -echo Save the long-running PID on file -echo $! > /container-pipes-pid - -echo Start systemd as PID 1 -exec /usr/lib/systemd/systemd - -echo Attaching to pipes of PID `cat container-pipes-pid` -exec /bin/bash < /proc/`cat container-pipes-pid`/fd/0 > /proc/`cat container-pipes-pid`/fd/1 2>/proc/`cat container-pipes-pid`/fd/2 - From 33c673ff64da6fb9fb8702e59e90e1971cd19eb9 Mon Sep 17 00:00:00 2001 From: syndr Date: Mon, 13 Jan 2025 20:00:18 -0700 Subject: [PATCH 15/18] Asynchronous ec2 instance creation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Refactor EC2 creation & destruction logic * Accept list input, use async for long-running tasks 🐇 Add ARA config for ec2_instance test Fix idempotence on inventory/instance config removal --- molecule/default/molecule.yml | 2 +- molecule/ec2_platform/molecule.yml | 8 +- roles/docker_platform/README.md | 17 +- roles/ec2_platform/defaults/main.yml | 21 +- roles/ec2_platform/tasks/absent.yml | 156 +++++++--- roles/ec2_platform/tasks/main.yml | 39 ++- roles/ec2_platform/tasks/present.yml | 321 +++++++++++++-------- roles/ec2_platform/tasks/validate_cfg.yml | 43 +++ roles/ec2_platform/vars/main.yml | 5 +- roles/platform/tasks/ansible_inventory.yml | 50 ++-- roles/platform/tasks/instance_config.yml | 14 +- roles/platform/tasks/validate_cfg.yml | 12 +- 12 files changed, 471 insertions(+), 217 deletions(-) create mode 100644 roles/ec2_platform/tasks/validate_cfg.yml diff --git a/molecule/default/molecule.yml b/molecule/default/molecule.yml index 72c4999..14a84fc 100644 --- a/molecule/default/molecule.yml +++ b/molecule/default/molecule.yml @@ -6,7 +6,7 @@ driver: name: default options: managed: true - login_cmd_template: 'docker exec -ti {instance} bash' + login_cmd_template: 'docker exec -ti {instance} bash --login' platforms: - name: ansible-collection-molecule-docker-rockylinux9 type: docker diff --git a/molecule/ec2_platform/molecule.yml b/molecule/ec2_platform/molecule.yml index 8a106b8..5be25a6 100644 --- a/molecule/ec2_platform/molecule.yml +++ b/molecule/ec2_platform/molecule.yml @@ -32,11 +32,17 @@ provisioner: gathering: explicit playbook_vars_root: top verbosity: ${ANSIBLE_VERBOSITY:-0} + env: + ARA_API_CLIENT: ${ARA_API_CLIENT:-'http'} + ARA_API_SERVER: ${ARA_API_SERVER:-'http://localhost:8000'} + ARA_DEFAULT_LABELS: ${ARA_DEFAULT_LABELS:-'testing,molecule'} + # To use Ara with molecule: + # export the ANSIBLE_CALLBACK_PLUGINS env var with the output of 'python3 -m ara.setup.callback_plugins' + ANSIBLE_CALLBACK_PLUGINS: ${ANSIBLE_CALLBACK_PLUGINS} scenario: create_sequence: - dependency - create - - prepare check_sequence: - dependency - cleanup diff --git a/roles/docker_platform/README.md b/roles/docker_platform/README.md index 3ea4029..86d5d6f 100644 --- a/roles/docker_platform/README.md +++ b/roles/docker_platform/README.md @@ -52,6 +52,9 @@ cap_drop: [] # Command to run in the container command: "" +# Container hostname +hostname: molecule-ci-{{ __docker_platform_instance.name | default('instance') }} + # Number of CPUs to allocate to the container cpus: 2 @@ -88,9 +91,6 @@ dns_servers: [] # The URL or Unix socket path used to connect to the Docker API docker_host: "{{ lookup('env', 'DOCKER_HOST') | default('unix:///var/run/docker.sock') }}" -# Path to a file, present on the controller, containing environment variables FOO=BAR -env_file: "" - # Dict of host-to-IP mappings, where each host name is a key in the dictionary. Each host name will be added to the container’s /etc/hosts file. # Instead of an IP address, the special value host-gateway can also be used, which resolves to the host’s gateway # IP and allows containers to connect to services running on the host. @@ -129,12 +129,20 @@ systemd: false # WARNING: # - This can cause issues with some containers # - Not required if the container is already built with systemd running as PID 1 -# - Expects the container to have systemd installed +# - Expects the container to have systemd packages present +# - Rebuilds the container with a custom entrypoint, provided by 'exec_systemd_path' exec_systemd: false # Path to the systemd binary in the container +# - This is only used if 'exec_systemd' is true exec_systemd_path: /usr/lib/systemd/systemd +# List of commands to run as part of the docker build process to enable systemd +# - This is only used if 'exec_systemd' is true +# - Each command should be a string +# - Commands are run in the order they are defined, using the docker RUN directive +exec_systemd_build_commands: [] + # Tmpfs mounts to add to the container tmpfs: [] @@ -148,7 +156,6 @@ tmpfs: [] # # SELinux hosts can additionally use z or Z to use a shared or private label for the volume. volumes: [] - ``` This role should not be used directly in a playbook, and should instead be used via the `molecule.platform` role. diff --git a/roles/ec2_platform/defaults/main.yml b/roles/ec2_platform/defaults/main.yml index 6d6f39d..0514063 100644 --- a/roles/ec2_platform/defaults/main.yml +++ b/roles/ec2_platform/defaults/main.yml @@ -4,8 +4,23 @@ # Name of this Molecule platform ec2_platform_name: instance +# Whether this platform should be deployed (present/absent) +ec2_platform_state: "{{ platform_target_state | default('present') }}" + +# Merge the defaults with any options provided to this role +ec2_platforms: >- + {%- set __platforms = [] -%} + {%- for __platform in ec2_platform_definition -%} + {%- set __platform = ec2_platform_defaults | combine(__platform) -%} + {%- set _ = __platforms.append(__platform) -%} + {%- endfor -%} + {{ __platforms }} + +# Molecule platform configuration (list of dictionaries) +ec2_platform_definition: "{{ [platform_target_config] if platform_target_config is defined else (molecule_yml.platforms | default([])) }}" + # Run config handling -ec2_platform_default_run_id: "{{ lookup('password', '/dev/null chars=ascii_lowercase length=5') }}" +ec2_platform_default_run_id: "{{ ec2_platform_run_config_from_file.run_id | default(lookup('password', '/dev/null chars=ascii_lowercase length=5')) }}" ec2_platform_default_run_config: run_id: "{{ ec2_platform_default_run_id }}" @@ -16,7 +31,8 @@ ec2_platform_run_config: '{{ ec2_platform_default_run_config | combine(ec2_platf # Platform settings handling ec2_platform_default_assign_public_ip: true ec2_platform_default_aws_profile: "{{ lookup('env', 'AWS_PROFILE') }}" -ec2_platform_default_boot_wait_seconds: 120 +ec2_platform_default_boot_wait_seconds: 10 +ec2_platform_default_instance_creation_timeout: 300 ec2_platform_default_instance_type: t3a.medium ec2_platform_default_key_inject_method: cloud-init # valid values: [cloud-init, ec2] ec2_platform_default_key_name: "molecule-{{ ec2_platform_run_config.run_id }}" @@ -47,6 +63,7 @@ ec2_platform_defaults: assign_public_ip: "{{ ec2_platform_default_assign_public_ip }}" aws_profile: "{{ ec2_platform_default_aws_profile }}" boot_wait_seconds: "{{ ec2_platform_default_boot_wait_seconds }}" + instance_creation_timeout: "{{ ec2_platform_default_instance_creation_timeout }}" instance_type: "{{ ec2_platform_default_instance_type }}" key_inject_method: "{{ ec2_platform_default_key_inject_method }}" key_name: "{{ ec2_platform_default_key_name }}" diff --git a/roles/ec2_platform/tasks/absent.yml b/roles/ec2_platform/tasks/absent.yml index d27d841..3d2feec 100644 --- a/roles/ec2_platform/tasks/absent.yml +++ b/roles/ec2_platform/tasks/absent.yml @@ -1,75 +1,135 @@ --- # Remove ec2 test resources -- name: Load Molecule instance config +- name: EC2 Deprovision | Load Molecule instance config ansible.builtin.set_fact: __ec2_instance_molecule_instance_config: "{{ (lookup('file', molecule_instance_config, errors='ignore') or '{}') | from_yaml }}" -- name: Validate platform configurations - ansible.builtin.assert: - that: - - ec2_platform is mapping - - ec2_platform.name is string and ec2_platform.name | length > 0 - - ec2_platform.aws_profile is string - - ec2_platform.key_inject_method is in ["cloud-init", "ec2"] - - ec2_platform.key_name is string and ec2_platform.key_name | length > 0 - - ec2_platform.region is string - - ec2_platform.security_group_name is string and ec2_platform.security_group_name | length > 0 - - ec2_platform.security_groups is sequence - - ec2_platform.vpc_id is string - - ec2_platform.vpc_subnet_id is string and ec2_platform.vpc_subnet_id | length > 0 - quiet: true - -- name: Look up subnets to determine VPCs (if needed) +- name: EC2 Deprovision | Look up subnets to determine VPCs (if needed) + loop: "{{ ec2_platforms | selectattr('vpc_id', 'undefined') }}" + loop_control: + loop_var: __ec2_platform + label: "{{ __ec2_platform.name }}" amazon.aws.ec2_vpc_subnet_info: - profile: "{{ ec2_platform.aws_profile | default(omit) }}" - region: "{{ ec2_platform.region | default(omit) }}" - subnet_ids: "{{ ec2_platform.vpc_subnet_id }}" - when: not ec2_platform.vpc_id + subnet_ids: "{{ __ec2_platform.vpc_subnet_id }}" + region: "{{ __ec2_platform.region }}" register: __ec2_platform_subnet_info -- name: Validate discovered information +- name: EC2 Deprovision | Validate discovered information + loop: "{{ ec2_platforms | selectattr('vpc_id', 'undefined') }}" + loop_control: + loop_var: __ec2_platform + label: "{{ __ec2_platform.name }}" + vars: + __ec2_platform_discovered_vpc_id: >- + {{ + __ec2_platform.vpc_id + or + (__ec2_platform_subnet_info.results + | selectattr('__ec2_platform.name', 'equalto', __ec2_platform.name)).subnets.vpc_id + }} ansible.builtin.assert: - that: ec2_platform.vpc_id or (__ec2_platform_subnet_info.subnets | length > 0) + that: + - (__ec2_platform.vpc_id or (__ec2_platform_discovered_vpc_id | length > 0)) quiet: true - fail_msg: "No VPCs found for subnet: {{ ec2_platform.vpc_subnet_id }}" -- name: Look up EC2 instance by tag +# TODO: Store instance id in platform run config on disk, and only look up instances if needed +# TODO: Don't delete all instances if more are returned than in the provided platform config +# (IE: if the platform config is a subset of the full scenario config) + +- name: EC2 Deprovision | Look up EC2 instance by tag + loop: "{{ ec2_platforms }}" + loop_control: + loop_var: __ec2_platform + label: "{{ __ec2_platform.name }}" amazon.aws.ec2_instance_info: - profile: "{{ ec2_platform.aws_profile | default(omit) }}" - region: "{{ ec2_platform.region | default(omit) }}" + profile: "{{ __ec2_platform.aws_profile | default(omit) }}" + region: "{{ __ec2_platform.region | default(omit) }}" filters: "tag:molecule-run-id": "{{ ec2_platform_run_config.run_id }}" + "tag:instance": "{{ __ec2_platform.name }}" register: __ec2_instance_info -- name: Destroy ephemeral EC2 instances - when: __ec2_instance_info.instances | length > 0 - amazon.aws.ec2_instance: - profile: "{{ ec2_platform.aws_profile | default(omit) }}" - region: "{{ ec2_platform.region | default(omit) }}" - instance_ids: "{{ __ec2_instance_info.instances | map(attribute='instance_id') | list }}" - vpc_subnet_id: "{{ ec2_platform.vpc_subnet_id }}" - state: absent - register: __ec2_instance_destroy +- name: EC2 Deprovision | Remove instances + vars: + __ec2_platform_existing_instances: "{{ __ec2_instance_info.results | map(attribute='instances') | flatten }}" + block: + - name: EC2 Deprovision | 🐜 Show EC2 instances + ansible.builtin.debug: + var: __ec2_platform_existing_instances + verbosity: 1 + + - name: EC2 Deprovision | Validate EC2 instances + ansible.builtin.assert: + # WARNING: This will fail if platforms were to support more than one instance + that: + - __ec2_platform_existing_instances | length <= ec2_platforms | length + fail_msg: "More instances found than expected! Found {{ __ec2_platform_existing_instances | length }} instances." + success_msg: "{{ __ec2_platform_existing_instances | length }} instances found for termination." + + - name: EC2 Deprovision | Destroy ephemeral EC2 instance + loop: "{{ __ec2_platform_existing_instances }}" + loop_control: + loop_var: __ec2_platform_existing_instance + label: "{{ __ec2_platform_existing_instance.tags.instance }}" + vars: + __ec2_platform: "{{ ec2_platforms | selectattr('name', 'equalto', __ec2_platform_existing_instance.tags.instance) | first }}" + amazon.aws.ec2_instance: + profile: "{{ __ec2_platform.aws_profile | default(omit) }}" + region: "{{ __ec2_platform.region | default(omit) }}" + instance_ids: "{{ __ec2_platform_existing_instance.instance_id }}" + vpc_subnet_id: "{{ __ec2_platform.vpc_subnet_id }}" + state: absent + register: __ec2_instance_destroy_job + async: 300 + poll: 0 + + - name: EC2 Deprovision | 🐜 Show EC2 instance destroy job status + ansible.builtin.debug: + var: __ec2_instance_destroy_job + verbosity: 1 + + - name: EC2 Deprovision | Instance destruction is complete + loop: "{{ __ec2_instance_destroy_job.results }}" + loop_control: + loop_var: __ec2_instance_destroy_result + label: "{{ __ec2_instance_destroy_result.__ec2_platform_existing_instance.tags.instance }}" + ansible.builtin.async_status: + jid: "{{ __ec2_instance_destroy_result.ansible_job_id }}" + register: __ec2_instance_destroy_results + until: __ec2_instance_destroy_results.finished + retries: "{{ (300 / 5) | int }}" + delay: 5 -- name: Destroy ephemeral security groups (if needed) +- name: EC2 Deprovision | Destroy ephemeral security groups (if needed) + loop: "{{ ec2_platforms | selectattr('security_groups', 'falsy') }}" + loop_control: + loop_var: __ec2_platform + label: "{{ __ec2_platform.name }}" amazon.aws.ec2_security_group: - profile: "{{ ec2_platform.aws_profile | default(omit) }}" - region: "{{ ec2_platform.region | default(omit) }}" - vpc_id: "{{ ec2_platform.vpc_id or __ec2_platform_subnet_info.subnets[0] }}" - name: "{{ ec2_platform.security_group_name }}" + profile: "{{ __ec2_platform.aws_profile | default(omit) }}" + region: "{{ __ec2_platform.region | default(omit) }}" + vpc_id: "{{ __ec2_platform.vpc_id or __ec2_platform_subnet_info.subnets[0] }}" + name: "{{ __ec2_platform.security_group_name }}" state: absent - when: ec2_platform.security_groups | length == 0 -- name: Destroy ephemeral keys (if needed) +- name: EC2 Deprovision | Destroy ephemeral keys (if needed) + loop: "{{ ec2_platforms | selectattr('key_inject_method', 'equalto', 'ec2') }}" + loop_control: + loop_var: __ec2_platform + label: "{{ __ec2_platform.name }}" amazon.aws.ec2_key: - profile: "{{ ec2_platform.aws_profile | default(omit) }}" - region: "{{ ec2_platform.region | default(omit) }}" - name: "{{ ec2_platform.key_name }}" + profile: "{{ __ec2_platform.aws_profile | default(omit) }}" + region: "{{ __ec2_platform.region | default(omit) }}" + name: "{{ __ec2_platform.key_name }}" state: absent - when: ec2_platform.key_inject_method == "ec2" -- name: Remove ec2 instance config file +- name: EC2 Deprovision | Remove ec2 instance config file + # Only run when this execution is operating on the full scenario configuration + # - we want to avoid removing the instance config if there may still be deployed instances + # (IE: if this role is called on individual items in the platforms list) + # - Molecule's builtin ephemeral dir cleanup should remove the instance config in this case + when: ec2_platform_definition | length == molecule_yml.platforms | default([]) | length ansible.builtin.file: path: "{{ ec2_platform_run_config_path }}" state: absent diff --git a/roles/ec2_platform/tasks/main.yml b/roles/ec2_platform/tasks/main.yml index 02eaffe..64f99eb 100644 --- a/roles/ec2_platform/tasks/main.yml +++ b/roles/ec2_platform/tasks/main.yml @@ -6,16 +6,14 @@ var: ec2_platform_definition verbosity: 1 -# Merge the defaults with any options provided to this role -- name: Generate runtime configuration - ansible.builtin.set_fact: - ec2_platform: "{{ ec2_platform_defaults | combine(ec2_platform_definition | default({})) }}" - - name: 🦋 Show ec2_platform ansible.builtin.debug: var: ec2_platform verbosity: 1 +- name: Validate configuration + ansible.builtin.include_tasks: "{{ role_path }}/tasks/validate_cfg.yml" + - name: Platform is deployed ansible.builtin.include_tasks: "{{ role_path }}/tasks/present.yml" when: ec2_platform_state == 'present' @@ -24,3 +22,34 @@ ansible.builtin.include_tasks: "{{ role_path }}/tasks/absent.yml" when: ec2_platform_state == 'absent' +- name: Display platform configuration + ansible.builtin.debug: + var: ec2_platform_instance_configs + verbosity: 1 + +- name: Export platform configuration + ansible.builtin.set_fact: + platform_exported_config: >- + {%- set __platform_instances = {} -%} + {%- for __platform in __platform_config -%} + {%- if ec2_platform_instance_configs is defined -%} + {%- set __platform_instance_config = ec2_platform_instance_configs | selectattr('instance', 'equalto', __platform.name) | first -%} + {%- set __platform_hostvars_merged = { + 'ansible_host': __platform_instance_config.address, + 'ansible_port': __platform_instance_config.port, + 'ansible_user': __platform_instance_config.user, + 'ansible_ssh_private_key_file': __platform_instance_config.identity_file, + 'ec2_instance_id': __platform_instance_config.instance_id, + } -%} + {%- endif -%} + {%- set _ = __platform_instances.update({ + __platform.name: { + 'hostvars': __platform.hostvars | default({}) + | combine(ec2_platform_hostvars_base) + | combine(__platform_hostvars_merged | default({})), + 'instance_config': __platform_instance_config | default({}) + } + }) -%} + {%- endfor -%} + {{ __platform_instances }} + diff --git a/roles/ec2_platform/tasks/present.yml b/roles/ec2_platform/tasks/present.yml index f533b0d..2c23d48 100644 --- a/roles/ec2_platform/tasks/present.yml +++ b/roles/ec2_platform/tasks/present.yml @@ -1,176 +1,261 @@ --- -# The ec2 platform has been created +# Deploy desired EC2 platform(s) +# +# Expected input: +# - ec2_platform: The current Molecule platform configurations (list) +# -- name: Validate platform configuration - {{ ec2_platform.name | default('invalid') }} - ansible.builtin.assert: - that: - - ec2_platform is mapping - - ec2_platform.name is string and ec2_platform.name | length > 0 - - ec2_platform.assign_public_ip is boolean - - ec2_platform.aws_profile is string - - ec2_platform.boot_wait_seconds is integer and ec2_platform.boot_wait_seconds >= 0 - - ec2_platform.cloud_config is mapping - - ec2_platform.image is string - - ec2_platform.image_name is string - - ec2_platform.image_owner is sequence or (ec2_platform.image_owner is string and ec2_platform.image_owner | length > 0) - - ec2_platform.instance_type is string and ec2_platform.instance_type | length > 0 - - ec2_platform.key_inject_method is in ["cloud-init", "ec2"] - - ec2_platform.key_name is string and ec2_platform.key_name | length > 0 - - ec2_platform.private_key_path is string and ec2_platform.private_key_path | length > 0 - - ec2_platform.public_key_path is string and ec2_platform.public_key_path | length > 0 - - ec2_platform.region is string - - ec2_platform.security_group_name is string and ec2_platform.security_group_name | length > 0 - - ec2_platform.security_group_description is string and ec2_platform.security_group_description | length > 0 - - ec2_platform.security_group_rules is sequence - - ec2_platform.security_group_rules_egress is sequence - - ec2_platform.security_groups is sequence - - ec2_platform.ssh_user is string and ec2_platform.ssh_user | length > 0 - - ec2_platform.ssh_port is integer and ec2_platform.ssh_port in range(1, 65536) - - ec2_platform.tags is mapping - - ec2_platform.volumes is sequence - - ec2_platform.vpc_id is string - - ec2_platform.vpc_subnet_id is string and ec2_platform.vpc_subnet_id | length > 0 - quiet: true - -# TODO: Merge, not overwrite -- already does? -- name: Write run config to file +- name: EC2 Provision | Write run config to file ansible.builtin.copy: dest: "{{ ec2_platform_run_config_path }}" content: "{{ ec2_platform_run_config | to_yaml }}" mode: "0600" -- name: Generate local key pairs +- name: EC2 Provision | Generate local key pairs + loop: "{{ ec2_platforms }}" + loop_control: + loop_var: __ec2_platform + label: "{{ __ec2_platform.name }}" community.crypto.openssh_keypair: - path: "{{ ec2_platform.private_key_path }}" + path: "{{ __ec2_platform.private_key_path }}" type: rsa size: 2048 regenerate: never - register: __ec2_platform_local_keypair + register: __ec2_platform_local_keypairs -- name: Look up EC2 AMI(s) by owner and name (if image not set) - amazon.aws.ec2_ami_info: - owners: "{{ ec2_platform.image_owner }}" - filters: "{{ ec2_platform.image_filters | default({}) | combine(__ec2_platform_image_name_map) }}" +- name: EC2 Provision | Look up EC2 AMI(s) by owner and name (if image not set) + loop: "{{ ec2_platforms | selectattr('image', 'undefined') }}" + loop_control: + loop_var: __ec2_platform + label: "{{ __ec2_platform.name }}" vars: - __ec2_platform_image_name_map: >- - "{% if ec2_platform.image_name is defined and ec2_platform.image_name | length > 0 %} - {{ {'name': ec2_platform.image_name} }} + ____ec2_platform_image_name_map: >- + "{% if __ec2_platform.image_name is defined and __ec2_platform.image_name | length > 0 %} + {{ {'name': __ec2_platform.image_name} }} {% else %}{}{% endif %}" - when: not ec2_platform.image + amazon.aws.ec2_ami_info: + owners: "{{ __ec2_platform.image_owner }}" + filters: "{{ __ec2_platform.image_filters | default({}) | combine(__ec2_platform_image_name_map) }}" register: __ec2_platform_ami_info -- name: Look up subnets to determine VPCs (if needed) +- name: EC2 Provision | Look up subnets to determine VPCs (if needed) + loop: "{{ ec2_platforms | selectattr('vpc_id', 'undefined') }}" + loop_control: + loop_var: __ec2_platform + label: "{{ __ec2_platform.name }}" amazon.aws.ec2_vpc_subnet_info: - subnet_ids: "{{ ec2_platform.vpc_subnet_id }}" - when: not ec2_platform.vpc_id + subnet_ids: "{{ __ec2_platform.vpc_subnet_id }}" register: __ec2_platform_subnet_info -- name: Validate discovered information +- name: EC2 Provision | Validate discovered information + loop: "{{ ec2_platforms }}" + loop_control: + loop_var: __ec2_platform + label: "{{ __ec2_platform.name }}" ansible.builtin.assert: that: - - ec2_platform.image or (__ec2_platform_ami_info.results[0].images | length > 0) - - ec2_platform.vpc_id or (__ec2_platform_subnet_info.results[0].subnets | length > 0) + - __ec2_platform.image or ((__ec2_platform_ami_info.results | selectattr('__ec2_platform.name', 'equalto', __ec2_platform.name)).images | length > 0) + - __ec2_platform.vpc_id or ((__ec2_platform_subnet_info.results | selectattr('__ec2_platform.name', 'equalto', __ec2_platform.name)).subnets | length > 0) quiet: true -- name: Create ephemeral EC2 keys (if needed) +- name: Show generated keypairs + loop: "{{ ec2_platforms }}" + loop_control: + loop_var: __ec2_platform + label: "{{ __ec2_platform.name }}" + ansible.builtin.debug: + var: (__ec2_platform_local_keypairs.results | selectattr('__ec2_platform.name', 'equalto', __ec2_platform.name) | first) + verbosity: 1 + +- name: EC2 Provision | Create ephemeral EC2 keys (if needed) + loop: "{{ ec2_platforms | selectattr('key_inject_method', 'equalto', 'ec2') }}" + loop_control: + loop_var: __ec2_platform + label: "{{ __ec2_platform.name }}" + vars: + __ec2_platform_ssh_pubkey: "{{ (__ec2_platform_local_keypairs.results | selectattr('__ec2_platform.name', 'equalto', __ec2_platform.name) | first).public_key }}" amazon.aws.ec2_key: - profile: "{{ ec2_platform.aws_profile | default(omit) }}" - region: "{{ ec2_platform.region | default(omit) }}" - name: "{{ ec2_platform.key_name }}" - key_material: "{{ __ec2_platform_local_keypair.public_key }}" - when: ec2_platform.key_inject_method == "ec2" + profile: "{{ __ec2_platform.aws_profile | default(omit) }}" + region: "{{ __ec2_platform.region | default(omit) }}" + name: "{{ __ec2_platform.key_name }}" + key_material: "{{ __ec2_platform_ssh_pubkey }}" register: __ec2_platform_ec2_keys -- name: Create ephemeral security groups (if needed) - amazon.aws.ec2_security_group: - profile: "{{ ec2_platform.aws_profile | default(omit) }}" - iam_instance_profile: "{{ ec2_platform.iam_instance_profile | default(omit) }}" - region: "{{ ec2_platform.region | default(omit) }}" - vpc_id: "{{ ec2_platform.vpc_id or vpc_subnet.vpc_id }}" - name: "{{ ec2_platform.security_group_name }}" - description: "{{ ec2_platform.security_group_description }}" - rules: "{{ ec2_platform.security_group_rules }}" - rules_egress: "{{ ec2_platform.security_group_rules_egress }}" - tags: - Name: "{{ ec2_platform.security_group_name }}" +- name: EC2 Provision | Create ephemeral security groups (if needed) + loop: "{{ ec2_platforms | selectattr('security_groups', 'falsy') }}" + loop_control: + loop_var: __ec2_platform + label: "{{ __ec2_platform.name }}" vars: vpc_subnet: "{{ __ec2_platform_subnet_info.results[0].subnets[0] }}" - when: ec2_platform.security_groups | length == 0 + amazon.aws.ec2_security_group: + profile: "{{ __ec2_platform.aws_profile | default(omit) }}" + iam_instance_profile: "{{ __ec2_platform.iam_instance_profile | default(omit) }}" + region: "{{ __ec2_platform.region | default(omit) }}" + vpc_id: "{{ __ec2_platform.vpc_id or vpc_subnet.vpc_id }}" + name: "{{ __ec2_platform.security_group_name }}" + description: "{{ __ec2_platform.security_group_description }}" + rules: "{{ __ec2_platform.security_group_rules }}" + rules_egress: "{{ __ec2_platform.security_group_rules_egress }}" + tags: + Name: "{{ __ec2_platform.security_group_name }}" + molecule-run-id: "{{ ec2_platform_run_config.run_id }}" -- name: Create ephemeral EC2 instance - amazon.aws.ec2_instance: - profile: "{{ ec2_platform.aws_profile | default(omit) }}" - region: "{{ ec2_platform.region | default(omit) }}" - filters: "{{ __ec2_platform_filters }}" - instance_type: "{{ ec2_platform.instance_type }}" - image_id: "{{ __ec2_platform_image_id }}" - vpc_subnet_id: "{{ ec2_platform.vpc_subnet_id }}" - security_groups: "{{ __ec2_platform_security_groups }}" - network: - assign_public_ip: "{{ ec2_platform.assign_public_ip }}" - volumes: "{{ ec2_platform.volumes }}" - key_name: "{{ (ec2_platform.key_inject_method == 'ec2') | ternary(ec2_platform.key_name, omit) }}" - tags: "{{ __ec2_platform_tags }}" - user_data: "{{ __ec2_platform_user_data }}" - state: "running" - wait: true +- name: EC2 Provision | Create ephemeral EC2 instance + loop: "{{ ec2_platforms }}" + loop_control: + loop_var: __ec2_platform + label: "{{ __ec2_platform.name }}" vars: - __ec2_platform_security_groups: "{{ ec2_platform.security_groups or [ec2_platform.security_group_name] }}" + __ec2_platform_security_groups: "{{ __ec2_platform.security_groups or [__ec2_platform.security_group_name] }}" __ec2_platform_generated_image_id: "{{ (ami_info.results[0].images | sort(attribute='creation_date', reverse=True))[0].image_id }}" - __ec2_platform_image_id: "{{ ec2_platform.image or __ec2_platform_generated_image_id }}" - + __ec2_platform_image_id: "{{ __ec2_platform.image or __ec2_platform_generated_image_id }}" + __ec2_platform_ssh_pubkey: "{{ (__ec2_platform_local_keypairs.results | selectattr('__ec2_platform.name', 'equalto', __ec2_platform.name) | first).public_key }}" __ec2_platform_generated_cloud_config: users: - - name: "{{ ec2_platform.ssh_user }}" + - name: "{{ __ec2_platform.ssh_user }}" ssh_authorized_keys: - - "{{ __ec2_platform_local_keypair.public_key }}" + - "{{ __ec2_platform_ssh_pubkey }}" sudo: "ALL=(ALL) NOPASSWD:ALL" __ec2_platform_cloud_config: >- - {{ (ec2_platform.key_inject_method == 'cloud-init') - | ternary((ec2_platform.cloud_config | combine(__ec2_platform_generated_cloud_config)), ec2_platform.cloud_config) }} + {{ (__ec2_platform.key_inject_method == 'cloud-init') + | ternary((__ec2_platform.cloud_config | combine(__ec2_platform_generated_cloud_config)), __ec2_platform.cloud_config) }} __ec2_platform_user_data: |- #cloud-config {{ __ec2_platform_cloud_config | to_yaml }} __ec2_platform_generated_tags: - instance: "{{ ec2_platform.name }}" + instance: "{{ __ec2_platform.name }}" "molecule-run-id": "{{ ec2_platform_run_config.run_id }}" - Name: molecule-{{ ec2_platform.name }} - __ec2_platform_tags: "{{ (ec2_platform.tags or {}) | combine(__ec2_platform_generated_tags) }}" + Name: molecule-{{ __ec2_platform.name }} + __ec2_platform_tags: "{{ (__ec2_platform.tags or {}) | combine(__ec2_platform_generated_tags) }}" __ec2_platform_filter_keys: "{{ __ec2_platform_generated_tags.keys() | map('regex_replace', '^(.+)$', 'tag:\\1') }}" __ec2_platform_filters: "{{ dict(__ec2_platform_filter_keys | zip(__ec2_platform_generated_tags.values())) }}" - register: __ec2_instance_creation + amazon.aws.ec2_instance: + profile: "{{ __ec2_platform.aws_profile | default(omit) }}" + region: "{{ __ec2_platform.region | default(omit) }}" + filters: "{{ __ec2_platform_filters }}" + instance_type: "{{ __ec2_platform.instance_type }}" + image_id: "{{ __ec2_platform_image_id }}" + vpc_subnet_id: "{{ __ec2_platform.vpc_subnet_id }}" + security_groups: "{{ __ec2_platform_security_groups }}" + network: + assign_public_ip: "{{ __ec2_platform.assign_public_ip }}" + volumes: "{{ __ec2_platform.volumes }}" + key_name: "{{ (__ec2_platform.key_inject_method == 'ec2') | ternary(__ec2_platform.key_name, omit) }}" + tags: "{{ __ec2_platform_tags }}" + user_data: "{{ __ec2_platform_user_data }}" + state: "running" + wait: true + register: __ec2_instance_creation_jobs + async: "{{ __ec2_platform.instance_creation_timeout | int }}" + poll: 0 + changed_when: false # Asynchronous task, always marked as changed +- name: EC2 Provision | Print instance creation job status + ansible.builtin.debug: + var: __ec2_instance_creation_jobs + verbosity: 1 -# NOTE: Var is used by the `platform` role to write Molecule instance configuration -- name: Collect instance configs +- name: EC2 Provision | Instance creation is complete + loop: "{{ __ec2_instance_creation_jobs.results }}" + loop_control: + loop_var: __ec2_instance_creation_job + label: "{{ __ec2_instance_creation_job.__ec2_platform.name }}" + ansible.builtin.async_status: + jid: "{{ __ec2_instance_creation_job.ansible_job_id }}" + register: __ec2_instance_creation_results + until: __ec2_instance_creation_results.finished + retries: "{{ (__ec2_instance_creation_job.__ec2_platform.instance_creation_timeout | int / 10) | int }}" + delay: 10 + # TODO: Update 'changed_when' to reflect the actual state of the task + changed_when: false + +- name: EC2 Provision | Print instance creation result + ansible.builtin.debug: + var: __ec2_instance_creation_results + verbosity: 1 + +# Referenced in exported platform configuration -- see main.yml +- name: EC2 Provision | Collect instance configs + loop: "{{ __ec2_instance_creation_results.results }}" + loop_control: + loop_var: __ec2_instance_record + label: "{{ __ec2_instance_record.__ec2_instance_creation_job.__ec2_platform.name }}" vars: - __ec2_platform_instance: "{{ __ec2_instance_creation.instances[0] }}" + __ec2_platform_instance_definition: "{{ ec2_platforms + | selectattr('name', 'equalto', __ec2_instance_record.__ec2_instance_creation_job.__ec2_platform.name) | first }}" + __ec2_platform_instance_data: "{{ __ec2_instance_record.instances + | selectattr('tags.instance', 'equalto', __ec2_instance_record.__ec2_instance_creation_job.__ec2_platform.name) | first }}" + __ec2_platform_instance: + instance: "{{ __ec2_platform_instance_definition.name }}" + address: "{{ __ec2_platform_instance_definition.assign_public_ip + | ternary(__ec2_platform_instance_data.public_ip_address, __ec2_platform_instance_data.private_ip_address) }}" + user: "{{ __ec2_platform_instance_definition.ssh_user }}" + port: "{{ __ec2_platform_instance_definition.ssh_port }}" + identity_file: "{{ __ec2_platform_instance_definition.private_key_path }}" + instance_id: "{{ __ec2_platform_instance_data.instance_id }}" ansible.builtin.set_fact: - ec2_platform_instance_config: - instance: "{{ ec2_platform.name }}" - address: "{{ ec2_platform.assign_public_ip | ternary(__ec2_platform_instance.public_ip_address, __ec2_platform_instance.private_ip_address) }}" - user: "{{ ec2_platform.ssh_user }}" - port: "{{ ec2_platform.ssh_port }}" - identity_file: "{{ ec2_platform.private_key_path }}" - instance_ids: - - "{{ __ec2_platform_instance.instance_id }}" - -- name: Wait for SSH connectivity + ec2_platform_instance_configs: "{{ ec2_platform_instance_configs | default([]) + [__ec2_platform_instance] }}" + +- name: EC2 Provision | Launch boot wait tasks + loop: "{{ ec2_platforms }}" + loop_control: + loop_var: __ec2_platform + label: "{{ __ec2_platform.name }}" + ansible.builtin.wait_for: + timeout: "{{ __ec2_platform.boot_wait_seconds }}" + register: __ec2_platform_boot_wait_job + async: "{{ __ec2_platform.boot_wait_seconds | int + 10 }}" # Add 10 seconds to the timeout to allow for the wait_for task to complete + poll: 0 + changed_when: false + +- name: EC2 Provision | Wait for boot process to finish + loop: "{{ __ec2_platform_boot_wait_job.results }}" + loop_control: + loop_var: __ec2_platform_boot_wait_result + label: "{{ __ec2_platform_boot_wait_result.__ec2_platform.name }}" + ansible.builtin.async_status: + jid: "{{ __ec2_platform_boot_wait_result.ansible_job_id }}" + register: __ec2_platform_boot_wait_results + until: __ec2_platform_boot_wait_results.finished + retries: "{{ (__ec2_platform_boot_wait_result.__ec2_platform.boot_wait_seconds | int / 10) | int }}" + delay: 10 + changed_when: false + +- name: EC2 Provision | Launch SSH connectivity check tasks + loop: "{{ ec2_platforms }}" + loop_control: + loop_var: __ec2_platform + label: "{{ __ec2_platform.name }}" + vars: + __ec2_platform_instance_config: "{{ ec2_platform_instance_configs + | selectattr('instance', 'equalto', __ec2_platform.name) | first }}" ansible.builtin.wait_for: - host: "{{ ec2_platform_instance_config.address }}" - port: "{{ ec2_platform_instance_config.port }}" + host: "{{ __ec2_platform_instance_config.address }}" + port: "{{ __ec2_platform_instance_config.port }}" search_regex: SSH delay: 10 timeout: 320 - msg: "SSH connectivity check to {{ ec2_platform_instance_config.address }} on port {{ ec2_platform_instance_config.port }} failed" + msg: "SSH connectivity check to {{ __ec2_platform_instance_config.address }} on port {{ __ec2_platform_instance_config.port }} failed" register: __ec2_platform_ssh_connectivity_check until: "'Connection reset by peer' not in __ec2_platform_ssh_connectivity_check.module_stderr | default('')" retries: 6 delay: 10 + async: 330 + poll: 0 + changed_when: false -# TODO: Add an actual check here instead of only waiting -- name: Wait for boot process to finish - ansible.builtin.pause: - seconds: "{{ ec2_platform.boot_wait_seconds }}" +- name: EC2 Provision | SSH connectivity is successful + loop: "{{ __ec2_platform_ssh_connectivity_check.results }}" + loop_control: + loop_var: __ec2_platform_ssh_connectivity_check_result + label: "{{ __ec2_platform_ssh_connectivity_check_result.__ec2_platform.name }}" + ansible.builtin.async_status: + jid: "{{ __ec2_platform_ssh_connectivity_check_result.ansible_job_id }}" + register: __ec2_platform_ssh_connectivity_check_results + until: __ec2_platform_ssh_connectivity_check_results.finished + retries: 60 + delay: 10 + changed_when: false diff --git a/roles/ec2_platform/tasks/validate_cfg.yml b/roles/ec2_platform/tasks/validate_cfg.yml new file mode 100644 index 0000000..3125931 --- /dev/null +++ b/roles/ec2_platform/tasks/validate_cfg.yml @@ -0,0 +1,43 @@ +--- +# Validate that configuration provided to the role is valid +# +# Expected variables: +# - ec2_platform_definition: The current Molecule platform configurations (list) +# + +- name: Validate | EC2 platform configuration + loop: "{{ ec2_platforms }}" + loop_control: + loop_var: __ec2_platform_item + label: "{{ __ec2_platform_item.name | default('undefined') }}" + ansible.builtin.assert: + that: + - __ec2_platform_item is mapping + - __ec2_platform_item.name is string and __ec2_platform_item.name | length > 0 + - __ec2_platform_item.assign_public_ip is boolean + - __ec2_platform_item.aws_profile is string + - __ec2_platform_item.boot_wait_seconds is integer and __ec2_platform_item.boot_wait_seconds >= 0 + - __ec2_platform_item.cloud_config is mapping + - __ec2_platform_item.image is string + - __ec2_platform_item.image_name is string + - __ec2_platform_item.image_owner is sequence or (__ec2_platform_item.image_owner is string and __ec2_platform_item.image_owner | length > 0) + - __ec2_platform_item.instance_type is string and __ec2_platform_item.instance_type | length > 0 + - __ec2_platform_item.key_inject_method is in ["cloud-init", "ec2"] + - __ec2_platform_item.key_name is string and __ec2_platform_item.key_name | length > 0 + - __ec2_platform_item.private_key_path is string and __ec2_platform_item.private_key_path | length > 0 + - __ec2_platform_item.public_key_path is string and __ec2_platform_item.public_key_path | length > 0 + - __ec2_platform_item.region is string + - __ec2_platform_item.security_group_name is string and __ec2_platform_item.security_group_name | length > 0 + - __ec2_platform_item.security_group_description is string and __ec2_platform_item.security_group_description | length > 0 + - __ec2_platform_item.security_group_rules is sequence + - __ec2_platform_item.security_group_rules_egress is sequence + - __ec2_platform_item.security_groups is sequence + - __ec2_platform_item.ssh_user is string and __ec2_platform_item.ssh_user | length > 0 + - __ec2_platform_item.ssh_port is integer and __ec2_platform_item.ssh_port in range(1, 65536) + - __ec2_platform_item.tags is mapping + - __ec2_platform_item.volumes is sequence + - __ec2_platform_item.vpc_id is string + - __ec2_platform_item.vpc_subnet_id is string and __ec2_platform_item.vpc_subnet_id | length > 0 + fail_msg: "Invalid EC2 platform configuration provided!" + success_msg: "EC2 platform configuration validated." + diff --git a/roles/ec2_platform/vars/main.yml b/roles/ec2_platform/vars/main.yml index d156fac..33afc90 100644 --- a/roles/ec2_platform/vars/main.yml +++ b/roles/ec2_platform/vars/main.yml @@ -2,10 +2,7 @@ # vars file for ec2_platform # Base hostvars for EC2 instances that should always be present +# NOTE: These are exported on platform removal as well. Include default values if data will not always exist! ec2_platform_hostvars_base: ansible_connection: ssh - ansible_host: "{{ ec2_platform_instance_config.address }}" - ansible_port: "{{ ec2_platform_instance_config.port }}" - ansible_user: "{{ ec2_platform_instance_config.user }}" - ansible_ssh_private_key_file: "{{ ec2_platform_instance_config.identity_file }}" diff --git a/roles/platform/tasks/ansible_inventory.yml b/roles/platform/tasks/ansible_inventory.yml index 3edf087..ffa4ad1 100644 --- a/roles/platform/tasks/ansible_inventory.yml +++ b/roles/platform/tasks/ansible_inventory.yml @@ -67,7 +67,7 @@ var: __platform_ansible_existing_unreferenced_hosts verbosity: 1 -- name: Platform Molecule Inventory | Build inventory object +- name: Build and manage inventory file vars: __platform_ansible_hosts_present: >- {{ @@ -76,28 +76,38 @@ if platform_state == 'present' else __platform_ansible_existing_unreferenced_hosts }} - ansible.builtin.set_fact: - __platform_molecule_inventory: >- - {{ - platform_inventory_config_from_file - | combine({ - 'all': { - 'children': { - 'molecule': { - 'hosts': __platform_ansible_hosts_present + block: + - name: Platform Molecule Inventory | Build inventory object + ansible.builtin.set_fact: + __platform_molecule_inventory: >- + {{ + platform_inventory_config_from_file + | combine({ + 'all': { + 'children': { + 'molecule': { + 'hosts': __platform_ansible_hosts_present + } + } } - } - } - }) - }} + }) + }} + - name: Platform Molecule Inventory | Write inventory file + when: + - __platform_ansible_hosts_present is truthy + ansible.builtin.copy: + content: | + {{ __platform_molecule_inventory | to_nice_yaml(indent=2) }} + dest: "{{ platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" + mode: '0644' -- name: Platform Molecule Inventory | Write inventory file - ansible.builtin.copy: - content: | - {{ __platform_molecule_inventory | to_nice_yaml(indent=2) }} - dest: "{{ platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" - mode: '0644' + - name: Platform Molecule Inventory | Remove inventory file + when: + - __platform_ansible_hosts_present is not truthy + ansible.builtin.file: + path: "{{ platform_molecule_ephemeral_directory }}/inventory/molecule_inventory.yml" + state: absent - name: Platform Molecule Inventory | Force inventory refresh ansible.builtin.meta: refresh_inventory diff --git a/roles/platform/tasks/instance_config.yml b/roles/platform/tasks/instance_config.yml index 7f7eca1..2a2ec75 100644 --- a/roles/platform/tasks/instance_config.yml +++ b/roles/platform/tasks/instance_config.yml @@ -49,15 +49,25 @@ __platform_merged_instance_config: >- {{ platform_instance_config_from_file - | rejectattr('instance', 'in', __platform_old_instance_configs | map(attribute='instance') | list) + | rejectattr('instance', 'in', platform_exported_config | selectattr('instance_config', 'defined') | map(attribute='instance_config') | list) | list }} - name: Platform Instance Config | Write new instance configuration - when: __platform_merged_instance_config is defined + when: + - __platform_merged_instance_config is defined + - __platform_merged_instance_config | length > 0 ansible.builtin.copy: content: | {{ __platform_merged_instance_config | to_nice_yaml(indent=2) }} dest: "{{ platform_molecule_ephemeral_directory }}/instance_config.yml" mode: "0644" +- name: Platform Instance Config | Remove instance configuration file + when: + - __platform_merged_instance_config is not defined + - __platform_merged_instance_config | length == 0 + ansible.builtin.file: + path: "{{ platform_molecule_ephemeral_directory }}/instance_config.yml" + state: absent + diff --git a/roles/platform/tasks/validate_cfg.yml b/roles/platform/tasks/validate_cfg.yml index a0b4b83..466f183 100644 --- a/roles/platform/tasks/validate_cfg.yml +++ b/roles/platform/tasks/validate_cfg.yml @@ -5,7 +5,7 @@ # - __platform_config: The current list of platform configurations # -- name: Show platform config +- name: Validate | Show platform config ansible.builtin.debug: var: __platform_config verbosity: 1 @@ -18,16 +18,6 @@ fail_msg: "No platform configuration provided, or platform configuration invalid!" success_msg: "Platform configuration provided." -- name: Show first item - ansible.builtin.debug: - var: __platform_config[0] - verbosity: 1 - -- name: Show first item name - ansible.builtin.debug: - var: __platform_config[0].name - verbosity: 1 - - name: Validate | Unique attributes ansible.builtin.assert: that: From 891c5010d88732ac93ddabdf3d8bf189d5df378e Mon Sep 17 00:00:00 2001 From: syndr Date: Thu, 16 Jan 2025 16:31:26 -0700 Subject: [PATCH 16/18] EC2 instance config fixes, consolidate Molecule playbooks --- molecule/default/molecule.yml | 2 +- molecule/ec2_platform/converge.yml | 52 ------------ molecule/ec2_platform/molecule.yml | 15 +++- molecule/ec2_platform/prepare.yml | 83 -------------------- molecule/{default => resources}/converge.yml | 0 molecule/resources/prepare.yml | 5 ++ roles/ec2_platform/defaults/main.yml | 6 +- roles/ec2_platform/tasks/main.yml | 4 +- roles/ec2_platform/tasks/present.yml | 18 +---- roles/platform/tasks/deprovision.yml | 1 - roles/platform/tasks/provision.yml | 2 +- 11 files changed, 26 insertions(+), 162 deletions(-) delete mode 100644 molecule/ec2_platform/converge.yml delete mode 100644 molecule/ec2_platform/prepare.yml rename molecule/{default => resources}/converge.yml (100%) diff --git a/molecule/default/molecule.yml b/molecule/default/molecule.yml index 14a84fc..8b0da90 100644 --- a/molecule/default/molecule.yml +++ b/molecule/default/molecule.yml @@ -45,7 +45,7 @@ provisioner: playbooks: create: ../resources/create.yml prepare: ../resources/prepare.yml - converge: converge.yml + converge: ../resources/converge.yml side_effect: ../resources/side_effect.yml verify: ../resources/verify.yml cleanup: ../resources/cleanup.yml diff --git a/molecule/ec2_platform/converge.yml b/molecule/ec2_platform/converge.yml deleted file mode 100644 index f904f40..0000000 --- a/molecule/ec2_platform/converge.yml +++ /dev/null @@ -1,52 +0,0 @@ ---- -# Verify that the target code runs successfullly. -# Note that this playbook (converge.yml) must be idempotent! - -# Check that the molecule inventory is correctly configured -- name: Fail if molecule group is missing - hosts: localhost - tasks: - - name: Print host inventory groups - ansible.builtin.debug: - msg: "{{ groups }}" - - - name: Assert group existence - ansible.builtin.assert: - that: "'molecule' in groups" - fail_msg: | - molecule group was not found inside inventory groups: {{ groups }} - -- name: Converge - hosts: molecule - tasks: - - name: Check uname - ansible.builtin.raw: uname -a - register: result - changed_when: false - - - name: Verify kernel type - ansible.builtin.assert: - that: result.stdout | regex_search("^Linux") - - - name: Do preparation - block: - - name: Load local host facts - ansible.builtin.setup: - gather_subset: - - '!all' - - '!min' - - local - - - name: Show local Ansible facts - ansible.builtin.debug: - var: ansible_facts.ansible_local - verbosity: 1 - - - name: Load preparation facts - ansible.builtin.set_fact: - test_prepare_fact: "{{ ansible_local.molecule.test_prepare_fact }}" - - - name: Add your project test configuration here - ansible.builtin.debug: - msg: Typically this will be via the ansible.builtin.include_role module or via import_playbook - diff --git a/molecule/ec2_platform/molecule.yml b/molecule/ec2_platform/molecule.yml index 5be25a6..bb04e1a 100644 --- a/molecule/ec2_platform/molecule.yml +++ b/molecule/ec2_platform/molecule.yml @@ -14,6 +14,17 @@ platforms: vpc_id: vpc-0eb9fd1391f4207ec vpc_subnet_id: subnet-0aa189c0d6fc53923 instance_type: t3.micro + ssh_user: molecule_runner + hostvars: + test_hostvar: test + - name: ec2-unbuntu2204 + type: ec2 + image: ami-0cb91c7de36eed2cb + region: us-east-2 + vpc_id: vpc-0eb9fd1391f4207ec + vpc_subnet_id: subnet-0aa189c0d6fc53923 + instance_type: t3.micro + ssh_user: molecule_runner hostvars: test_hostvar: test provisioner: @@ -21,8 +32,8 @@ provisioner: log: True playbooks: create: ../resources/create.yml - prepare: prepare.yml - converge: converge.yml + prepare: ../resources/prepare.yml + converge: ../resources/converge.yml side_effect: ../resourcs/side_effect.yml verify: ../resources/verify.yml cleanup: ../resources/cleanup.yml diff --git a/molecule/ec2_platform/prepare.yml b/molecule/ec2_platform/prepare.yml deleted file mode 100644 index a08e4df..0000000 --- a/molecule/ec2_platform/prepare.yml +++ /dev/null @@ -1,83 +0,0 @@ ---- - -- name: Prepare controller for execution - hosts: localhost - tags: always - tasks: - - name: Configure for standalone role testing - ansible.builtin.include_role: - name: syndr.molecule.prepare_controller - vars: - prepare_controller_project_type: collection - -- name: Prepare target host for execution - hosts: molecule - tags: always - become: true - tasks: - ## - # Creating an admin service account for Molecule/Ansible to use for testing - # - # - If you run Ansible as a service account (you should) on your hosts and - # not as root, it is wise to also test as a non-root user! - # - # - To use this account, add the following to any plays targeting test - # infrastructure (such as in converge.yml): - # - # vars: - # ansible_user: molecule_runner - ## - - - name: Create ansible service account - vars: - molecule_user: molecule_runner - block: - - name: Create ansible group - ansible.builtin.group: - name: "{{ molecule_user }}" - - - name: Create ansible user - ansible.builtin.user: - name: "{{ molecule_user }}" - group: "{{ molecule_user }}" - - - name: Sudoers.d directory exists - ansible.builtin.file: - path: /etc/sudoers.d - state: directory - owner: root - group: root - mode: 0751 - - - name: Ansible user has sudo - ansible.builtin.copy: - content: | - {{ molecule_user }} ALL=(ALL) NOPASSWD: ALL - dest: /etc/sudoers.d/ansible - owner: root - group: root - mode: 0600 - - - name: "Save vars to host (IE: generated test credentials, etc.)" - become: true - block: - - name: Ansible facts directory exists - ansible.builtin.file: - path: /etc/ansible/facts.d - state: directory - owner: root - group: root - mode: 0755 - - - name: Persistent data saved to local Ansible facts - ansible.builtin.copy: - dest: /etc/ansible/facts.d/molecule.fact - content: "{{ {'test_prepare_fact': 'this is an example!'} | to_json }}" - owner: root - group: root - mode: 0644 - - - name: Add your host preparation tasks here! - ansible.builtin.debug: - msg: "IE: adding system users, installing required packages, etc." - diff --git a/molecule/default/converge.yml b/molecule/resources/converge.yml similarity index 100% rename from molecule/default/converge.yml rename to molecule/resources/converge.yml diff --git a/molecule/resources/prepare.yml b/molecule/resources/prepare.yml index f8ba73f..62fbeca 100644 --- a/molecule/resources/prepare.yml +++ b/molecule/resources/prepare.yml @@ -58,6 +58,7 @@ # ansible_user: molecule_runner ## - name: Create ansible service account + become: true block: - name: Create ansible group ansible.builtin.group: @@ -111,6 +112,7 @@ - name: Install packages # NOTE: If testing with a distro not mentioned in this block, you'll need to add code for it here! + become: true block: - name: Install packages for ubuntu block: @@ -130,6 +132,7 @@ pkg: - ansible - wget + - acl when: ansible_distribution | lower == 'ubuntu' - name: Install packages (RHEL Family) @@ -158,6 +161,7 @@ ansible.builtin.file: path: ~/.ansible/collections/ansible_collections state: directory + mode: "0755" - name: Project dir does not exist (Clean Slate 󰩸) ansible.builtin.file: @@ -168,6 +172,7 @@ ansible.builtin.file: path: ~/.ansible/collections/ansible_collections/{{ collection_namespace }}/{{ collection_name }} state: directory + mode: "0755" - name: Deploy project to host ansible.builtin.unarchive: diff --git a/roles/ec2_platform/defaults/main.yml b/roles/ec2_platform/defaults/main.yml index 0514063..94f5bf5 100644 --- a/roles/ec2_platform/defaults/main.yml +++ b/roles/ec2_platform/defaults/main.yml @@ -7,6 +7,9 @@ ec2_platform_name: instance # Whether this platform should be deployed (present/absent) ec2_platform_state: "{{ platform_target_state | default('present') }}" +# Molecule platform configuration (list of dictionaries) +ec2_platform_definition: "{{ [platform_target_config] if platform_target_config is defined else (molecule_yml.platforms | default([])) }}" + # Merge the defaults with any options provided to this role ec2_platforms: >- {%- set __platforms = [] -%} @@ -16,9 +19,6 @@ ec2_platforms: >- {%- endfor -%} {{ __platforms }} -# Molecule platform configuration (list of dictionaries) -ec2_platform_definition: "{{ [platform_target_config] if platform_target_config is defined else (molecule_yml.platforms | default([])) }}" - # Run config handling ec2_platform_default_run_id: "{{ ec2_platform_run_config_from_file.run_id | default(lookup('password', '/dev/null chars=ascii_lowercase length=5')) }}" ec2_platform_default_run_config: diff --git a/roles/ec2_platform/tasks/main.yml b/roles/ec2_platform/tasks/main.yml index 64f99eb..4b41c0c 100644 --- a/roles/ec2_platform/tasks/main.yml +++ b/roles/ec2_platform/tasks/main.yml @@ -6,9 +6,9 @@ var: ec2_platform_definition verbosity: 1 -- name: 🦋 Show ec2_platform +- name: 🦋 Show ec2_platforms ansible.builtin.debug: - var: ec2_platform + var: ec2_platforms verbosity: 1 - name: Validate configuration diff --git a/roles/ec2_platform/tasks/present.yml b/roles/ec2_platform/tasks/present.yml index 2c23d48..83b8aea 100644 --- a/roles/ec2_platform/tasks/present.yml +++ b/roles/ec2_platform/tasks/present.yml @@ -223,7 +223,7 @@ delay: 10 changed_when: false -- name: EC2 Provision | Launch SSH connectivity check tasks +- name: EC2 Provision | SSH connectivity check is successful loop: "{{ ec2_platforms }}" loop_control: loop_var: __ec2_platform @@ -242,20 +242,4 @@ until: "'Connection reset by peer' not in __ec2_platform_ssh_connectivity_check.module_stderr | default('')" retries: 6 delay: 10 - async: 330 - poll: 0 - changed_when: false - -- name: EC2 Provision | SSH connectivity is successful - loop: "{{ __ec2_platform_ssh_connectivity_check.results }}" - loop_control: - loop_var: __ec2_platform_ssh_connectivity_check_result - label: "{{ __ec2_platform_ssh_connectivity_check_result.__ec2_platform.name }}" - ansible.builtin.async_status: - jid: "{{ __ec2_platform_ssh_connectivity_check_result.ansible_job_id }}" - register: __ec2_platform_ssh_connectivity_check_results - until: __ec2_platform_ssh_connectivity_check_results.finished - retries: 60 - delay: 10 - changed_when: false diff --git a/roles/platform/tasks/deprovision.yml b/roles/platform/tasks/deprovision.yml index 6cfb308..af7b087 100644 --- a/roles/platform/tasks/deprovision.yml +++ b/roles/platform/tasks/deprovision.yml @@ -19,7 +19,6 @@ - name: Remove {{ __platform_config | length }} Molecule platform(s) vars: __platform_type: "{{ __platform_config | map(attribute='type') | first }}" - platform_target_config: "{{ __platform_config }}" platform_target_state: absent ansible.builtin.include_role: name: "{{ ansible_collection_name }}.{{ __platform_type }}_platform" diff --git a/roles/platform/tasks/provision.yml b/roles/platform/tasks/provision.yml index ea7df9c..10223be 100644 --- a/roles/platform/tasks/provision.yml +++ b/roles/platform/tasks/provision.yml @@ -18,8 +18,8 @@ - name: Configure {{ __platform_config | length }} Molecule platform(s) vars: + # WARNING: This assumes all platforms are of the same type __platform_type: "{{ __platform_config | map(attribute='type') | first }}" - platform_target_config: "{{ __platform_config }}" ansible.builtin.include_role: name: "{{ ansible_collection_name }}.{{ __platform_type }}_platform" From 8b4c5446675560d3f6146b368d2ed5485e784aaf Mon Sep 17 00:00:00 2001 From: syndr Date: Tue, 21 Jan 2025 19:00:30 -0700 Subject: [PATCH 17/18] Update init role file layout Deploy files into 'resources' directory --- roles/init/defaults/main.yml | 53 +++---- roles/init/defaults/platforms/docker.yml | 10 ++ roles/init/defaults/platforms/ec2.yml | 11 ++ roles/init/tasks/asserts.yml | 47 ++++-- roles/init/tasks/main.yml | 135 +++++++----------- roles/init/tasks/meta.yml | 53 +++++++ .../cleanup.yml => templates/cleanup.yml.j2} | 0 .../converge.yml.j2} | 2 + roles/init/templates/molecule.yml.j2 | 64 +++++---- .../init/templates/platform_cfg/docker.yml.j2 | 9 ++ .../side_effect.yml.j2} | 0 .../verify.yml => templates/verify.yml.j2} | 0 roles/init/vars/main.yml | 3 + 13 files changed, 227 insertions(+), 160 deletions(-) create mode 100644 roles/init/defaults/platforms/docker.yml create mode 100644 roles/init/defaults/platforms/ec2.yml create mode 100644 roles/init/tasks/meta.yml rename roles/init/{files/cleanup.yml => templates/cleanup.yml.j2} (100%) rename roles/init/{files/converge.yml => templates/converge.yml.j2} (98%) create mode 100644 roles/init/templates/platform_cfg/docker.yml.j2 rename roles/init/{files/side_effect.yml => templates/side_effect.yml.j2} (100%) rename roles/init/{files/verify.yml => templates/verify.yml.j2} (100%) diff --git a/roles/init/defaults/main.yml b/roles/init/defaults/main.yml index d561e71..d599215 100644 --- a/roles/init/defaults/main.yml +++ b/roles/init/defaults/main.yml @@ -1,7 +1,7 @@ --- # defaults file for init -# The type of project that this Molecule configuration will be integrated into +# The type of project that this Molecule configuration will be integrated into (role, collection, playbook, monolith) init_project_type: auto # The type of platform that this Molecule configuration will be testing on (docker, ec2) @@ -15,12 +15,10 @@ init_collection_version: latest # Source of the collection that this role is part of (galaxy, git) init_collection_source: git -# Filesystem location of the molecule scenario being initialized -init_scenario_dir: "{{ molecule_scenario_directory | default(playbook_dir) }}" - -# The filesystem location of the project being tested by this Molecule configuration -# - default value assumes that your Molecule project is located at /molecule/ -init_project_dir: "{{ init_scenario_dir.split('/')[:-2] | join('/') }}" +# Path to the ansible secret file that should be used by the Molecule test +# - Variable substitution can be used as described here: https://ansible.readthedocs.io/projects/molecule/configuration/#variable-substitution +# - Set to "" to disable +init_ansible_secret_path: "{{ lookup('env', 'ANSIBLE_VAULT_PASSWORD_FILE') | default('') }}" # Platforms that this test configuration should test # list of dicts, each required to contain: @@ -42,33 +40,22 @@ init_platforms: [] # Create backups of any files that would be clobbered by running this role init_file_backup: true +# Initialize ARA support in the Molecule configuration -- https://ara.recordsansible.org/ +init_ara_support: true + # Default configuration to be added to generated molecule.yml if init_platforms is not defined -init_platform_defaults: - docker: - - name: docker-rockylinux9 - type: docker - config: - image: "geerlingguy/docker-rockylinux9-ansible:latest" - systemd: true - ec2: - - name: ec2-rockylinux9 - type: ec2 - config: - image: "ami-01bd836275f79352c" - instance_type: "t3.micro" - region: "us-east-2" - vpc_id: "vpc-12345678" - vpc_subnet_id: "subnet-12345678" +init_platform_defaults: "{{ lookup('file', 'defaults/platforms/' + init_platform_type + '.yml') | from_yaml }}" -# Path to the ansible secret file that should be used by the Molecule test -# - Variable substitution can be used as described here: https://ansible.readthedocs.io/projects/molecule/configuration/#variable-substitution -# - Set to "" to disable -init_ansible_secret_path: "" +# Filesystem location of the molecule scenario being initialized +init_scenario_dir: "{{ molecule_scenario_directory | default(playbook_dir) }}" -# Configuration defaults to be used if the collection manifest is not accessible -init_collection_defaults: - repository: https://github.com/syndr/ansible-collection-molecule - name: molecule - namespace: syndr - version: latest +# The filesystem location of the project being tested by this Molecule configuration +# - default value assumes that your Molecule project is located at /molecule/ +init_project_dir: >- + {{ + init_scenario_dir.split('/')[:-2] | join('/') + if init_scenario_dir.split('/')[:-1] | last == 'molecule' + else + init_scenario_dir.split('/')[:-3] | join('/') + }} diff --git a/roles/init/defaults/platforms/docker.yml b/roles/init/defaults/platforms/docker.yml new file mode 100644 index 0000000..3e643b8 --- /dev/null +++ b/roles/init/defaults/platforms/docker.yml @@ -0,0 +1,10 @@ +--- +# Default configuration for Docker platforms. + +name: docker-platform +type: docker +image: "geerlingguy/docker-rockylinux9-ansible:latest" +systemd: true +privileged: false +pull: always + diff --git a/roles/init/defaults/platforms/ec2.yml b/roles/init/defaults/platforms/ec2.yml new file mode 100644 index 0000000..b363140 --- /dev/null +++ b/roles/init/defaults/platforms/ec2.yml @@ -0,0 +1,11 @@ +--- +# Default configuration for ec2 platforms + +name: ec2-platform +type: ec2 +image: "ami-01bd836275f79352c" +instance_type: "t3.micro" +region: "us-east-2" +vpc_id: "vpc-12345678" +vpc_subnet_id: "subnet-12345678" + diff --git a/roles/init/tasks/asserts.yml b/roles/init/tasks/asserts.yml index b211f78..6a825bc 100644 --- a/roles/init/tasks/asserts.yml +++ b/roles/init/tasks/asserts.yml @@ -15,25 +15,33 @@ success_msg: Sanity check passed - name: Check for project file paths - ansible.builtin.stat: - path: "{{ __init_item }}" loop: - "{{ init_scenario_dir }}" - "{{ init_project_dir }}" loop_control: loop_var: __init_item + ansible.builtin.stat: + path: "{{ __init_item }}" register: __init_filepath_stat - name: File path exists + loop: "{{ __init_filepath_stat.results }}" + loop_control: + label: "{{ __init_item.stat.path }}" + loop_var: __init_item ansible.builtin.assert: that: - __init_item.stat.exists is true fail_msg: Specified file path does not exist! quiet: true - loop: "{{ __init_filepath_stat.results }}" - loop_control: - label: "{{ __init_item.stat.path }}" - loop_var: __init_item + +- name: Validate Molecule directory layout + ansible.builtin.assert: + that: + - init_scenario_dir.split('/')[:-1] | last == 'molecule' or + init_scenario_dir.split('/')[:-2] | last == 'molecule' + fail_msg: Molecule directory layout is not sane! Molecule test configuration must be in a directory named 'molecule'. + success_msg: Sanity check passed - name: Check for secret if specified block: @@ -50,16 +58,33 @@ when: init_ansible_secret_path is truthy - name: Platform configuration is sane + loop: "{{ init_platforms }}" + loop_control: + label: "{{ __init_item.name }}" + loop_var: __init_item ansible.builtin.assert: that: - __init_item.name is string - __init_item.name is truthy - __init_item.type in __init_supported_platform_types - - __init_item.config is mapping fail_msg: Platform configuration is not sane! success_msg: Sanity check passed - loop: "{{ init_platforms }}" - loop_control: - label: "{{ __init_item.name }}" - loop_var: __init_item + +- name: Validate platform support + ansible.builtin.assert: + that: + - init_platform_defaults is defined + - init_platform_defaults | type_debug == 'dict' + fail_msg: Unable to find platform defaults! Add support to the 'init' role for the platform '{{ init_platform_type }}'. + success_msg: Platform defaults found. + +- name: Validate runtime platform configuration + ansible.builtin.assert: + that: + - __init_runtime_platforms | type_debug == 'list' + - __init_runtime_platforms | map(attribute='name') | unique | length == __init_runtime_platforms | length + - __init_runtime_platforms | map(attribute='type') | unique | length == 1 + - __init_runtime_platforms | length > 0 + fail_msg: Runtime platform configuration is not sane! + success_msg: Sanity check passed. diff --git a/roles/init/tasks/main.yml b/roles/init/tasks/main.yml index f58e1d2..ee461b7 100644 --- a/roles/init/tasks/main.yml +++ b/roles/init/tasks/main.yml @@ -5,97 +5,60 @@ ansible.builtin.include_tasks: "{{ role_path }}/tasks/asserts.yml" - name: Autodetect repository type - ansible.builtin.include_tasks: "{{ role_path }}/tasks/auto.yml" when: init_project_type == 'auto' - -- name: Build base platform definition - when: init_platforms is not truthy - block: - - name: Build base docker platform definition - when: init_platform_type == 'docker' - ansible.builtin.set_fact: - init_platforms: "{{ init_platforms | default(init_platform_defaults.docker, true) }}" - - - name: Build base ec2 platform definition - when: init_platform_type == 'ec2' - ansible.builtin.set_fact: - init_platforms: "{{ init_platforms | default(init_platform_defaults.ec2, true) }}" - - - name: Platform definition is valid - ansible.builtin.assert: - that: - - init_platforms is defined - - init_platforms | length > 0 - - init_platforms[0].name is string - - init_platforms[0].type is string - - init_platforms[0].config is mapping - fail_msg: "Platform definition failed! Check the platform configuration." - success_msg: "Platform definition is valid" + ansible.builtin.include_tasks: "{{ role_path }}/tasks/auto.yml" - name: Load collection meta information - block: - - name: Load collection meta data - ansible.builtin.slurp: - src: "{{ role_path }}/../../MANIFEST.json" - register: __init_collection_meta - ignore_errors: true - - - name: 🐜 Show collection meta data - ansible.builtin.debug: - var: __init_collection_meta.content | b64decode | from_json - verbosity: 1 - ignore_errors: true + ansible.builtin.include_tasks: "{{ role_path }}/tasks/meta.yml" - - name: Extract collection meta info - when: - - __init_collection_meta is not failed - - __init_collection_meta.content is defined - - (__init_collection_meta.content | b64decode | from_json).collection_info.version is defined - ansible.builtin.set_fact: - __init_collection_meta: "{{ (__init_collection_meta.content | b64decode | from_json) }}" - -- name: Set collection information - ansible.builtin.set_fact: - __init_collection_version: >- - {{ init_collection_version if init_collection_version is truthy - else __init_collection_meta.collection_info.version | default(init_collection_defaults.version) }} - __init_collection_name: "{{ __init_collection_meta.collection_info.name | default(init_collection_defaults.name) }}" - __init_collection_namespace: "{{ __init_collection_meta.collection_info.namespace | default(init_collection_defaults.namespace) }}" - __init_collection_repository: "{{ __init_collection_meta.collection_info.repository | default(init_collection_defaults.repository) }}" +- name: Deploy Molecule configuration files + vars: + __init_scenario_dir: "{{ init_scenario_dir if init_scenario_dir.split('/')[-2] == 'molecule' else init_scenario_dir + '/default' }}" + __init_resources_dir: "{{ __init_scenario_dir.split('/')[:-2] | join('/') }}/resources" + __init_scenario_name: "{{ init_scenario_dir | basename }}" + block: + - name: Create molecule scenario directory + # Assume that we're inside a 'molecule' directory (checked via asserts.yml) + ansible.builtin.file: + path: "{{ __init_scenario_dir }}" + state: directory + mode: "0755" -- name: Deploy molecule configuration - ansible.builtin.template: - src: "{{ role_path }}/templates/molecule.yml.j2" - dest: "{{ init_scenario_dir }}/molecule.yml" - mode: 0644 - backup: "{{ init_file_backup }}" + - name: Deploy molecule configuration + ansible.builtin.template: + src: "{{ role_path }}/templates/molecule.yml.j2" + dest: "{{ __init_scenario_dir }}/molecule.yml" + mode: 0644 + backup: "{{ init_file_backup }}" -- name: Deploy template - ansible.builtin.template: - src: "{{ role_path }}/templates/{{ __init_item }}.j2" - dest: "{{ init_scenario_dir }}/{{ __init_item }}" - mode: 0644 - backup: "{{ init_file_backup }}" - loop: - - collections.yml - - requirements.yml - - create.yml - - prepare.yml - - destroy.yml - loop_control: - loop_var: __init_item + - name: Deploy resource template + loop: + - collections.yml + - requirements.yml + - create.yml + - prepare.yml + - destroy.yml + - converge.yml + - side_effect.yml + - verify.yml + - cleanup.yml + loop_control: + loop_var: __init_item + ansible.builtin.template: + src: "{{ role_path }}/templates/{{ __init_item }}.j2" + dest: "{{ __init_resources_dir }}/{{ __init_item }}" + mode: 0644 + backup: "{{ init_file_backup }}" -- name: Deploy file - ansible.builtin.copy: - src: "{{ role_path }}/files/{{ __init_item }}" - dest: "{{ init_scenario_dir }}/{{ __init_item }}" - mode: 0644 - backup: "{{ init_file_backup }}" - loop: - - converge.yml - - side_effect.yml - - verify.yml - - cleanup.yml - loop_control: - loop_var: __init_item + - name: Link Ansible dependency + loop: + - requirements.yml + - collections.yml + loop_control: + loop_var: __init_item + ansible.builtin.file: + src: "{{ __init_resources_dir }}/{{ __init_item }}" + dest: "{{ __init_scenario_dir }}/{{ __init_item }}" + state: link + force: true diff --git a/roles/init/tasks/meta.yml b/roles/init/tasks/meta.yml new file mode 100644 index 0000000..2f3b41b --- /dev/null +++ b/roles/init/tasks/meta.yml @@ -0,0 +1,53 @@ +--- +# Load collection meta data from manifest for this collection + +- name: Load collection meta data from manifest + ansible.builtin.slurp: + src: "{{ role_path }}/../../MANIFEST.json" + register: __init_collection_manifest_meta_b64 + ignore_errors: true + +- name: Load collection meta data from galaxy config + when: __init_collection_manifest_meta_b64 is failed + ansible.builtin.slurp: + src: "{{ role_path }}/../../galaxy.yml" + register: __init_collection_galaxy_meta_b64 + ignore_errors: true + +- name: 🐜 Show collection manifest meta data + ansible.builtin.debug: + var: __init_collection_manifest_meta_b64.content | b64decode | from_json + verbosity: 1 + +- name: 🐜 Show collection galaxy meta data + ansible.builtin.debug: + var: __init_collection_galaxy_meta_b64.content | b64decode | from_yaml + verbosity: 1 + +- name: Extract collection meta info + vars: + __init_collection_manifest_meta: "{{ (__init_collection_manifest_meta_b64.content | b64decode | from_json).collection_info | default({}) }}" + __init_collection_galaxy_meta: "{{ (__init_collection_galaxy_meta_b64.content | b64decode | from_yaml) | default({}) }}" + ansible.builtin.set_fact: + __init_collection_meta: "{{ __init_collection_manifest_meta | default(__init_collection_galaxy_meta, true) }}" + +- name: Validate meta info + ansible.builtin.assert: + that: + - __init_collection_meta is truthy + - __init_collection_meta is mappint + - __init_collection_meta.name is defined + - __init_collection_meta.version is defined + - __init_collection_meta.namespace is defined + - __init_collection_meta.repository is defined + fail_msg: "Collection meta information is missing or incomplete. Validate the collection manifest or galaxy.yml file." + success_msg: "Collection meta information is valid." + +- name: Set collection information + ansible.builtin.set_fact: + __init_collection_version: >- + {{ init_collection_version if init_collection_version is truthy + else __init_collection_meta.collection_info.version | default(init_collection_defaults.version) }} + __init_collection_name: "{{ __init_collection_meta.collection_info.name | default(init_collection_defaults.name) }}" + __init_collection_namespace: "{{ __init_collection_meta.collection_info.namespace | default(init_collection_defaults.namespace) }}" + __init_collection_repository: "{{ __init_collection_meta.collection_info.repository | default(init_collection_defaults.repository) }}" diff --git a/roles/init/files/cleanup.yml b/roles/init/templates/cleanup.yml.j2 similarity index 100% rename from roles/init/files/cleanup.yml rename to roles/init/templates/cleanup.yml.j2 diff --git a/roles/init/files/converge.yml b/roles/init/templates/converge.yml.j2 similarity index 98% rename from roles/init/files/converge.yml rename to roles/init/templates/converge.yml.j2 index f904f40..882174c 100644 --- a/roles/init/files/converge.yml +++ b/roles/init/templates/converge.yml.j2 @@ -3,6 +3,7 @@ # Note that this playbook (converge.yml) must be idempotent! # Check that the molecule inventory is correctly configured +{% raw -%} - name: Fail if molecule group is missing hosts: localhost tasks: @@ -50,3 +51,4 @@ ansible.builtin.debug: msg: Typically this will be via the ansible.builtin.include_role module or via import_playbook +{% endraw %} diff --git a/roles/init/templates/molecule.yml.j2 b/roles/init/templates/molecule.yml.j2 index 5367117..31c85d4 100644 --- a/roles/init/templates/molecule.yml.j2 +++ b/roles/init/templates/molecule.yml.j2 @@ -6,42 +6,32 @@ driver: name: default options: managed: true -{% if 'docker' in init_platforms | map(attribute='type') %} - login_cmd_template: 'docker exec -ti {instance} bash' +{% if 'docker' in init_platforms | map(attribute='type') %} + login_cmd_template: 'docker exec -ti {instance} bash --login' {% endif %} platforms: {% for init_platform in init_platforms %} - - name: {{ init_platform.name }} + - name: {{ init_platform.name | default('docker-platform') }} type: {{ init_platform.type }} -{% if init_platform.type == 'docker' %} - image: {{ init_platform.config.image }} - systemd: {{ init_platform.config.systemd | default(false) }} - modify_image: {{ init_platform.config.modify_image | default(false) }} -{% if init_platform.config.modify_image_buildpath is defined %} - modify_image_buildpath: {{ init_platform.config.modify_image_buildpath }} -{% endif %} - privileged: {{ init_platform.config.privileged | default(false) }} - hostvars: {} -{% endif %} -{% if init_platform.type == 'ec2' %} - image: {{ init_platform.config.image }} - region: {{ init_platform.config.region }} - vpc_id: {{ init_platform.config.vpc_id }} - vpc_subnet_id: {{ init_platform.config.vpc_subnet_id }} -{% endif %} - hostvars: {} -{% endfor %} + # See platform documentation at {{ __init_collection_repository }}/tree/main/roles/{{ init_platform.type }}_platform/README.md + {%- for key, value in init_platform.items() -%} + {%- if key not in ['name', 'type', 'hostvars'] %} + {{ key }}: {{ value }} + {% endif -%} + {%- endfor %} + hostvars: {{ init_platform.hostvars | default({}) }} + {% endfor %} provisioner: name: ansible log: True playbooks: - create: create.yml - prepare: prepare.yml - converge: converge.yml - side_effect: side_effect.yml - verify: verify.yml - cleanup: cleanup.yml - destroy: destroy.yml + create: ../resources/create.yml + prepare: ../resources/prepare.yml + converge: ../resources/converge.yml + side_effect: ../resources/side_effect.yml + verify: ../resources/verify.yml + cleanup: ../resources/cleanup.yml + destroy: ../resources/destroy.yml config_options: defaults: gathering: explicit @@ -54,16 +44,30 @@ provisioner: env: ANSIBLE_ROLES_PATH: /usr/share/ansible/roles:/etc/ansible/roles:~/.ansible/roles:${PWD}/roles ANSIBLE_COLLECTIONS_PATH: /usr/share/ansible/collections:~/.ansible/collections:${PWD}/collections -{% endif %} -{% if init_project_type == 'playbook' %} +{% elif init_project_type == 'playbook' %} env: ANSIBLE_ROLES_PATH: /usr/share/ansible/roles:/etc/ansible/roles:~/.ansible/roles:${PWD}/roles:${PWD}/../roles:${PWD}/../../roles ANSIBLE_COLLECTIONS_PATH: /usr/share/ansible/collections:~/.ansible/collections:${PWD}/collections:${PWD}/../collections:${PWD}/../../collections +{% elif init_ara_support %} + env: +{% else %} + env: {} +{% endif %} +{% if init_ara_support %} + ARA_API_CLIENT: ${ARA_API_CLIENT:-'http'} + ARA_API_SERVER: ${ARA_API_SERVER:-'http://localhost:8000'} + ARA_DEFAULT_LABELS: ${ARA_DEFAULT_LABELS:-'testing,molecule'} + # To use Ara with molecule: + # export the ANSIBLE_CALLBACK_PLUGINS env var with the output of 'python3 -m ara.setup.callback_plugins' + ANSIBLE_CALLBACK_PLUGINS: ${ANSIBLE_CALLBACK_PLUGINS} {% endif %} scenario: create_sequence: - dependency - create + prepare_sequence: + - dependency + - create - prepare check_sequence: - dependency diff --git a/roles/init/templates/platform_cfg/docker.yml.j2 b/roles/init/templates/platform_cfg/docker.yml.j2 new file mode 100644 index 0000000..9c4205c --- /dev/null +++ b/roles/init/templates/platform_cfg/docker.yml.j2 @@ -0,0 +1,9 @@ +- name: {{ init_platform.name | default('docker-platform') }} + type: {{ init_platform.type }} + # See platform documentation at {{ __init_collection_repository }}/tree/main/roles/{{ init_platform.type }}_platform/README.md + {% for key, value in init_platform.items() %} + {% if key not in ['name', 'type', 'hostvars'] %} + {{ key }}: {{ value }} + {% endif %} + {% endfor %} + hostvars: {{ init_platform.hostvars | default({}) }} diff --git a/roles/init/files/side_effect.yml b/roles/init/templates/side_effect.yml.j2 similarity index 100% rename from roles/init/files/side_effect.yml rename to roles/init/templates/side_effect.yml.j2 diff --git a/roles/init/files/verify.yml b/roles/init/templates/verify.yml.j2 similarity index 100% rename from roles/init/files/verify.yml rename to roles/init/templates/verify.yml.j2 diff --git a/roles/init/vars/main.yml b/roles/init/vars/main.yml index 59e7c45..1a995af 100644 --- a/roles/init/vars/main.yml +++ b/roles/init/vars/main.yml @@ -15,3 +15,6 @@ __init_supported_platform_types: - docker - ec2 +# The platforms to be configured by this role +__init_runtime_platforms: "{{ init_platforms | combine(init_platforms_defaults) }}" + From 3a614c3392bef8aa2efe69cff5dea51fef96d0aa Mon Sep 17 00:00:00 2001 From: syndr Date: Tue, 21 Jan 2025 22:26:25 -0700 Subject: [PATCH 18/18] Refactor init role Update default template Default to no overwrite for resource playbooks More modular platform support (just add config under defaults/platforms --- roles/docker_platform/meta/main.yml | 1 + roles/init/defaults/main.yml | 7 ++++-- roles/init/files/init.yml | 30 +++++++++++++++++++++++++ roles/init/tasks/asserts.yml | 9 +++++--- roles/init/tasks/main.yml | 14 ++++++++---- roles/init/tasks/meta.yml | 11 ++++----- roles/init/templates/collections.yml.j2 | 3 ++- roles/init/templates/molecule.yml.j2 | 14 ++++++------ roles/init/templates/verify.yml.j2 | 2 ++ roles/init/vars/main.yml | 10 ++++++++- 10 files changed, 78 insertions(+), 23 deletions(-) diff --git a/roles/docker_platform/meta/main.yml b/roles/docker_platform/meta/main.yml index 3c2728b..42fac39 100644 --- a/roles/docker_platform/meta/main.yml +++ b/roles/docker_platform/meta/main.yml @@ -1,6 +1,7 @@ galaxy_info: author: syndr description: Create a docker-based test platform for Molecule + role_name: docker_platform # If the issue tracker for your role is not on github, uncomment the # next line and provide a value diff --git a/roles/init/defaults/main.yml b/roles/init/defaults/main.yml index d599215..cf6a03f 100644 --- a/roles/init/defaults/main.yml +++ b/roles/init/defaults/main.yml @@ -9,7 +9,7 @@ init_project_type: auto init_platform_type: docker # Version of this collection that should be used by the Molecule test -# - Set to "" to attempt to use the running version +# - Set to "current" to attempt to use the running version init_collection_version: latest # Source of the collection that this role is part of (galaxy, git) @@ -40,11 +40,14 @@ init_platforms: [] # Create backups of any files that would be clobbered by running this role init_file_backup: true +# Overwrite any existing 'resource' playbook files in the 'molecule/resources' directory +init_overwrite_resources: false + # Initialize ARA support in the Molecule configuration -- https://ara.recordsansible.org/ init_ara_support: true # Default configuration to be added to generated molecule.yml if init_platforms is not defined -init_platform_defaults: "{{ lookup('file', 'defaults/platforms/' + init_platform_type + '.yml') | from_yaml }}" +init_platform_defaults: "{{ lookup('file', role_path + '/defaults/platforms/' + init_platform_type + '.yml') | from_yaml }}" # Filesystem location of the molecule scenario being initialized init_scenario_dir: "{{ molecule_scenario_directory | default(playbook_dir) }}" diff --git a/roles/init/files/init.yml b/roles/init/files/init.yml index a2bbd1e..b8ed64c 100644 --- a/roles/init/files/init.yml +++ b/roles/init/files/init.yml @@ -3,6 +3,8 @@ - name: Provision file structure hosts: localhost + connection: local + gather_facts: false tasks: - name: Launch provisioner ansible.builtin.include_role: @@ -10,9 +12,37 @@ vars: # Supported platform types are: docker, ec2 init_platform_type: docker + # Supported collection sources are: git, galaxy init_collection_source: git + # Version of this collection that should be used by the Molecule test # - Set to "" to attempt to use the running version init_collection_version: latest + # Initialize ARA support in the Molecule configuration -- https://ara.recordsansible.org/ + init_ara_support: true + + # Create backups of any files that would be clobbered by running this role + init_file_backup: true + + # Overwrite any existing 'resource' playbook files in the 'molecule/resources' directory + init_overwrite_resources: false + + # Platforms that this test configuration should test + # list of dicts, each required to contain: + # name: (string) + # type: (string) + # config: (dictionary, configuration for specified "type") + # + # for example, in the case of the "docker" type: + # config: + # image: (string, container image path) + # systemd: (true/false) + # modify_image: (true/false) + # modify_image_buildpath: (string) # path to directory containing Dockerfile + # privileged: (true/false) + # + # If not specified, the role will attempt to use the default platform configuration + init_platforms: [] + diff --git a/roles/init/tasks/asserts.yml b/roles/init/tasks/asserts.yml index 6a825bc..a34fe05 100644 --- a/roles/init/tasks/asserts.yml +++ b/roles/init/tasks/asserts.yml @@ -76,7 +76,7 @@ - init_platform_defaults is defined - init_platform_defaults | type_debug == 'dict' fail_msg: Unable to find platform defaults! Add support to the 'init' role for the platform '{{ init_platform_type }}'. - success_msg: Platform defaults found. + success_msg: Platform defaults found. Defaults are '{{ init_platform_defaults }}' - name: Validate runtime platform configuration ansible.builtin.assert: @@ -85,6 +85,9 @@ - __init_runtime_platforms | map(attribute='name') | unique | length == __init_runtime_platforms | length - __init_runtime_platforms | map(attribute='type') | unique | length == 1 - __init_runtime_platforms | length > 0 - fail_msg: Runtime platform configuration is not sane! - success_msg: Sanity check passed. + fail_msg: > + Runtime platform configuration is not sane! + Platform configuration: + {{ __init_runtime_platforms }} + success_msg: Sanity check passed with platforms '{{ __init_runtime_platforms }}' diff --git a/roles/init/tasks/main.yml b/roles/init/tasks/main.yml index ee461b7..c023567 100644 --- a/roles/init/tasks/main.yml +++ b/roles/init/tasks/main.yml @@ -14,13 +14,18 @@ - name: Deploy Molecule configuration files vars: __init_scenario_dir: "{{ init_scenario_dir if init_scenario_dir.split('/')[-2] == 'molecule' else init_scenario_dir + '/default' }}" - __init_resources_dir: "{{ __init_scenario_dir.split('/')[:-2] | join('/') }}/resources" + __init_resources_dir: "{{ __init_scenario_dir.split('/')[:-1] | join('/') }}/resources" __init_scenario_name: "{{ init_scenario_dir | basename }}" block: - - name: Create molecule scenario directory + - name: Create molecule directory # Assume that we're inside a 'molecule' directory (checked via asserts.yml) + loop: + - "{{ __init_scenario_dir }}" + - "{{ __init_resources_dir }}" + loop_control: + loop_var: __init_directory ansible.builtin.file: - path: "{{ __init_scenario_dir }}" + path: "{{ __init_directory }}" state: directory mode: "0755" @@ -47,8 +52,9 @@ ansible.builtin.template: src: "{{ role_path }}/templates/{{ __init_item }}.j2" dest: "{{ __init_resources_dir }}/{{ __init_item }}" - mode: 0644 + mode: "0644" backup: "{{ init_file_backup }}" + force: "{{ init_overwrite_resources }}" - name: Link Ansible dependency loop: diff --git a/roles/init/tasks/meta.yml b/roles/init/tasks/meta.yml index 2f3b41b..85eeffe 100644 --- a/roles/init/tasks/meta.yml +++ b/roles/init/tasks/meta.yml @@ -35,7 +35,7 @@ ansible.builtin.assert: that: - __init_collection_meta is truthy - - __init_collection_meta is mappint + - __init_collection_meta is mapping - __init_collection_meta.name is defined - __init_collection_meta.version is defined - __init_collection_meta.namespace is defined @@ -47,7 +47,8 @@ ansible.builtin.set_fact: __init_collection_version: >- {{ init_collection_version if init_collection_version is truthy - else __init_collection_meta.collection_info.version | default(init_collection_defaults.version) }} - __init_collection_name: "{{ __init_collection_meta.collection_info.name | default(init_collection_defaults.name) }}" - __init_collection_namespace: "{{ __init_collection_meta.collection_info.namespace | default(init_collection_defaults.namespace) }}" - __init_collection_repository: "{{ __init_collection_meta.collection_info.repository | default(init_collection_defaults.repository) }}" + else __init_collection_meta.version }} + __init_collection_name: "{{ __init_collection_meta.name }}" + __init_collection_namespace: "{{ __init_collection_meta.namespace }}" + __init_collection_repository: "{{ __init_collection_meta.repository }}" + diff --git a/roles/init/templates/collections.yml.j2 b/roles/init/templates/collections.yml.j2 index 5f634a3..ef5eb5f 100644 --- a/roles/init/templates/collections.yml.j2 +++ b/roles/init/templates/collections.yml.j2 @@ -5,9 +5,10 @@ collections: {% if init_collection_source == 'git' %} - name: git+{{ __init_collection_repository }}.git type: git - version: v{{ __init_collection_version }} {% elif init_collection_source == 'galaxy' %} - name: {{ __init_collection_namespace }}.{{ __init_collection_name }} +{% endif %} +{% if __init_collection_version is truthy %} version: {{ __init_collection_version }} {% endif %} diff --git a/roles/init/templates/molecule.yml.j2 b/roles/init/templates/molecule.yml.j2 index 31c85d4..64c375c 100644 --- a/roles/init/templates/molecule.yml.j2 +++ b/roles/init/templates/molecule.yml.j2 @@ -10,17 +10,17 @@ driver: login_cmd_template: 'docker exec -ti {instance} bash --login' {% endif %} platforms: -{% for init_platform in init_platforms %} + # See platform documentation at {{ __init_collection_repository }}/tree/main/roles/{{ init_platform_type }}_platform/README.md +{% for init_platform in __init_runtime_platforms %} - name: {{ init_platform.name | default('docker-platform') }} type: {{ init_platform.type }} - # See platform documentation at {{ __init_collection_repository }}/tree/main/roles/{{ init_platform.type }}_platform/README.md - {%- for key, value in init_platform.items() -%} - {%- if key not in ['name', 'type', 'hostvars'] %} +{% for key, value in init_platform.items() -%} +{% if key not in ['name', 'type', 'hostvars'] %} {{ key }}: {{ value }} - {% endif -%} - {%- endfor %} +{% endif %} +{%- endfor %} hostvars: {{ init_platform.hostvars | default({}) }} - {% endfor %} +{% endfor %} provisioner: name: ansible log: True diff --git a/roles/init/templates/verify.yml.j2 b/roles/init/templates/verify.yml.j2 index 3237d2c..f842ba8 100644 --- a/roles/init/templates/verify.yml.j2 +++ b/roles/init/templates/verify.yml.j2 @@ -1,6 +1,7 @@ --- # Verify that the role being tested has done what it's supposed to +{% raw %} - name: Verify hosts: molecule tasks: @@ -18,4 +19,5 @@ - name: Add your verification tasks here ansible.builtin.debug: msg: "IE: For a 'users' role, check that the test user exists" +{% endraw %} diff --git a/roles/init/vars/main.yml b/roles/init/vars/main.yml index 1a995af..fc3ab2e 100644 --- a/roles/init/vars/main.yml +++ b/roles/init/vars/main.yml @@ -16,5 +16,13 @@ __init_supported_platform_types: - ec2 # The platforms to be configured by this role -__init_runtime_platforms: "{{ init_platforms | combine(init_platforms_defaults) }}" +__init_runtime_platforms: >- + {%- set __init_platforms = [] -%} + {%- for __init_platform in init_platforms -%} + {%- set _ = __init_platforms.append(__init_platform | combine(init_platform_defaults)) -%} + {%- endfor -%} + {%- if __init_platforms | length == 0 -%} + {%- set __init_platforms = [init_platform_defaults] -%} + {%- endif -%} + {{ __init_platforms }}