Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial version to make it run on Podman #69

Open
wants to merge 24 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
cf48e08
Initial version to make it run on Podman
Thulium-Drake Jul 14, 2023
ea3b3ad
Fix handler for restarting container as podman_container does not do …
Thulium-Drake Jul 17, 2023
89d4303
Only attempt migrating SQLite db when ARA has actually been configure…
Thulium-Drake Jul 17, 2023
8223fc9
Create container, do not start, wait for config file to be written pr…
Thulium-Drake Jul 17, 2023
67a016f
Reworked service generation for podman service, which also in turn ma…
Thulium-Drake Jul 17, 2023
34832a7
Added tests for Podman
Thulium-Drake Jul 17, 2023
455f182
Added zuul job
Thulium-Drake Jul 17, 2023
4cadbc5
Enable jobs
Thulium-Drake Jul 17, 2023
19348f0
Fix typo
Thulium-Drake Jul 17, 2023
c4e67a8
Process feedback
Thulium-Drake Jul 22, 2023
8a2b2c5
Processed feedback, look in detail later
Thulium-Drake Jul 23, 2023
c8dea74
Change the command to generate a random string to one that does not r…
Thulium-Drake Sep 6, 2023
4f51a14
Fix SELinux context for containers, this has no impact on non-SELinux…
Thulium-Drake Sep 7, 2023
3254c8b
Fix handling in case no secret key has been defined or it is null in …
Thulium-Drake Sep 7, 2023
2d30ce9
Enable service after creating it
Thulium-Drake Sep 11, 2023
ede3612
Removed migration tasks to prevent opening can of worms, only warn ab…
Thulium-Drake Apr 29, 2024
fa5c12e
Reworked podman tasks into one task (as you do, you learn :) )
Thulium-Drake Sep 9, 2024
41cf660
Clean up unused variable and made image variable (to allow custom loc…
Thulium-Drake Oct 9, 2024
81d5f58
fix quoting
Thulium-Drake Oct 14, 2024
e3a08b3
Only configure selinux context when selinux is enabled
Thulium-Drake Oct 21, 2024
1bf1325
Something seems off with the vars_file, lets see what happens with di…
Thulium-Drake Oct 21, 2024
cc62969
Test podman job with ansible 9
dmsimard Oct 30, 2024
c6ad33a
s/ansible_version/ansible-version/
dmsimard Oct 30, 2024
19546a6
Fully quality container image name
dmsimard Oct 30, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .zuul.d/jobs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,16 @@
authentication enabled.
run: tests/with_client_cert.yaml

- job:
name: ara-role-api-podman
parent: ara-role-integration-base
nodeset: ara-multinode
description: |
Desploys the ARA API server on Fedora 36 as well as CentOS Stream 8/9
in a Podman container and tests it using the default sqlite database backend.
run: tests/with_podman.yaml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool! We should bump that to Fedora 38 like I did recently for ara but I can take care of that in another PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's split that, first let's make this work ^^

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like that there is already a job for testing podman and that it is even passing, nice work :)

ansible-version: '9'

# TODO: The job should build a package from current source and test that package
# instead of the package in the stable distribution.
- job:
Expand Down
2 changes: 2 additions & 0 deletions .zuul.d/project.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
- ara-role-api-postgresql
- ara-role-api-gunicorn-nginx
- ara-role-api-gunicorn-nginx-client-cert
- ara-role-api-podman
- ara-role-api-fedora-packages:
voting: false
gate:
Expand All @@ -16,3 +17,4 @@
- ara-role-api-postgresql
- ara-role-api-gunicorn-nginx
- ara-role-api-gunicorn-nginx-client-cert
- ara-role-api-podman
6 changes: 5 additions & 1 deletion roles/ara_api/defaults/main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,12 +43,16 @@ ara_api_venv_path: "{{ ara_api_root_dir }}/virtualenv"
# - source [default]: installs from a local or remote git repository
# - distribution: installs from distribution packages, if available
# - pypi : installs from pypi
# - podman : installs as a podman container
ara_api_install_method: source

# When installing from source, the URL or filesystem path where the git source
# repository can be cloned from.
ara_api_source: "https://github.com/ansible-community/ara"

# Image to pull from the container registry when running with Podman
ara_api_image: 'quay.io/recordsansible/ara-api'

# When installing from source, location where the source repository will be checked out to.
ara_api_source_checkout: "{{ ara_api_root_dir }}/git/ara"

Expand All @@ -57,7 +61,7 @@ ara_api_source_checkout: "{{ ara_api_root_dir }}/git/ara"
# When installing from PyPi, it would be a version number that has been released.
# When using "latest" as the source version, HEAD will be used
# When using "latest" as the pypi version, the latest release will be used
ara_api_version: master
ara_api_version: latest

# The frontend/web server for serving the ARA API
# It is recommended to specify a web server when deploying a production environment.
Expand Down
13 changes: 6 additions & 7 deletions roles/ara_api/tasks/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,17 +40,16 @@
ara_api_secret_key: "{{ config[ara_api_env]['SECRET_KEY'] }}"
no_log: "{{ ara_api_secure_logging }}"

# If no secret key has been provided and this is the first time we are
# running, generate a new random secret key that will be persisted in the
# If no secret key has been provided or it is not present in the current
# configuration, generate a new random secret key that will be persisted in the
# configuration file.
- when:
- ara_api_secret_key is none
- not settings_stat.stat.exists
- ara_api_secret_key is none or ara_api_secret_key == ''
block:
- name: Generate a random secret key
environment:
PATH: "{{ path_with_virtualenv }}"
command: "{{ ara_api_python_command }} -c 'from django.utils.crypto import get_random_string; print(get_random_string(length=50))'"
shell: |
set -o pipefail
tr -dc A-Za-z0-9 </dev/urandom | head -c 50 ; echo ''
no_log: "{{ ara_api_secure_logging }}"
register: generated_key

Expand Down
38 changes: 38 additions & 0 deletions roles/ara_api/tasks/install/podman.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
---
- name: Detecting existing PyPI installation
ansible.builtin.stat:
path: "{{ ara_api_venv_path }}"
register: existing_pypi_install

- name: Notify about existing PyPI installation
ansible.builtin.debug:
msg: |
You seem to have ARA-API installed via PyPI in the past, you might
want to clean up that installation and migrate your data
when: existing_pypi_install['stat']['exists']

- name: Override file locations with path in container
ansible.builtin.set_fact:
ara_api_database_name: "/opt/ara/ansible.sqlite"
ara_api_log_dir: "/opt/ara/logs"
ara_api_settings: "{{ ara_api_root_dir }}/settings.yaml"

- name: Ensure ARA API container
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is kind of a gotcha here.

I think it is succeeding in starting the container when running from an unprivileged user (since podman can start rootless no problem) but then it tries to set up the systemd unit for it and fails due to lack of privileges.

If we set become: true on the task that starts the container, I would presume that it'd start it as root and not as the unprivileged user. It would work, but wouldn't be the same thing. If we go that route it would be worthy of being mentioned as a caveat since the pip/source installs supports running as unprivileged.

Note that there would also be a failure on the following task (service) since it doesn't have become: true.

Copy link
Contributor

@dmsimard dmsimard Nov 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I propose that we don't use the generate_systemd argument of the podman_container module and instead use podman_generate_systemd: https://docs.ansible.com/ansible/latest/collections/containers/podman/podman_generate_systemd_module.html#examples

# Example of creating a container and integrate it into systemd
- name: A postgres container must exist, stopped
  containers.podman.podman_container:
    name: postgres_local
    image: docker.io/library/postgres:latest
    state: stopped

- name: Systemd unit files for postgres container must exist
  containers.podman.podman_generate_systemd:
    name: postgres_local
    dest: ~/.config/systemd/user/

- name: Postgres container must be started and enabled on systemd
  ansible.builtin.systemd:
    name: container-postgres_local
    scope: user
    daemon_reload: true
    state: started
    enabled: true

This way we don't need to elevate privileges for the task that starts the container so the ara API container can remain rootless unless the role is run as root.

When running as root it'd look more like this (from the examples):

- become: true
  block:
    - name: Generate systemd unit file for postgres container
      containers.podman.podman_generate_systemd:
        name: postgres
        new: true
        no_header: true
        dest: /etc/systemd/system

    - name: Ensure postgres container is started and enabled
      ansible.builtin.systemd:
        name: container-postgres
        daemon_reload: true
        state: started
        enabled: true

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have tried to do this but it's still WIP: dmsimard@8fa017e

In truth I am running into unrelated issues amongst which debian 12 is not supported so it would break when I wanted to test this with it: #68

I will revive that PR.

containers.podman.podman_container:
name: ara-api
image: "{{ ara_api_image }}:{{ ara_api_version }}"
pull: newer
state: present
auto_remove: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it really be "auto_remove" ? If it deletes itself on exit, then the systemd service wouldn't really have the opportunity to stop or restart it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I put it there to make the podman generate systemd command below idempotent with what is created :)

The systemd service knows how to handle it, but it needs an existing container to template from

generate_systemd: "{{ ara_api_systemd_config }}"
ports:
- 127.0.0.1:8000:8000
volume:
- "{{ ara_api_root_dir }}:/opt/ara{{ (ansible_facts['selinux']['status'] == 'enabled') | ternary(':z', '') }}"

- name: Ensure service
ansible.builtin.systemd:
name: ara-api.service
state: started
enabled: true
daemon_reload: true
5 changes: 4 additions & 1 deletion roles/ara_api/tasks/main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,13 @@

- name: Include configuration of the database engine
include_tasks: "database_engine/{{ ara_api_database_engine }}.yaml"
when: ara_api_install_method != 'podman'

- name: Include installation of the WSGI backend server
include_tasks: "wsgi_server/{{ ara_api_wsgi_server }}.yaml"
when: ara_api_wsgi_server is not none
when:
- ara_api_wsgi_server is not none
- ara_api_install_method != 'podman'

- name: Include installation of the frontend server
include_role:
Expand Down
9 changes: 9 additions & 0 deletions roles/ara_api/vars/main.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
ara_api_systemd_config:
path: '/etc/systemd/system'
restart_policy: 'always'
time: 120
names: true
new: true
container_prefix: ''
separator: ''
31 changes: 31 additions & 0 deletions tests/with_podman.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
---
# Copyright (c) 2020 The ARA Records Ansible authors
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)

- name: Deploy and test ARA API with podman
hosts: ara-api-server
gather_facts: yes
vars:
ara_api_install_method: podman
ara_api_version: latest
ara_api_root_dir: "{{ ansible_user_dir }}/.ara-tests"
ara_api_secret_key: testing
ara_api_debug: true
ara_api_log_level: DEBUG
# Configure cleanup crons to exercise the code path during tests
ara_api_configure_cron: true
tasks:
- name: Install podman
become: yes
package:
name: podman
state: present

- name: Set up the API with the ara_api Ansible role
include_role:
name: ara_api
public: yes

# These are tasks rather than a standalone playbook to give us an easy
# access to all the variables within the same play.
- include_tasks: test_tasks.yaml