Skip to content

Commit

Permalink
docs: Mention reusable workflows. Remove maintenance steps for workfl…
Browse files Browse the repository at this point in the history
…ows that are always reused.
  • Loading branch information
jpmckinney committed Nov 27, 2024
1 parent e5058f5 commit 22a581a
Show file tree
Hide file tree
Showing 12 changed files with 104 additions and 51 deletions.
26 changes: 18 additions & 8 deletions docs/docker/build.rst
Original file line number Diff line number Diff line change
@@ -1,12 +1,29 @@
Continuous integration
======================

Create a ``.github/workflows/docker.yml`` file.
Create a ``.github/workflows/docker.yml`` file. As a base, use:

.. literalinclude:: samples/docker.yml
:language: yaml

.. tip::

In most cases, you can reuse either the `docker-single <https://github.com/open-contracting/.github/blob/main/.github/workflows/docker-single.yml>`__ or `docker-django <https://github.com/open-contracting/.github/blob/main/.github/workflows/docker-django.yml>`__ workflow. For example:

.. code-block:: yaml
jobs:
docker:
uses: open-contracting/.github/.github/workflows/docker-single.yml@main
.. note::

This assumes there is already a :doc:`"CI" workflow <../python/ci>` that runs tests on the ``main`` branch.

.. note::

The `docker/build-push-action <https://github.com/docker/build-push-action>`__ step uses `BuildKit <https://docs.docker.com/build/buildkit/>`__ by default.

If you need to build multiple images, then for each image:

#. Include a ``docker/build-push-action`` step.
Expand All @@ -17,13 +34,6 @@ If you need to build multiple images, then for each image:

#. Add a suffix to the repository name under the ``tags`` key.

.. literalinclude:: samples/docker.yml
:language: yaml

.. note::

The `docker/build-push-action <https://github.com/docker/build-push-action>`__ step uses `BuildKit <https://docs.docker.com/build/buildkit/>`__ by default.

..
The following would simplify the workflow somewhat. However, it would not work when building multiple images, producing an inconsistent approach across repositories.
Expand Down
21 changes: 20 additions & 1 deletion docs/docker/samples/docker.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,23 @@ on:
- completed
jobs:
docker:
uses: open-contracting/.github/.github/workflows/docker-single.yml@main
if: ${{ github.event.workflow_run.conclusion == 'success' }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# https://github.com/docker/login-action#github-container-registry
- uses: docker/login-action@v3
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
# https://github.com/docker/setup-buildx-action#usage
- uses: docker/setup-buildx-action@v3
# https://github.com/docker/build-push-action#usage
- uses: docker/build-push-action@v6
with:
push: true
tags: |
ghcr.io/${{ github.repository }}:latest
cache-from: type=gha
cache-to: type=gha,mode=max
12 changes: 12 additions & 0 deletions docs/javascript/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,18 @@ Create a ``.github/workflows/js.yml`` file. As a base, use:
.. literalinclude:: samples/js.yml
:language: yaml

.. tip::

In most cases, you can reuse the `js <https://github.com/open-contracting/.github/blob/main/.github/workflows/js.yml>`__ workflow. For example:

.. code-block:: yaml
jobs:
lint:
uses: open-contracting/.github/.github/workflows/js.yml@main
with:
filenames: path/to/file.js
Reference
---------

Expand Down
11 changes: 7 additions & 4 deletions docs/javascript/samples/js.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,10 @@
name: Lint JavaScript
on: [push, pull_request]
jobs:
lint:
uses: open-contracting/.github/.github/workflows/js.yml@main
with:
filenames: path/to/file.js
build:
if: ${{ github.event_name == 'push' || github.event.pull_request.head.repo.full_name != github.repository }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: biomejs/setup-biome@v2
- run: biome ci --indent-style=space --line-width=119 .
16 changes: 2 additions & 14 deletions docs/python/ci.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ Continuous integration

Workflows for:

- Linting :doc:`Python<linting>`, :ref:`JavaScript<javascript-ci>` and :ref:`shell scripts<shell-ci>`
- :ref:`Releasing packages<python-package-release-process>`
- Linting :ref:`Python<linting-ci>`, :ref:`JavaScript<javascript-ci>` and :ref:`shell scripts<shell-ci>`
- :ref:`Releasing packages<pypi-ci>`
- :ref:`Checking translations<i18n-ci>`
- :ref:`Checking accessibility<a11y-ci>`

Expand Down Expand Up @@ -291,12 +291,6 @@ Find ``ci.yml`` files without ``lint.yml`` files, and vice versa:
find . \( -name lint.yml \) -exec bash -c 'if [[ -z $(find $(echo {} | cut -d/ -f2) -name ci.yml) ]]; then echo {}; fi' \;
find . \( -name ci.yml \) ! -path '*/node_modules/*' -exec bash -c 'if [[ -z $(find $(echo {} | cut -d/ -f2) -name lint.yml) ]]; then echo {}; fi' \;
Find and compare ``lint.yml`` files:

.. code-block:: bash
find . -name lint.yml -exec bash -c 'sha=$(shasum {} | cut -d" " -f1); if [[ ! "9773a893d136df0dc82deddedd8af8563969c04a 9222eac95ab63f3c2d983ba3cf4629caea53a72e fc3eff616a7e72f41c96e48214d244c9058dbc83 953ef7f0815d49226fd2d05db8df516fff2e3fdb dfe1c0d1fbdb18bb1e2b3bcfb1f0c10fe6b06bc4" =~ $sha ]]; then echo -e "\n\033[0;32m{}\033[0m"; echo $sha; cat {}; fi' \;
Find and compare ``pypi.yml`` files:

.. code-block:: bash
Expand All @@ -309,12 +303,6 @@ Find repositories for Python packages but without ``pypi.yml`` files:
find . -name pyproject.toml ! -path '*/node_modules/*' -exec bash -c 'if grep classifiers {} > /dev/null && [[ -z $(find $(echo {} | cut -d/ -f2) -name pypi.yml) ]]; then echo {}; fi' \;
Find and compare ``i18n.yml`` files:

.. code-block:: bash
find . -name i18n.yml -exec bash -c 'echo $(shasum {} | cut -d" " -f1) {}' \;
Find repositories with ``LC_MESSAGES`` directories but without ``i18n.yml`` files:

.. code-block:: bash
Expand Down
4 changes: 2 additions & 2 deletions docs/python/i18n.rst
Original file line number Diff line number Diff line change
Expand Up @@ -111,12 +111,12 @@ Repositories that support multiple locales should test that translation files ar

Create a ``.github/workflows/i18n.yml`` file.

For a Django application, use the following. Change the ``python-version`` to match the version used to compile the :doc:`requirements_dev.txt file<requirements>`.
For a Django application, reuse the `i18n-django <https://github.com/open-contracting/.github/blob/main/.github/workflows/i18n-django.yml>`__ workflow. Change the ``python-version`` to match the version used to compile the :doc:`requirements_dev.txt file<requirements>`.

.. literalinclude:: ../../cookiecutter-django/{{cookiecutter.project_slug}}/.github/workflows/i18n.yml
:language: yaml

For a Babel project, adapt:
For a Babel project, reuse the `i18n-babel <https://github.com/open-contracting/.github/blob/main/.github/workflows/i18n-babel.yml>`__ workflow. Change the ``command`` as needed:

.. literalinclude:: samples/i18n/babel.yml
:language: yaml
Expand Down
2 changes: 1 addition & 1 deletion docs/python/layout.rst
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ Packages
.. note::

We don't use the `src/ layout <https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure>`__ (`more <https://blog.ionelmc.ro/2015/02/24/the-problem-with-packaging-in-python/>`__). In practice, we rarely encounter the problems it solves, and our use of :ref:`check-manifest<python-package-release-process>` and `test_requirements.py <https://github.com/open-contracting/standard-maintenance-scripts/blob/main/tests/test_requirements.py>`__ guard against those problems.
We don't use the `src/ layout <https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure>`__ (`more <https://blog.ionelmc.ro/2015/02/24/the-problem-with-packaging-in-python/>`__). In practice, we rarely encounter the problems it solves, and our use of `check-manifest <https://github.com/open-contracting/.github/blob/main/.github/workflows/lint.yml>`__ and `test_requirements.py <https://github.com/open-contracting/standard-maintenance-scripts/blob/main/tests/test_requirements.py>`__ guard against those problems.

Modules
-------
Expand Down
4 changes: 3 additions & 1 deletion docs/python/linting.rst
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,9 @@ To avoid pushing commits that fail formatting or linting checks, new projects sh
Continuous integration
----------------------

Create a ``.github/workflows/lint.yml`` file. The :doc:`django` and :doc:`Pypackage<packages>` Cookiecutter templates contain default workflows.
Create a ``.github/workflows/lint.yml`` file.

The :doc:`django` and :doc:`Pypackage<packages>` Cookiecutter templates contain default workflows.

.. seealso::

Expand Down
22 changes: 13 additions & 9 deletions docs/python/packages.rst
Original file line number Diff line number Diff line change
Expand Up @@ -75,21 +75,25 @@ The template reads the documentation from a ``README.rst`` file. To convert a ``

:doc:`Python documentation guide<documentation>`

.. _python-package-release-process:
Publish releases
----------------

Release process
---------------
.. _pypi-ci:

Continuous integration
~~~~~~~~~~~~~~~~~~~~~~

.. admonition:: One-time setup
To publish tagged releases to PyPI, create a ``.github/workflows/pypi.yml`` file:

To publish tagged releases to PyPI, create a ``.github/workflows/pypi.yml`` file:
.. literalinclude:: ../../cookiecutter-pypackage/{{cookiecutter.repository_name}}/.github/workflows/pypi.yml
:language: yaml

.. literalinclude:: ../../cookiecutter-pypackage/{{cookiecutter.repository_name}}/.github/workflows/pypi.yml
:language: yaml
The *open-contracting* organization sets the ``PYPI_API_TOKEN`` `organization secret <https://github.com/organizations/open-contracting/settings/secrets/actions>`__ to the API token of the *opencontracting* `PyPI user <https://pypi.org/manage/account/#api-tokens>`__, and ``TEST_PYPI_API_TOKEN`` to that of the *opencontracting* Test PyPI user.

The *open-contracting* organization sets the ``PYPI_API_TOKEN`` `organization secret <https://github.com/organizations/open-contracting/settings/secrets/actions>`__ to the API token of the *opencontracting* `PyPI user <https://pypi.org/manage/account/#api-tokens>`__, and ``TEST_PYPI_API_TOKEN`` to that of the *opencontracting* Test PyPI user.
After publishing the first release to PyPI, :ref:`add additional owners <pypi-access>`.

After publishing the first release to PyPI, :ref:`add additional owners <pypi-access>`.
Release process
~~~~~~~~~~~~~~~

#. Ensure that you are on an up-to-date ``main`` branch:

Expand Down
4 changes: 2 additions & 2 deletions docs/services/admin.rst
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ Third-party sysadmins can be added with "Global Administrator" access.
PyPI
----

For each package, there should be a minimum of two `Owner <https://pypi.org/help/#collaborator-roles>`__ users from OCP, including the `opencontracting <https://pypi.org/user/opencontracting/>`__ user, whose API token is used in `pypi.yml workflows <python-package-release-process>`__.
For each package, there should be a minimum of two `Owner <https://pypi.org/help/#collaborator-roles>`__ users from OCP, including the `opencontracting <https://pypi.org/user/opencontracting/>`__ user, whose API token is used in `pypi.yml workflows <pypi-ci>`__.

Only users who are reasonably expected to upload releases should have the Maintainer role.

Expand Down Expand Up @@ -204,7 +204,7 @@ Third-party sysadmins can be added with "Member" access.
Test PyPI
---------

For each package, the `opencontracting <https://test.pypi.org/user/opencontracting/>`__ user can be the single Owner, whose API token is used in `pypi.yml workflows <python-package-release-process>`__.
For each package, the `opencontracting <https://test.pypi.org/user/opencontracting/>`__ user can be the single Owner, whose API token is used in `pypi.yml workflows <pypi-ci>`__.

Transifex
---------
Expand Down
20 changes: 13 additions & 7 deletions docs/shell/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,23 @@ And:
Continuous integration
----------------------

Create a ``.github/workflows/shell.yml`` file with:
Create a ``.github/workflows/shell.yml`` file. As a base, use:

.. literalinclude:: samples/shell.yml
:language: yaml

.. tip::

In most cases, you can reuse the `shell <https://github.com/open-contracting/.github/blob/main/.github/workflows/shell.yml>`__ workflow. For example:

.. code-block:: yaml
jobs:
lint:
uses: open-contracting/.github/.github/workflows/shell.yml@main
with:
ignore: file.sh
Maintenance
~~~~~~~~~~~

Expand All @@ -83,12 +95,6 @@ Find repositories with shell scripts but without ``shell.yml`` files:
find . \( -path '*/script/*' -o -name '*.sh' \) ! -path '*/.mypy_cache/*' ! -path '*/node_modules/*' ! -path '*/vendor/*' -exec bash -c 'if [[ -z $(find $(echo {} | cut -d/ -f2) -name shell.yml) ]]; then echo {}; fi' \;
Find and compare ``shell.yml`` files:

.. code-block:: bash
find . -name shell.yml -exec bash -c 'echo $(shasum {} | cut -d" " -f1) {}' \;
Reference
---------

Expand Down
13 changes: 11 additions & 2 deletions docs/shell/samples/shell.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
name: Lint Shell
on: [push, pull_request]
jobs:
lint:
uses: open-contracting/.github/.github/workflows/shell.yml@main
build:
if: ${{ github.event_name == 'push' || github.event.pull_request.head.repo.full_name != github.repository }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: |
sudo apt update
sudo apt install devscripts shellcheck shfmt
- run: checkbashisms $(shfmt -f .)
- run: shellcheck $(shfmt -f .)
- run: shfmt -d -i 4 -sr $(shfmt -f .)

0 comments on commit 22a581a

Please sign in to comment.