Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cache to docker builder #548

Merged
merged 6 commits into from
Oct 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 30 additions & 3 deletions .github/workflows/docker_ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
id: meta
uses: docker/metadata-action@v5
with:
images: fiware/wirecloud
images: fiware/wirecloud, quay.io/fiware/wirecloud

# https://github.com/docker/setup-qemu-action
- name: Set up QEMU
Expand All @@ -41,12 +41,37 @@ jobs:
id: buildx
uses: docker/setup-buildx-action@v3

# Get cache and inject it into docker (https://github.com/reproducible-containers/buildkit-cache-dance)
- name: Cache
uses: actions/cache@v4
id: cache
with:
path: |
var-cache-apt
root-cache-pip
key: cache-${{ hashFiles('Dockerfile') }}
- name: inject cache into docker
uses: reproducible-containers/buildkit-cache-dance@v3
with:
cache-map: |
{
"var-cache-apt": "/var/cache/apt",
"root-cache-pip": "/root/.cache/pip"
}

- name: Login to DockerHub
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Login to Quay.io
if: github.event_name != 'pull_request'
uses: docker/login-action@v3
with:
registry: quay.io
username: ${{ secrets.QUAY_USERNAME }}
password: ${{ secrets.QUAY_TOKEN }}

- name: Build amd64 to test
uses: docker/build-push-action@v5
Expand All @@ -55,7 +80,7 @@ jobs:
platforms: linux/amd64
tags: fiware/wirecloud:latest
load: true #Load into docker daemon to test
cache-from: type=registry,ref=ghcr.io/dmunozv04/isponsorblocktv:buildcache
cache-from: type=gha

- name: Setup python to test
uses: actions/setup-python@v5
Expand All @@ -78,4 +103,6 @@ jobs:
platforms: linux/amd64, linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
labels: ${{ steps.meta.outputs.labels }}
cache-from: type=gha
cache-to: type=gha,mode=max
5 changes: 3 additions & 2 deletions docker-platform/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,8 @@
ARG WIRECLOUD_PYTHON_VERSION=3.10
ARG WIRECLOUD_DEBIAN_VERSION=bookworm

FROM python:${WIRECLOUD_PYTHON_VERSION}-${WIRECLOUD_DEBIAN_VERSION} as builder
#$BUILDPLATFORM allows to build the image on the native platform of the builder, avoiding emulation
FROM --platform=$BUILDPLATFORM python:${WIRECLOUD_PYTHON_VERSION}-${WIRECLOUD_DEBIAN_VERSION} AS builder

RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
apt-get update && \
Expand Down Expand Up @@ -44,7 +45,7 @@ RUN --mount=type=bind,from=builder,source=/wirecloud/src/dist/,target=/dist/ \
--mount=type=cache,target=/root/.cache/pip,sharing=locked \
pip install /dist/*.whl \
social-auth-app-django "gunicorn==21.2" "psycopg2==2.8.6" pylibmc pysolr "elasticsearch==2.4.1" \
"regex==2019.02.18" "channels<2.3" "channels-redis" "wirecloud-keycloak>=0.3.0" && \
"regex==2019.02.18" "channels<2.3" "channels-redis" "wirecloud-keycloak>=0.3.0" "whitenoise" && \
pip uninstall selenium -y

RUN adduser --system --group --shell /bin/bash wirecloud && \
Expand Down
14 changes: 11 additions & 3 deletions docker-platform/docker-entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,22 +20,30 @@ case "$1" in
createsuperuser)
manage.py createsuperuser
;;
gunicorn)
gunicorn|gunicorn-aio)

manage.py collectstatic --noinput
manage.py migrate --fake-initial
manage.py populate

# select wirecloud_instance.wsgi:application or wirecloud_instance.wsgi:allInOneApplication depending on the value of $1
if [ "$1" = 'gunicorn' ]; then
WSGI_APPLICATION='wirecloud_instance.wsgi:application'
else
WSGI_APPLICATION='wirecloud_instance.wsgi:allInOneApplication'
fi

# allow the container to be started with `--user`
if [ "$(id -u)" = '0' ]; then
exec gosu wirecloud /usr/local/bin/gunicorn wirecloud_instance.wsgi:application \
exec gosu wirecloud /usr/local/bin/gunicorn $WSGI_APPLICATION \
--forwarded-allow-ips "${FORWARDED_ALLOW_IPS}" \
--workers ${WORKERS} \
--threads ${THREADS} \
--bind 0.0.0.0:8000 \
--log-file - \
--logger-class wirecloud.glogger.GunicornLogger
else
exec /usr/local/bin/gunicorn wirecloud_instance.wsgi:application \
exec /usr/local/bin/gunicorn $WSGI_APPLICATION \
--forwarded-allow-ips "${FORWARDED_ALLOW_IPS}" \
--workers ${WORKERS} \
--threads ${THREADS} \
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,3 +30,12 @@
# Apply WSGI middleware here.
# from helloworld.wsgi import HelloWorldApplication
# application = HelloWorldApplication(application)

try:
from whitenoise import WhiteNoise
allInOneApplication = WhiteNoise(application, root = "/var/www/static", prefix="static/", autorefresh=True)
except ImportError:
# Whitenoise is not installed
# Only error if wgsi attempts to access allInOneApplication
def allInOneApplication(environ, start_response):
raise Exception("Whitenoise is not installed, wirecloud cannot run in allInOne mode")
Loading