-
Notifications
You must be signed in to change notification settings - Fork 137
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: build CI Docker images with GH Actions #2110
base: master
Are you sure you want to change the base?
feat: build CI Docker images with GH Actions #2110
Conversation
Simple Docker image to manually test the build-and-push Github Action
@meteorcloudy Here's a proposal for a GH Actions workflow that will build the CI images for I would then probably modify one or some of the CI images to have a Also, as I mentioned in #2020 this action can initially run in parallel to the official builds, until you decide if this will be the master way of building them (or not :). Plus, GH Actions also support external workers so maybe it could also be adapted to run against capacity in GCP, if running in GH is a problem (although I think the free minutes should be enough to build the images, they are heavy and there's a bunch of them but I think it will be ok). What do you think? |
1db46b2
to
9df2014
Compare
Workflow to build and deploy Bazel CI Docker images This workflow builds the images, pushes them to the GHCR registry and links them with this repo. For each `Dockerfile` it builds different types of images, each corresponding with a named build stage (`... AS <NAME>`) in the `Dockerfile`. The workflow is triggered when a push to the `main`/`master` or `testing` branch contains changes to one or more of the CI `Dockerfile`s (`buildkite/docker/*/Dockerfile`). It can also be triggered manually via the Actions web UI, the GH REST API or the GH CLI tool, e.g.: ```sh gh workflow run build-ci-docker-images ``` When triggered by a `push` event the workflow will: 1. Determine which `Dockerfile`s were changed. 2. For those `Dockerfile`s, determine its build context (the `Dockerfile` directory) and the build targets (the named build stages in it). 3. Filter the build targets: * first with `RE_TARGET_INCLUDE`, which defaults to empty so it will match all the named build stages. * then with `RE_TARGET_EXCLUDE`: set by default to remove some of the build stages, e.g. the deprecated images like `centos7` and some targets that we don't want to build as images like the `nojdk` ones. 4. Then, it will also exclude all `testimage` targets because that image is only used for manually testing the workflow. 5. Finally, it will spawn a `docker/build-push-action` job for each of the build targets. For every image built, it will push to the registry three image tags: * a `sha` tag with the short hash of the commit that triggered the push * a `date` tag with the current date in ISO format (`YYYYMMDD`) * a `latest` tag When triggered manually (`workflow_dispatch` event) the workflow will default to "running in test mode": it will follow the same steps as a `push` run but with different default values (see `workflow_dispatch.inputs`): * `RE_TARGET_INCLUDE` set to `testimage` (a very simple image to exercise the build that doesn't take much compute) and * `RE_TARGET_EXCLUDE` set to the same pattern as in the `push` event. This effectively limits the build targets to only those in `testimage` not excluded by `RE_TARGET_EXCLUDE` (e.g. `nojdk`). The "test run" also limits the `PLATFORMS` to `linux/amd64`, to further reduce the cost and time of a test run. Finally, it will build those `testimage` targets but **it won't tag `latest` or push any of the image tags to the registry**. This "test mode" behavior can be changed by setting the `workflow_dispatch.inputs` variables: `RE_TARGET_EXCLUDE`, `RE_TARGET_INCLUDE`, `PLATFORMS`, `TAG_DATE`, `TAG_LATEST` and `PUSH`, e.g.: ```sh gh workflow run build-ci-docker-images \ -f RE_TARGET_INCLUDE=ubuntu2404 -f TAG_DATE=20241101 ```
9df2014
to
800cac6
Compare
Fixes #2020
Workflow to build and deploy Bazel CI Docker images
This workflow builds the images, pushes them to the GHCR registry and links them with this repo.
For each
Dockerfile
it builds different types of images, each corresponding with a named build stage (... AS <NAME>
) in theDockerfile
.The workflow is triggered when a push to the
main
/master
ortesting
branch contains changes to one or more of the CIDockerfile
s (buildkite/docker/*/Dockerfile
). It can also be triggered manually via the Actions web UI, the GH REST API or the GH CLI tool, e.g.:When triggered by a
push
event the workflow will:Dockerfile
s were changed.Dockerfile
s, determine its build context (theDockerfile
directory) and the build targets (the named build stages in it).RE_TARGET_INCLUDE
, which defaults to empty so it will match all the named build stages.RE_TARGET_EXCLUDE
: set by default to remove some of the build stages, e.g. the deprecated images likecentos7
and some targets that we don't want to build as images like thenojdk
ones.Then, it will also exclude all
testimage
targets because that image is only used for manually testing the workflow.Finally, it will spawn a
docker/build-push-action
job for each ofthe build targets. For every image built, it will push to the
registry three image tags:
sha
tag with the short hash of the commit that triggered thepush
date
tag with the current date in ISO format (YYYYMMDD
)latest
tagWhen triggered manually (
workflow_dispatch
event) the workflow will default to "running in test mode": it will follow the same steps as apush
run but with different default values (seeworkflow_dispatch.inputs
):RE_TARGET_INCLUDE
set totestimage
(a very simple image to exercise the build that doesn't take much compute) andRE_TARGET_EXCLUDE
set to the same pattern as in thepush
event.This effectively limits the build targets to only those in
testimage
not excluded byRE_TARGET_EXCLUDE
(e.g.nojdk
).The "test run" also limits the
PLATFORMS
tolinux/amd64
, to further reduce the cost and time of a test run.Finally, it will build those
testimage
targets but it won't taglatest
or push any of the image tags to the registry.This "test mode" behavior can be changed by setting the
workflow_dispatch.inputs
variables:RE_TARGET_EXCLUDE
,RE_TARGET_INCLUDE
,PLATFORMS
,TAG_DATE
,TAG_LATEST
andPUSH
, e.g.: