Make the images available in ghcr.io #37

glehmann · 2025-07-24T15:14:21Z

Based on #35

I had to move the user logic to the image entrypoint to make the images usable by any user even if he/she is not using the usual 1000:1000 ids.

The logic for the repository creation was moved to the Dockerfiles to make the images buildable with the standard github action docker/build-push-action.

Hopefully it will help with the issue with the repository selection in #35.

psafont

🎉🎉

(I've only reviewed the last 3 commits)

glehmann · 2025-07-25T11:30:57Z

It works with docker but not with podman: we can't run with --userns=keep-id to be root at the container startup, but we need --userns=keep-id to keep the ownership of the files in the shared volume.

I'll propose something else.

glehmann · 2025-07-25T13:58:34Z

It should be good now.

I've used podman's option --userns=keep-id:uid=1000,gid=1000, to map the builder user to but I'm not sure if that's an option that appeared recently or not.
@ydirson could you test it with your podman version?

glehmann · 2025-07-25T14:00:10Z

I've also added 88d5bec to reduce the image sizes.

ydirson · 2025-07-30T11:48:11Z

I've also added 88d5bec to reduce the image sizes.

There is an idiom to squash everything into a single layer, but I also see docker build --squash.

Dockerfile-7.x

Dockerfile-8.x

Dockerfile-7.x

ydirson · 2025-07-30T13:31:09Z

run.py

@@ -102,11 +102,13 @@ def main():
    docker_arch = args.platform or ("linux/amd64/v2" if branch == "9.0" else "linux/amd64")

    docker_args = [RUNNER, "run", "-i", "-t",
-                   "-u", "builder",


I see that with Docker we delegate to the entrypoint the change of user, but I see nothing for the podman case?

podman, with --userns=keep-id, already starts with the user ids of the user launching the command.
podman does the mapping of the ids in the mounted volumes, so the files are created with the right owner/group.
docker doesn't do that mapping in the mounted volumes, so we have to switch the user in the container to the ids of the users launching the command

we have to switch the user in the container to the ids of the users launching the command

isn't that only required when docker runs as root (which is both the default because of the daemon, and bad practice to start with, and the essential reason I'm suggesting we just drop its support)?

Yes it is. But many dev are still using docker, so I don't think it's time yet to drop docker support

files/entrypoint.sh

build.sh

ydirson · 2025-07-30T13:52:34Z

.github/workflows/docker.yml

+  #         platforms: |
+  #           linux/amd64/v2


Explicit platform for 8.x likely does not hurt, if only for documentation purposes

but for 8.x, the arch is linux/amd64

indeed, fixed!

Dockerfile-9.x

ydirson · 2025-07-31T09:49:32Z

.github/workflows/docker.yml

+          context: .
+          file: ./Dockerfile-8.x
+          push: true
+          tags: ghcr.io/${{ github.repository }}:8.2


We likely want those official floating tags to be set only when run on master, maybe we set particular tags for PRs?
Also, timestamped tags as is common may be interesting to have.

The workflow is only configured on master.
Other tags may be useful, as well as building for PRs, but we must consider cleaning up the old images.

The workflow is only configured on master.

Actually it seems to be configured for main instead :)

Also, there would be a reason for allowing it to run not just on master: detecting pipeline errors before they reach master

yes, that would be nice. Maybe push to the registry when on the master branch then.

ydirson · 2025-08-01T15:31:54Z

run.py

-        docker_args += ["--userns=keep-id", "--security-opt", "label=disable"]
+        # With podman we use the `--userns` option to map the builder user to
+        # the user on the system.
+        docker_args += [f"--userns=keep-id:uid=1000,gid=1000", "--security-opt", "label=disable"]


The fact the uid is preserved so entrypoint is entered as builder and not as root as in the Docker case is likely useful to document. But is it really a good idea to have such a big difference between the 2 runners? (especially as we want to move more root actions into the entrypoint in the future)

ydirson · 2025-08-01T15:32:47Z

run.py

+        # With docker, we modify the builder user in the entrypoint to match the
+        # uid:gid of the user launching the container, and then continue with
+        # the builder user thanks to gosu.
+        docker_args += ["-e", f'BUILDER_UID={os.getuid()}', "-e", f'BUILDER_GID={os.getgid()}']


a reader might ask "why gosu and and just su"

Then he/she will quickly find on the internet why :-)

I usually try to only document what is specific to the project.
Do you think there is something specific about gosu in that context that should be commented?

Signed-off-by: Gaëtan Lehmann <[email protected]>

This way the image is usable by everybody, independantly of its ids. With docker, we modify the builder user in the entrypoint to match the uid:gid of the user launching the container, and then continue with the builder user thanks to gosu. With podman we use the `--userns` option to map the builder user to the user on the system. I haven't found a way with podman to use the same mechanism as in docker, and vis versa. Signed-off-by: Gaëtan Lehmann <[email protected]>

The images are always built, but only pushed to the registry when running on the master branch. Signed-off-by: Gaëtan Lehmann <[email protected]>

for a significantly smaller image size. Before: ghcr.io/xcp-ng/xcp-ng-build-env 8.2 e985e704c252 10 minutes ago 1.03 GB ghcr.io/xcp-ng/xcp-ng-build-env 8.3 b557dcb6d541 11 minutes ago 1.14 GB ghcr.io/xcp-ng/xcp-ng-build-env 9.0 c738df7d5577 11 minutes ago 857 MB After: ghcr.io/xcp-ng/xcp-ng-build-env 8.2 a9a91fa03db8 7 seconds ago 491 MB ghcr.io/xcp-ng/xcp-ng-build-env 8.3 a36b1399ac01 About a minute ago 557 MB ghcr.io/xcp-ng/xcp-ng-build-env 9.0 78011dcbd6f3 11 minutes ago 708 MB Signed-off-by: Gaëtan Lehmann <[email protected]>

glehmann requested review from ydirson and psafont July 24, 2025 15:14

psafont approved these changes Jul 24, 2025

View reviewed changes

glehmann force-pushed the gln/build-env-improvements-lzqx branch 6 times, most recently from 9e73f03 to 87724c2 Compare July 25, 2025 08:14

glehmann force-pushed the gln/build-env-improvements-lzqx branch 3 times, most recently from 78407fa to 7998f39 Compare July 25, 2025 13:57

glehmann force-pushed the gln/build-env-improvements-lzqx branch from 7998f39 to 88d5bec Compare July 25, 2025 14:22

ydirson requested changes Jul 30, 2025

View reviewed changes

ydirson reviewed Jul 31, 2025

View reviewed changes

glehmann force-pushed the gln/build-env-improvements-lzqx branch 2 times, most recently from e6311bc to 248ec8f Compare August 1, 2025 14:12

glehmann marked this pull request as ready for review August 1, 2025 14:12

glehmann requested a review from ydirson August 1, 2025 14:13

ydirson reviewed Aug 1, 2025

View reviewed changes

glehmann force-pushed the gln/build-env-improvements-lzqx branch from 248ec8f to 57683dd Compare August 1, 2025 15:47

glehmann added 4 commits August 1, 2025 19:31

Remove XCP-ng 7 support

9ab1160

Signed-off-by: Gaëtan Lehmann <[email protected]>

Use ghcr.io to store the images

76873f2

The images are always built, but only pushed to the registry when running on the master branch. Signed-off-by: Gaëtan Lehmann <[email protected]>

glehmann force-pushed the gln/build-env-improvements-lzqx branch from 57683dd to e21ff80 Compare August 1, 2025 17:33

Make the images available in ghcr.io #37

Are you sure you want to change the base?

Make the images available in ghcr.io #37

Uh oh!

Conversation

glehmann commented Jul 24, 2025

Uh oh!

psafont left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

glehmann commented Jul 25, 2025

Uh oh!

glehmann commented Jul 25, 2025

Uh oh!

glehmann commented Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ydirson commented Jul 30, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

psafont left a comment •

edited

Loading

glehmann commented Jul 25, 2025 •

edited

Loading