From 5dd5f46c381d219275cdf0110af75bd3a55f6f27 Mon Sep 17 00:00:00 2001 From: accetto <34798830+accetto@users.noreply.github.com> Date: Sun, 12 Mar 2023 18:09:07 +0100 Subject: [PATCH] Release 23.03.1 --- CHANGELOG.md | 4 + README.md | 97 ++++++++++++------------ docker/Dockerfile.xfce.22-04 | 4 +- docker/hooks/build | 3 +- docker/hooks/cache | 4 +- docker/hooks/env.rc | 6 ++ docker/hooks/post_push | 2 +- docker/hooks/pre_build | 3 +- docker/hooks/push | 3 +- docker/hooks/release_of | 6 +- docker/xfce-firefox/README-dockerhub.md | 9 ++- docker/xfce-firefox/README.md | 9 ++- docker/xfce/README.md | 2 +- docker/xfce/src/home/config/dconf/user | Bin 1614 -> 0 bytes examples/Dockerfile.extend | 7 +- readme-builder.md | 82 ++++++++++++++++++-- readme-ci-builder.md | 84 +++++++++++++++++--- readme-g3-cache.md | 9 ++- readme-local-building-example.md | 37 +++++++++ utils/readme-util-readme-examples.md | 6 -- 20 files changed, 280 insertions(+), 97 deletions(-) delete mode 100644 docker/xfce/src/home/config/dconf/user diff --git a/CHANGELOG.md b/CHANGELOG.md index 30b52f1..9c775b3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,10 @@ *** +### Release 23.03.1 + +This is a maintenance release aiming to improve the scripts and documentation. + ### Release 23.03 - updated with `TigerVNC 1.13.1` bugfix release diff --git a/README.md b/README.md index 9e3fb84..7718a9f 100644 --- a/README.md +++ b/README.md @@ -25,34 +25,34 @@ Version: G3v4 - [Headless Ubuntu/Xfce containers with VNC/noVNC](#headless-ubuntuxfce-containers-with-vncnovnc) - [Project `accetto/ubuntu-vnc-xfce-g3`](#project-accettoubuntu-vnc-xfce-g3) - - [Introduction](#introduction) - - [TL;DR](#tldr) - - [Installing packages](#installing-packages) - - [Shared memory size](#shared-memory-size) - - [Extending images](#extending-images) - - [Building images](#building-images) - - [Sharing devices](#sharing-devices) - - [Image generations](#image-generations) - - [Project versions](#project-versions) - - [Project goals](#project-goals) - - [Changes and new features](#changes-and-new-features) - - [Naming scheme](#naming-scheme) - - [Slimmer images](#slimmer-images) - - [Fewer and more flexible Dockerfiles](#fewer-and-more-flexible-dockerfiles) - - [Concept of features](#concept-of-features) - - [Faster building with `g3-cache`](#faster-building-with-g3-cache) - - [Overriding of container user parameters](#overriding-of-container-user-parameters) - - [Overriding of VNC/noVNC parameters](#overriding-of-vncnovnc-parameters) - - [Different use of version sticker](#different-use-of-version-sticker) - - [Image metadata](#image-metadata) - - [Simple self-containing CI](#simple-self-containing-ci) - - [Separated builder and deployment repositories](#separated-builder-and-deployment-repositories) - - [Separate README files for Docker Hub](#separate-readme-files-for-docker-hub) - - [New startup script](#new-startup-script) + - [Introduction](#introduction) + - [TL;DR](#tldr) + - [Installing packages](#installing-packages) + - [Shared memory size](#shared-memory-size) + - [Extending images](#extending-images) + - [Building images](#building-images) + - [Sharing devices](#sharing-devices) + - [Image generations](#image-generations) + - [Project versions](#project-versions) + - [Project goals](#project-goals) + - [Changes and new features](#changes-and-new-features) + - [Naming scheme](#naming-scheme) + - [Slimmer images](#slimmer-images) + - [Fewer and more flexible Dockerfiles](#fewer-and-more-flexible-dockerfiles) + - [Concept of features](#concept-of-features) + - [Faster building with `g3-cache`](#faster-building-with-g3-cache) + - [Overriding of container user parameters](#overriding-of-container-user-parameters) + - [Overriding of VNC/noVNC parameters](#overriding-of-vncnovnc-parameters) + - [Different use of version sticker](#different-use-of-version-sticker) + - [Image metadata](#image-metadata) + - [Simple self-containing CI](#simple-self-containing-ci) + - [Separated builder and deployment repositories](#separated-builder-and-deployment-repositories) + - [Separate README files for Docker Hub](#separate-readme-files-for-docker-hub) + - [New startup script](#new-startup-script) - [Issues, Wiki and Discussions](#issues-wiki-and-discussions) - [Credits](#credits) -### Introduction +## Introduction This repository contains resources for building Docker images based on [Ubuntu 22.04 LTS and 20.04 LTS][docker-ubuntu] with [Xfce][xfce] desktop environment and [VNC][tigervnc]/[noVNC][novnc] servers for headless use. @@ -62,7 +62,7 @@ There are also sibling projects containing images for headless programming ([acc There is also another sibling project [accetto/debian-vnc-xfce-g3][accetto-github-debian-vnc-xfce-g3] containing similar images based on [Debian][docker-debian]. -### TL;DR +## TL;DR There are currently resources for the following Docker images: @@ -75,7 +75,7 @@ There are currently resources for the following Docker images: - [accetto/ubuntu-vnc-xfce-firefox-g3][accetto-docker-ubuntu-vnc-xfce-firefox-g3] - [full Readme][this-readme-image-firefox] -#### Installing packages +### Installing packages I try to keep the images slim. Consequently you can sometimes encounter missing dependencies while adding more applications yourself. You can track the missing libraries on the [Ubuntu Packages Search][ubuntu-packages-search] page and install them subsequently. @@ -88,7 +88,7 @@ sudo apt-get update sudo apt --fix-broken install ``` -#### Shared memory size +### Shared memory size Note that some applications require larger shared memory than the default 64MB. Using 256MB usually solves crashes or strange behavior. @@ -100,7 +100,7 @@ df -h /dev/shm The older sibling Wiki page [Firefox multi-process][that-wiki-firefox-multiprocess] describes several ways, how to increase the shared memory size. -#### Extending images +### Extending images The provided example file `Dockerfile.extend` shows how to use the images as the base for your own images. @@ -108,7 +108,7 @@ Your concrete `Dockerfile` may need more statements, but the concept should be c The compose file `example.yml` shows how to switch to another non-root user and how to set the VNC password and resolution. -#### Building images +### Building images The fastest way to build the images: @@ -142,7 +142,7 @@ The fastest way to build the images: You can still execute the individual hook scripts as before (see the folder `/docker/hooks/`). However, the provided utilities `builder.sh` and `ci-builder.sh` are more convenient. Before pushing the images to the **Docker Hub** you have to prepare and source the file `secrets.rc` (see `example-secrets.rc`). The script `builder.sh` builds the individual images. The script `ci-builder.sh` can build various groups of images or all of them at once. Check the [builder-utility-readme][this-builder-readme], [local-building-example][this-readme-local-building-example] and [Wiki][this-wiki] for more information. -#### Sharing devices +### Sharing devices Sharing the audio device for video with sound works only with `Chromium` and only on Linux: @@ -180,11 +180,11 @@ docker run -it -P --rm \ xhost -local:$(whoami) ``` -### Image generations +## Image generations This is the **third generation** (G3) of my headless images. The **second generation** (G2) contains the GitHub repository [accetto/xubuntu-vnc-novnc][accetto-github-xubuntu-vnc-novnc]. The **first generation** (G1) contains the GitHub repository [accetto/ubuntu-vnc-xfce][accetto-docker-ubuntu-vnc-xfce]. -### Project versions +## Project versions This file describes the **fourth version** (G3v4) of the project. @@ -206,6 +206,7 @@ The version `G3v3` brings the following major changes comparing to the previous - If `FEATURES_FIREFOX_PLUS="0"` and `FEATURES_FIREFOX="1"`, then - image with Firefox will not include the *Firefox Plus features* - image tag will get the `-default` suffix (e.g. `latest-firefox-default` or also `latest-firefox-default-vnc` etc.) +- The images based on `Ubuntu 22.04 LTS` and `Ubuntu 20.04 LTS` (formerly only on `Ubuntu 20.04 LTS`). The version `G3v3` has brought the following major changes comparing to the previous version `G3v2`: @@ -223,7 +224,7 @@ The version `G3v2` has brought the following major changes comparing to the prev The changes affect only the building pipeline, not the Docker images themselves. The `Dockerfile`, apart from using the new local `g3-cache`, stays conceptually unchanged. -### Project goals +## Project goals Unlike the first two generations, this `G3` generation aims to support CI/CD. @@ -241,51 +242,51 @@ None of the above service providers is really required. All images can be built Building process is implemented to minimize image pollution. New images are pushed to the repositories only if something essential has changed. This can be overridden if needed. -### Changes and new features +## Changes and new features **Hint:** More detailed information about new features can be found in [Wiki][this-wiki]. -#### Naming scheme +### Naming scheme Unlike the first two generations, this one will aim to use less Docker Hub **image repositories** with more **image tags**. For example, previously there have been two Docker Hub repositories `xubuntu-vnc` and `ubuntu-vnc-novnc`. Now there will be only one Docker Hub repository `accetto/ubuntu-vnc-xfce-g3`. -#### Slimmer images +### Slimmer images New images are significantly slimmer than the previous ones. It's because that the most of the packages are installed with the `apt-get` switch `--no-install-recommends` by default. This could have consequences, so it is configurable. -#### Fewer and more flexible Dockerfiles +### Fewer and more flexible Dockerfiles Image variations are build from fewer Dockerfiles. This is allowed by using *multi-stage builds* and the [Buildkit][docker-doc-build-with-buildkit]. On the other hand, flexible and configurable Dockerfiles are slightly more complex. -#### Concept of features +### Concept of features Flexibility in Dockerfiles is supported by introducing the concept of **features**. These are variables that control the building process. For example, the variable **FEATURES_BUILD_SLIM** controls the `--no-install-recommends` switch, the variable **FEATURES_NOVNC** controls the inclusion of `noVNC` and so on. Some other available features include, for example, the **FEATURES_SCREENSHOOTING** and **FEATURES_THUMBNAILING** variables. Also the web browsers [Chromium][chromium] and [Firefox][firefox] are defined as features controlled by the variables **FEATURES_CHROMIUM**, **FEATURES_FIREFOX** and **FEATURES_FIREFOX_PLUS**. Selected features that are enabled by default can be explicitly disabled via environment variables. See [readme-local-building-example.md][this-readme-local-building-example] for more information. -#### Faster building with `g3-cache` +### Faster building with `g3-cache` Building performance has been significantly improved by introducing a local cache (`g3-cache`), which contains the external packages that would be otherwise downloaded by each build. Refreshing the cache is part of the building pipeline. The Dockerfiles fall back to the ad-hoc downloading if the local cache is not available. -#### Overriding of container user parameters +### Overriding of container user parameters The user ID, user name, user group ID, user group name and the initial `sudo` password can be overridden during the build time (`docker build`). The user ID and the group ID can be overridden also in run time (`docker run`). The overridden user gets permissions for `sudo`. The `sudo` password can be changed inside the container. -#### Overriding of VNC/noVNC parameters +### Overriding of VNC/noVNC parameters Several ways of overriding the VNC/noVNC parameters are supported. The password, display, resolution, color depth, view mode and the ports can be overridden at the image build-time, the container startup-time and the VNC startup-time. Using of empty VNC/noVNC password is also supported because it is independent from the container user password. If your session disconnects, it might be related to a network equipment (load-balancer, reverse proxy, ...) dropping the websocket session for inactivity (more info [here](https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_read_timeout) and [here](https://nginx.org/en/docs/http/websocket.html) for nginx). In such case, try defining the **NOVNC_HEARTBEAT=XX** environment variable at startup-time, where **XX** is the number of seconds between [websocket ping/pong](https://github.com/websockets/ws/issues/977) packets. -#### Different use of version sticker +### Different use of version sticker The concept of version sticker has been introduced in the second generation and later implemented also in the first generation. Check this [Wiki page](https://github.com/accetto/xubuntu-vnc/wiki/Version-sticker) for more information. However, the usage of the version sticker has been changed in the third generation. Previously it has been used for testing, if there are any newer packages available by following the *try-and-fail* pattern. That was sufficient for human controlled building process, but it became a problem for CI/CD. Therefore it is used differently now. The verbose version sticker is used for minimizing image pollution. The short form of the version sticker is available as an image *label* and a *badge* in the README file. The *version sticker badge* is also linked with the *verbose version sticker gist*, so it is possible to check the actual image configuration even without downloading it. -#### Image metadata +### Image metadata The image metadata are now stored exclusively as image *labels*. The previous environment variables like **REFRESHED_AT** or **VERSION_STICKER** have been removed. Most of the labels are namespaced according the [OCI IMAGE SPEC](https://github.com/opencontainers/image-spec) recommendations. However, the `version-sticker` label is in the namespace `any.accetto` for obvious reasons. -#### Simple self-containing CI +### Simple self-containing CI The **first version** of the third generation (G3v1) implemented a relatively simple self-containing CI by utilizing the **Docker Hub** builder *hooks*. The same build pipeline could be executed also manually if building locally. For example, an image could be refreshed by executing the `/hooks/pre_build` and `/hooks/build` scripts. The script `/hooks/push` would push the image to the deployment repository. The script `/hooks/post_push` would update the *gist* data and trigger the **GitHub Actions** workflow, which would publish the image's README file to the **Docker Hub**. @@ -295,7 +296,7 @@ The **second version** (G3v2) of the building pipeline does not depend on the ** The **local stage** is the default for the new building pipeline. Also a local **GitLab** installation in a Docker container has already been successfully tested as a CI building stage. -#### Separated builder and deployment repositories +### Separated builder and deployment repositories While there is only one GitHub repository, containing the resources for building all the images, the **first pipeline version** (G3v1) have used two kinds of repositories on the **Docker Hub**. A single *builder repository* has been used for building all the images. The final images have then been published into one or more *deployment repositories*. This separation allowed to keep permutations by naming reasonable. Not all repositories had to have the same visibility, they could be private or public as required. @@ -306,7 +307,7 @@ All the images are still build in a single **building repository** and then publ The **builder repository** can also server as a **secondary deployment repository** during development and testing. -#### Separate README files for Docker Hub +### Separate README files for Docker Hub Each **deployment repository** has its own `README` file for the **Docker Hub**. @@ -316,7 +317,7 @@ The **second pipeline version** (G3v2) does not try to publish the `README` file The source `README` files for the **Docker Hub** are split into two parts. The part containing the badge links is separated into a **template file**. The final `README` files are then generated by the utility script. These files are usually shorter, because their length is limited by the **Docker Hub**. Therefore there are also the full-length versions, that are published only on the **GitHub**. -#### New startup script +### New startup script The startup script has been completely redesigned with the help of the [argbash][argbash-doc] tool and the image [accetto/argbash-docker][accetto-docker-argbash-docker]. Several new startup switches has been added. For example, there are startup switches `--wait`, `--skip-startup`, `--tail-null`, `--tail-vnc`, `--version-sticker` and `--version-sticker-verbose`. There are also startup modifiers `--skip-vnc`, `--skip-novnc`, `--debug` and `--verbose`. Also the utility switches `--help-usage`, `--help` and `--version` are available. diff --git a/docker/Dockerfile.xfce.22-04 b/docker/Dockerfile.xfce.22-04 index f3d41e9..6be8d23 100644 --- a/docker/Dockerfile.xfce.22-04 +++ b/docker/Dockerfile.xfce.22-04 @@ -322,9 +322,7 @@ Pin-Priority: 501" \ > /etc/apt/preferences.d/mozilla-firefox \ && apt-get update \ && DEBIAN_FRONTEND=noninteractive apt-get install -y "${ARG_APT_NO_RECOMMENDS:+--no-install-recommends}" \ - firefox \ - && apt-get -y autoremove \ - && rm -rf /var/lib/apt/lists/* + firefox COPY ./xfce-firefox/src/home/Desktop "${HOME}"/Desktop/ diff --git a/docker/hooks/build b/docker/hooks/build index 8152a67..7d2916c 100644 --- a/docker/hooks/build +++ b/docker/hooks/build @@ -1,4 +1,4 @@ -#!/bin/bash -e +#!/bin/bash echo -e "\n==> EXECUTING @$(date -u +'%Y-%m-%d_%H-%M-%S'): ${0} $@\n" @@ -66,7 +66,6 @@ main() { ### no quotes around 'params' docker build ${params} \ -f "${DOCKERFILE_PATH}" \ - --build-arg BUILDKIT_INLINE_CACHE=1 \ --build-arg BASEIMAGE="${BASEIMAGE}" \ --build-arg BASETAG="${BASETAG}" \ --build-arg ARG_CREATED="${now}" \ diff --git a/docker/hooks/cache b/docker/hooks/cache index b42182c..bbb5025 100644 --- a/docker/hooks/cache +++ b/docker/hooks/cache @@ -1,4 +1,5 @@ -#!/bin/bash -e +#!/bin/bash + ### @accetto, September 2022 echo -e "\n==> EXECUTING @$(date -u +'%Y-%m-%d_%H-%M-%S'): ${0} $@\n" @@ -187,7 +188,6 @@ main() { declare _quiet="" declare _mark="\n==> G3-CACHE" -declare _script_release_of="release_of" declare _cache="${_build_context}/.g3-cache" declare _shared_cache_path="${SHARED_G3_CACHE_PATH}" diff --git a/docker/hooks/env.rc b/docker/hooks/env.rc index faf597c..ce82915 100644 --- a/docker/hooks/env.rc +++ b/docker/hooks/env.rc @@ -155,7 +155,13 @@ case "${_branch}" in if [[ "${_blend}" =~ chromium ]] ; then FEATURES_CHROMIUM=1 + CHROMIUM_VERSION=$( "${_mydir}"/${_script_release_of} 'chromium-1804' ) + + if [[ $? -ne 0 ]] ; then + echo "${CHROMIUM_VERSION}" + exit 1 + fi _deploy_repo="${_owner}/${DEPLOYMENT_REPO_CHROMIUM}" _readme_context="docker/xfce-chromium" diff --git a/docker/hooks/post_push b/docker/hooks/post_push index 3821c92..b217317 100644 --- a/docker/hooks/post_push +++ b/docker/hooks/post_push @@ -1,4 +1,4 @@ -#!/bin/bash -e +#!/bin/bash ### 2022-09-11 Warning ### It seems like the README publishing is currently not always working. diff --git a/docker/hooks/pre_build b/docker/hooks/pre_build index 36bd948..2529b36 100644 --- a/docker/hooks/pre_build +++ b/docker/hooks/pre_build @@ -1,4 +1,4 @@ -#!/bin/bash -e +#!/bin/bash echo -e "\n==> EXECUTING @$(date -u +'%Y-%m-%d_%H-%M-%S'): ${0} $@\n" @@ -61,7 +61,6 @@ main() { ### no quotes around 'params' docker build ${params} \ -f "${DOCKERFILE_PATH}" \ - --build-arg BUILDKIT_INLINE_CACHE=1 \ --build-arg BASEIMAGE="${BASEIMAGE}" \ --build-arg BASETAG="${BASETAG}" \ \ diff --git a/docker/hooks/push b/docker/hooks/push index 373ebfd..19c5f03 100644 --- a/docker/hooks/push +++ b/docker/hooks/push @@ -1,4 +1,5 @@ -#!/bin/bash -e +#!/bin/bash + ### @accetto, August 2021 echo -e "\n==> EXECUTING @$(date -u +'%Y-%m-%d_%H-%M-%S'): ${0} $@\n" diff --git a/docker/hooks/release_of b/docker/hooks/release_of index 75367b7..309fbbc 100644 --- a/docker/hooks/release_of +++ b/docker/hooks/release_of @@ -1,4 +1,4 @@ -#!/bin/bash -e +#!/bin/bash main() { local result="" @@ -7,7 +7,7 @@ main() { chromium-1804 ) # result=$(wget -qO- \ - result=$(curl -sL \ + result=$(curl -sLN \ http://archive.ubuntu.com/ubuntu/pool/universe/c/chromium-browser/ \ | grep -Po -m1 '(?<=href=")[^_]*_([0-9.]+-0ubuntu\d+\.18\.04\.[^_"]*)_[^"]*' \ | cut -d _ -f 2 \ @@ -25,7 +25,7 @@ main() { echo "${result}" else - echo "Exiting '${0}' with code '1' because not able to get version of '$1'" + echo "Exiting '${0}' with code '1' because unable to get version of '$1'" return 1 fi } diff --git a/docker/xfce-firefox/README-dockerhub.md b/docker/xfce-firefox/README-dockerhub.md index f6e611a..edec862 100644 --- a/docker/xfce-firefox/README-dockerhub.md +++ b/docker/xfce-firefox/README-dockerhub.md @@ -26,10 +26,12 @@ *** -**Warning** about the images with Firefox +**Remarks** about the images with Firefox There is no single-process Firefox image in this repository any more and the **multi-process mode** is always enabled. Be aware, that the multi-process mode requires larger shared memory (`/dev/shm`). At least 256MB is recommended. Please check the **Firefox multi-process** page in this older sibling [Wiki][that-wiki-firefox-multiprocess] for more information and the instructions, how to set the shared memory size in different scenarios. +The [Firefox][firefox] installed in the images based on `Ubuntu 22.04 LTS` is the current non-snap version from the Mozilla Team PPA. It's because the `Ubuntu 22.04 LTS` distribution contains only the `snap` version and the `snap` is currently not working correctly in Docker containers. + *** ### Introduction @@ -142,7 +144,10 @@ The main features and components of the images in the default configuration are: - current version of [tini][tini] as the entry-point initial process (PID 1) - support for overriding both the container user and the group - support of **version sticker** (see the [full-length README][this-readme-full] on the **GitHub**) -- current version of [Firefox][firefox] web browser and some additional **Firefox plus features** described in the [full-length README][this-readme-full] on the GitHub +- [Firefox][firefox] web browser + - images based on `Ubuntu 20.04 LTS` - current version from the `Ubuntu 20.04 LTS` distribution + - images based on `Ubuntu 22.04 LTS` - current non-snap version from the Mozilla Team PPA +- additional **Firefox plus features** described in the [full-length README][this-readme-full] on the **GitHub** The history of notable changes is documented in the [CHANGELOG][this-changelog]. diff --git a/docker/xfce-firefox/README.md b/docker/xfce-firefox/README.md index 8a05ef5..0cd5e4e 100644 --- a/docker/xfce-firefox/README.md +++ b/docker/xfce-firefox/README.md @@ -46,10 +46,12 @@ *** -**Warning** about the images with Firefox +**Remarks** about the images with Firefox There is no single-process Firefox image in this repository any more and the **multi-process mode** is always enabled. Be aware, that the multi-process mode requires larger shared memory (`/dev/shm`). At least 256MB is recommended. Please check the **Firefox multi-process** page in this older sibling [Wiki][that-wiki-firefox-multiprocess] for more information and the instructions, how to set the shared memory size in different scenarios. +The [Firefox][firefox] installed in the images based on `Ubuntu 22.04 LTS` is the current non-snap version from the Mozilla Team PPA. It's because the `Ubuntu 22.04 LTS` distribution contains only the `snap` version and the `snap` is currently not working correctly in Docker containers. + *** ### Introduction @@ -162,7 +164,10 @@ The main features and components of the images in the default configuration are: - current version of [tini][tini] as the entry-point initial process (PID 1) - support for overriding both the container user and the group - support of **version sticker** (see below) -- current version of [Firefox][firefox] web browser and some additional **Firefox plus features** described below +- [Firefox][firefox] web browser + - images based on `Ubuntu 20.04 LTS` - current version from the `Ubuntu 20.04 LTS` distribution + - images based on `Ubuntu 22.04 LTS` - current non-snap version from the Mozilla Team PPA +- additional **Firefox plus features** described below The history of notable changes is documented in the [CHANGELOG][this-changelog]. diff --git a/docker/xfce/README.md b/docker/xfce/README.md index 502f5ae..2a706c4 100644 --- a/docker/xfce/README.md +++ b/docker/xfce/README.md @@ -92,7 +92,7 @@ The fastest way to build the images: ### PWD = project root ### prepare and source the 'secrets.rc' file first (see 'example-secrets.rc') -### examples of building and publishing the individual images +### examples of building and publishing the individual images ./builder.sh latest all ### just building the image, skipping the publishing and the version sticker update diff --git a/docker/xfce/src/home/config/dconf/user b/docker/xfce/src/home/config/dconf/user deleted file mode 100644 index 10714a98d8dd205e91262b944202d740dedbbf3c..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 1614 zcmZ9MU1%It7>1{5vrU?9yKPO{+D0Q43<<*qdeKm-6bz_T%izt7r`5=MWUb(@TS-sFGMQsML@(KREkv)3hMLDPSVhUhxeU#&z?Ew z`)1B#&wGjWqRc6-1L)G${APTt$of#BxF5D}VgNq~I|_!-GVFw1u=_45-0y*X1;gkF z+zW@`FpR=j!RoE2YZ{|94LS49CG;{ZJ^SlHJyX8ldD8g|{Q?$`y}noD<;#q}f?kE- z)t4*E<@b=Up+CUIiCn&)d|(^dE`f!XMh)G2;O^t^gHG9 zdhQr{1Xj*`@S*l4zk_@VJqiooPnI;ly!tzho`V-^rIVURUhBMozODSHhxRF#-$lNR zURM6el?mnY>dPwnHGIB!`;>Bd_2n9R9nL;5bzFUs*ZeopTX6f)@>PwOSN$Cpy%lc! zb@G^UdF2&!1V$(OCRBg<66-&Vj=`5lT;8*u*Z3*26Y$cxi_1vvOXQDpa~ix(7P~g8 zn~hJ@UziO{RdctzEC^jc@dA@Nj-Q5c*KOL!xF)LGs6nR7Pi!O1+?JO#Y~;4%)Mhr0 zYWm4`Z9NO+ob&p?xsU2`w~D<7?gxF(Y}x58!QJ?S-U-JRX&kwk=QB$0h@WO&)=7OY zap!Gn{iZ4MIB8V*UF%-3Eqm6~^@HQL;!bMXUcIV*>)UE4X4WJoVp~>x|P`cc)=AXRdf0>)` are passed to the Docker CLI commands used internally depending on the usage mode (see below). -## Prerequisites +## Preparation -Before building and publishing the images prepare and source a file containing the necessary environment variables. You can use the provided file `example-secrets.rc` as a template. +### Ensure file attributes after cloning -If you name your file `secrets.rc` and you store it into the folder `docker/hooks/`, then it will sourced automatically by the hook script `env.rc`. +It may be necessary to repair the executable files attributes after cloning the repository (by `git clone`). -Otherwise you can source it in the terminal manually, for example: +You can do that by executing the following commands from the project's root directory: ```shell -source secrets.rc +find . -type f -name "*.sh" -exec chmod +x '{}' \; +chmod +x docker/hooks/* +``` + +For example, if the files in the folder `docker/hooks` would not be executable, then you would get errors similar to this: + +```shell +$ ./builder.sh latest build + +==> EXECUTING @2023-03-05_16-42-57: ./builder.sh + +./builder.sh: line 84: ./docker/hooks/build: Permission denied +``` + +### Set environment variables before building + +Open a terminal windows and change the current directory to the root of the project (where the license file is). + +Make a copy of the secrets example file, modify it and then source it in the terminal: + +```shell +### make a copy and then modify it +cp examples/example-secrets.rc secrets.rc + +### source the secrets +source ./secrets.rc ### or also -. secrets.rc +. ./secrets.rc ``` +**TIP**: If you copy a file named `secrets.rc` into the folder `docker/hooks/`, then it will be automatically sourced by the hook script `env.rc`. + +Be aware that the following environment variables are mandatory and must be always set: + +- `REPO_OWNER_NAME` +- `BUILDER_REPO` + +Ensure that your `secrets.rc` file contains at least the lines similar to these: + +```shell +export REPO_OWNER_NAME="accetto" +export BUILDER_REPO="headless-ubuntu-g3" +``` + +You can use your own names if you wish. + +Alternatively you can modify the hook script file env.rc like this: + +```shell +### original lines +declare _owner="${REPO_OWNER_NAME:?Need repo owner name}" +DOCKER_REPO="${_owner}/${BUILDER_REPO:?Need builder repo name}" + +### modified lines +declare _owner="${REPO_OWNER_NAME:-accetto}" +DOCKER_REPO="${_owner}/${BUILDER_REPO:-headless-ubuntu-g3}" +``` + +Again, you can use your own names if you wish. + +You can also use other ways to set the variables. + +### Ensure `wget` utility + +If you are on Windows, you can encounter the problem of missing `wget` utility. It is used by refreshing the `g3-cache` and it's available on Linux by default. + +On Windows you have generally two choices. You can build your images inside the `WSL` environment or you can download the `wget.exe` application for Windows. Make sure to update also the `PATH` environment variable appropriately. + ## Executing complete pipeline Building the individual images and publishing them to the **Docker Hub**: @@ -86,7 +152,7 @@ You can also provide additional parameters for the internally used Docker `build ### docker build --no-cache ... ``` -The optional `` are passed only to the `pre_build` hook script, which passes them to the internally used `docker build` command. +The optional `` are passed only to the `pre_build` hook script, which passes them to the internally used `docker build` command. The `cache` hook script, however, doesn't use any Docker CLI commands. ## Executing individual pipeline steps diff --git a/readme-ci-builder.md b/readme-ci-builder.md index 80149fb..a4ed131 100644 --- a/readme-ci-builder.md +++ b/readme-ci-builder.md @@ -2,7 +2,10 @@ - [Utility `ci-builder.sh`](#utility-ci-buildersh) - [Introduction](#introduction) - - [Prerequisites](#prerequisites) + - [Preparation](#preparation) + - [Ensure file attributes after cloning](#ensure-file-attributes-after-cloning) + - [Set environment variables before building](#set-environment-variables-before-building) + - [Ensure `wget` utility](#ensure-wget-utility) - [Usage modes](#usage-modes) - [Group mode](#group-mode) - [Group mode examples](#group-mode-examples) @@ -10,7 +13,7 @@ - [Family mode examples](#family-mode-examples) - [Log processing](#log-processing) - [Digest command](#digest-command) - - [Stickers commands](#stickers-commands) + - [Stickers command](#stickers-command) - [Timing command](#timing-command) - [Errors command](#errors-command) - [Additional building parameters](#additional-building-parameters) @@ -69,22 +72,85 @@ The optional parameter `--no-cache` will be passed to the internally used script The optional parameter `--log-all` will cause that the script's output will be written into the log file in all cases. Normally the command line errors or the **log processing mode** commands are not logged. -## Prerequisites +## Preparation -Before building and publishing the images prepare and source a file containing the necessary environment variables. You can use the provided file `example-secrets.rc` as a template. +### Ensure file attributes after cloning -If you name your file `secrets.rc` and you store it into the folder `docker/hooks/`, then it will sourced automatically by the hook script `env.rc`. +It may be necessary to repair the executable files attributes after cloning the repository (by `git clone`). -Otherwise you can source it in the terminal manually, for example: +You can do that by executing the following commands from the project's root directory: ```shell -source secrets.rc +find . -type f -name "*.sh" -exec chmod +x '{}' \; +chmod +x docker/hooks/* +``` + +For example, if the files in the folder `docker/hooks` would not be executable, then you would get errors similar to this: + +```shell +$ ./builder.sh latest build + +==> EXECUTING @2023-03-05_16-42-57: ./builder.sh + +./builder.sh: line 84: ./docker/hooks/build: Permission denied +``` + +### Set environment variables before building + +Open a terminal windows and change the current directory to the root of the project (where the license file is). + +Make a copy of the secrets example file, modify it and then source it in the terminal: + +```shell +### make a copy and then modify it +cp examples/example-secrets.rc secrets.rc + +### source the secrets +source ./secrets.rc ### or also -. secrets.rc +. ./secrets.rc ``` +**TIP**: If you copy a file named `secrets.rc` into the folder `docker/hooks/`, then it will be automatically sourced by the hook script `env.rc`. + +Be aware that the following environment variables are mandatory and must be always set: + +- `REPO_OWNER_NAME` +- `BUILDER_REPO` + +Ensure that your `secrets.rc` file contains at least the lines similar to these: + +```shell +export REPO_OWNER_NAME="accetto" +export BUILDER_REPO="headless-ubuntu-g3" +``` + +You can use your own names if you wish. + +Alternatively you can modify the hook script file env.rc like this: + +```shell +### original lines +declare _owner="${REPO_OWNER_NAME:?Need repo owner name}" +DOCKER_REPO="${_owner}/${BUILDER_REPO:?Need builder repo name}" + +### modified lines +declare _owner="${REPO_OWNER_NAME:-accetto}" +DOCKER_REPO="${_owner}/${BUILDER_REPO:-headless-ubuntu-g3}" +``` + +Again, you can use your own names if you wish. + +You can also use other ways to set the variables. + +### Ensure `wget` utility + +If you are on Windows, you can encounter the problem of missing `wget` utility. It is used by refreshing the `g3-cache` and it's available on Linux by default. + +On Windows you have generally two choices. You can build your images inside the `WSL` environment or you can download the `wget.exe` application for Windows. Make sure to update also the `PATH` environment variable appropriately. + ## Usage modes ### Group mode @@ -203,7 +269,7 @@ No build needed for 'headless-ubuntu-g3:latest-chromium'. No build needed for 'headless-ubuntu-g3:latest-firefox'. ``` -#### Stickers commands +#### Stickers command The `stickers` command extracts the information about the **version stickers** of the ephemeral helper images that have been built by the `pre_build` hook script. That does not mean that the final persistent images have also been built (and optionally also published). diff --git a/readme-g3-cache.md b/readme-g3-cache.md index a0ebad2..13869d5 100644 --- a/readme-g3-cache.md +++ b/readme-g3-cache.md @@ -2,6 +2,7 @@ - [Custom `g3-cache`](#custom-g3-cache) - [Introduction](#introduction) + - [Ensure `wget` utility](#ensure-wget-utility) - [Local `g3-cache`](#local-g3-cache) - [Shared g3-cache](#shared-g3-cache) - [Helper script `cache`](#helper-script-cache) @@ -12,12 +13,18 @@ The custom `g3-cache` has been introduced in the **second version** (G3v2) of th The local `g3-cache` is an additional cache used by the building pipeline and it should not be confused with the **Docker builder cache** maintained by the [Docker Build][docker-doc-docker-build] itself. -The `g3-cache`stores the selected pre-downloaded packages used by the Dockerfiles, that would be otherwise repeatedly downloaded from the external sources by each build. +The `g3-cache` stores the selected pre-downloaded packages used by the Dockerfiles, that would be otherwise repeatedly downloaded from the external sources by each build. It results in a significantly higher performance by building sets of images or by repeated builds. You can learn more about the concept on the Wiki page ["Concepts of `g3-cache`"][this-wiki-concepts-of-g3-cache] and about the implementation on the Wiki page ["How `g3-cache` works"][this-wiki-how-g3-cache-works]. +### Ensure `wget` utility + +If you are on Windows, you can encounter the problem of missing `wget` utility. It is used by refreshing the `g3-cache` and it's available on Linux by default. + +On Windows you have generally two choices. You can build your images inside the `WSL` environment or you can download the `wget.exe` application for Windows. Make sure to update also the `PATH` environment variable appropriately. + ## Local `g3-cache` The local `g3-cache` of this project has the following **cache sections**: diff --git a/readme-local-building-example.md b/readme-local-building-example.md index 26021fb..18a5d78 100644 --- a/readme-local-building-example.md +++ b/readme-local-building-example.md @@ -5,6 +5,7 @@ - [Preparation](#preparation) - [Ensure file attributes after cloning](#ensure-file-attributes-after-cloning) - [Set environment variables before building](#set-environment-variables-before-building) + - [Ensure `wget` utility](#ensure-wget-utility) - [Building pipeline](#building-pipeline) - [Three ways of building images](#three-ways-of-building-images) - [Building and publishing sets of images](#building-and-publishing-sets-of-images) @@ -72,6 +73,42 @@ source ./secrets.rc **TIP**: If you copy a file named `secrets.rc` into the folder `docker/hooks/`, then it will be automatically sourced by the hook script `env.rc`. +Be aware that the following environment variables are mandatory and must be always set: + +- `REPO_OWNER_NAME` +- `BUILDER_REPO` + +Ensure that your `secrets.rc` file contains at least the lines similar to these: + +```shell +export REPO_OWNER_NAME="accetto" +export BUILDER_REPO="headless-ubuntu-g3" +``` + +You can use your own names if you wish. + +Alternatively you can modify the hook script file env.rc like this: + +```shell +### original lines +declare _owner="${REPO_OWNER_NAME:?Need repo owner name}" +DOCKER_REPO="${_owner}/${BUILDER_REPO:?Need builder repo name}" + +### modified lines +declare _owner="${REPO_OWNER_NAME:-accetto}" +DOCKER_REPO="${_owner}/${BUILDER_REPO:-headless-ubuntu-g3}" +``` + +Again, you can use your own names if you wish. + +You can also use other ways to set the variables. + +### Ensure `wget` utility + +If you are on Windows, you can encounter the problem of missing `wget` utility. It is used by refreshing the `g3-cache` and it's available on Linux by default. + +On Windows you have generally two choices. You can build your images inside the `WSL` environment or you can download the `wget.exe` application for Windows. Make sure to update also the `PATH` environment variable appropriately. + ## Building pipeline The actual building pipeline consists of the following hook scripts stored in the folder `docker/hooks`: diff --git a/utils/readme-util-readme-examples.md b/utils/readme-util-readme-examples.md index 8112ab0..625312b 100644 --- a/utils/readme-util-readme-examples.md +++ b/utils/readme-util-readme-examples.md @@ -1,11 +1,5 @@ # Examples `util-readme.sh` -**Remark** - -The previously contained `publish` command has been removed, because it was not working properly any more. Some changes on the **Docker Hub** side seemed to be the reason. However, you can always copy-and-pase the content generated by the `preview` command (the file `scrap-readme.md`) to the **Docker Hub** manually. - -*** - ## Preparation Open a terminal window and change the current directory to `utils/`.