From 8b1a3d96585616821d189027260a5185b27e44b9 Mon Sep 17 00:00:00 2001 From: classabbyamp Date: Mon, 10 Jul 2023 15:27:20 -0400 Subject: [PATCH] docs/src/upkeep/images: document github CI process, switch to minisign --- docs/src/upkeep/images.md | 244 +++++++++----------------------------- 1 file changed, 54 insertions(+), 190 deletions(-) diff --git a/docs/src/upkeep/images.md b/docs/src/upkeep/images.md index 1411ff7d..d5b0b38f 100644 --- a/docs/src/upkeep/images.md +++ b/docs/src/upkeep/images.md @@ -1,229 +1,93 @@ # Distributable Images -Void prepares and distributes multiple live images. These are -prepared manually due to the need for full root authority during -build, and for the need to sign them after building. - -## Prepare the Environment - -The images must be built from the root mirror in order to be done in a -reasonable amount of time. This is constrained by network bandwidth, -available compute power, and security requirements for signing the -images. - -First stop all non-essential services. This includes the buildmaster, -local buildslaves, and if the redis server is consuming too much -memory then that too. - -``` -# sv down void-buildmaster -# sv down void-buildslave-armv6l -# sv down void-buildslave-armv7l -# sv down void-buildslave-i686 -# sv down void-buildslave-x86_64 -``` - -Now install the required extra packages: - -``` -# xbps-install qemu-user-static xz git lz4 liblz4 -``` - -Obtain the contents of mklive: - -``` -$ git clone https://github.com/void-linux/void-mklive.git -``` - -Perform the initial make for templating: - -``` -$ cd void-mklive -$ make -``` - -Now the contents of lib.sh should be edited to pull the packages -directly from disk rather than via http since the root mirror is -available locally. - -Locate the `XBPS_REPOSITORY` declaration at the bottom of the file and -change the references from `http://alpha.de.repo.voidlinux.org` to -`/hostdir/binpkgs`. +Void prepares and distributes multiple live images. These are prepared manually +due to the need for full root authority during build, and for the need to sign +them after building. ## Building the Images -Building the images takes several hours with large gaps of time in -between. Make sure you have time to finish the build before starting. - -*!! Warning !!* Beware of UTC midnight while building. The -intermediate stages are all keyed with a datecode, and if the build -crosses midnight UTC you'll suddenly find multiple copies of rootfs's -being built. Since the mirror is locked this shouldn't be a problem, -but its still best to try and avoid it. - -### Building the root filesystems. - -All the rest of the builds depend on the availability of the root -filesystems for each architecture. This process should be run first -and usually if something is going to go wrong it will go wrong here. - -Verify that the root filesystem list is correct: +The images should be built using Github CI in the void-mklive repository. This +can be triggered [on +Github](https://github.com/void-linux/void-mklive/actions/workflows/gen-images.yml) +or by using the `release.sh` script in void-mklive: ``` -$ make rootfs-all-print -void-i686-ROOTFS-20190217.tar.xz -void-x86_64-ROOTFS-20190217.tar.xz -void-x86_64-musl-ROOTFS-20190217.tar.xz -void-armv6l-ROOTFS-20190217.tar.xz -void-armv6l-musl-ROOTFS-20190217.tar.xz -void-armv7l-ROOTFS-20190217.tar.xz -void-armv7l-musl-ROOTFS-20190217.tar.xz -void-aarch64-ROOTFS-20190217.tar.xz -void-aarch64-musl-ROOTFS-20190217.tar.xz +$ ./release.sh start ``` -Your list should match with the exception of dates. If it does not, -either the makefile or the docs are out of date, figure out which one -and send a patch. +By default, this will build: -Now you are ready to build the root filesystems: +- Live ISOs with `base` and `xfce` variants for `x86_64*` and `i686` +- ROOTFSes for `x86_64*`, `i686`, `aarch64*`, `armv7l*`, and `armv6l*` +- PLATFORMFSes for `aarch64*`, `armv7l*`, and `armv6l*` Raspberry Pis +- SBC images for `aarch64*`, `armv7l*`, and `armv6l*` Raspberry Pis -``` -$ sudo make rootfs-all -``` - -Make here should be called with sudo to ensure that sudo doesn't time -out inside of make. - -### Building the platform filesystems. +This will take approximately 2 hours for the default settings. To ensure all +images have the same datecode, the datecode is cached at the beginning of the +run. The CI workflow will also generate `sha256sum.txt` for the built images. -Once the root filesystems have built, they can be specialized into -platform filesystems. These have system specific adaptations added to -them which make them suitable for installation targets. - -Verify that the platformfs list is correct: - -``` -$ make platformfs-all-print -void-rpi-PLATFORMFS-20190217.tar.xz -void-rpi-musl-PLATFORMFS-20190217.tar.xz -void-rpi2-PLATFORMFS-20190217.tar.xz -void-rpi2-musl-PLATFORMFS-20190217.tar.xz -void-rpi3-PLATFORMFS-20190217.tar.xz -void-rpi3-musl-PLATFORMFS-20190217.tar.xz -void-beaglebone-PLATFORMFS-20190217.tar.xz -void-beaglebone-musl-PLATFORMFS-20190217.tar.xz -void-cubieboard2-PLATFORMFS-20190217.tar.xz -void-cubieboard2-musl-PLATFORMFS-20190217.tar.xz -void-odroid-c2-PLATFORMFS-20190217.tar.xz -void-odroid-c2-musl-PLATFORMFS-20190217.tar.xz -void-usbarmory-PLATFORMFS-20190217.tar.xz -void-usbarmory-musl-PLATFORMFS-20190217.tar.xz -void-GCP-PLATFORMFS-20190217.tar.xz -void-GCP-musl-PLATFORMFS-20190217.tar.xz -``` - -As above your list should match with the exception of the datecodes. -If it does not match, it is likely this list is out of date and -another platform has been added or one of the old ones has been -dropped. Send a patch to update this page. - -As above call make with sudo to build the filesystems: - -``` -$ sudo make platformfs-all -``` - -### Building the PXE images. - -The PXE images are a specialized set of images for x86_64 and -x86_64-musl that are intended to allow running Void at large scale. -These images can be booted over the network to support installing Void -on a large number of machines at once. - - -As before, confirm the correct images will be built: - -``` -$ make pxe-all-print -void-x86_64-NETBOOT-20190217.tar.gz -void-x86_64-musl-NETBOOT-20190217.tar.gz -``` - -And finally build the images: - -``` -$ sudo make pxe-all -``` - -### Building the ISO images. - -The ISO images do not follow the same build process as the rest of the -system. To build these you should use the `build-x86-images.sh` -script which does a one-shot build of all the images. +## Collecting the Images -First build the x86_64 images: +Once all images are built, they need to be collected from the Github CI +artifacts. This can be done via the Github CI web interface, on the "Summary" +tab of the CI run, or void-mklive's `release.sh` can download them to a +directory called `void-live-` with: ``` -$ sudo ./build-x86-images.sh -r /hostdir/binpkgs +$ ./release.sh dl ``` -Then build the musl equivalents: - -``` -$ sudo ./build-x86_64-images.sh -a x86_64musl -r /hostdir/binpkgs/musl -``` +> Note: this currently assumes latest successful CI run is the one to download. -And finally the x86 images: +Once downloaded, verify all sha256sums match: ``` -$ sudo ./build-x86-images.sh -a i686 -r /hostdir/binpkgs +$ cd void-live- +$ sha256sum -C * ``` -## Collecting the Images +The images can then be uploaded to [DevSpace](/services/devspace.md) or the +mirrors for testing. -Once all images are built, they need to be collected into a directory, -signed, and moved into place. +## Signing the Images -``` -$ mkdir -$ mv void** / -``` +Signing the images is done after all the images have been checked and validated, +and after the decision has been made to promote the set to `current`. -Generate the sha256sums: +Generate a new signing key: ``` -$ cd -$ sha256 * > sha256.txt +$ export DATECODE= +$ pwgen -cny 25 1 > void-release-$DATECODE.key +$ minisign -G -p void-release-$DATECODE.pub -s void-release-$DATECODE.key \ + -c "This key is only valid for images with date $DATECODE." ``` -## Signing the Images +Copy the public half of this key to the `void-release-keys` package in +`void-packages` and make a release. Copy all key material to +`secret/releng/image-keys//{passphrase,privkey,pubkey}` in Vault and +ensure that the copy has been completed successfully. -Signing the images is done after all the images have been checked and -validated, and after the decision has been made to promote the set to -`current`. - -Generate a new signing key: +Copy the `sha256sum.txt` file to your local workstation and sign it with the +appropriate key. ``` -$ pwgen -cny 25 1 > void-release-.key -$ signify -G -p void-release-.pub -s void-release-.sec -c "This key is only valid for images with date ." +$ minisign -S -x sha256sum.sig -s void-release-$DATECODE.key \ + -c "This key is only valid for images with date $DATECODE." \ + -t "This key is only valid for images with date $DATECODE." \ + -m sha256sum.txt ``` -Copy the public half of this key to the void-release-keys package in -void-packages and make a release. Copy all key material to the -owner's team in keybase and ensure that the copy has been completed -successfully. - -Copy the `sha256.txt` file to your local workstation and sign it with the appropriate key. +Alternatively, key generation and signing can be done with `release.sh` in +void-mklive, which will generate the proper keys and sign the files as described +above: ``` -$ signify -S -e -s void-release-.sec -m sha256.txt -x sha256sum.sig +$ ./release.sh sign sha256sum.txt ``` -Copy the signed file back up to the master mirror and change the -current symlink to point to the now signed ISOs. +Copy the signed file back up to the master mirror and change the current symlink +to point to the now signed ISOs. -Once you have confirmed that the link has updated, post an update to -the website and arrange for the new key to be distributed as widely as -possible. +Once you have confirmed that the link has updated, post an update to the website +and arrange for the new key to be distributed as widely as possible.