cmd-osbuild: Add support for LiveISO #3861

ravanelli · 2024-08-19T19:15:42Z

Add support for OSBUILD via COSA_USE_LIVEISO env var for now

jlebon

Approach looks sane overall! Want to fold the commits in #3847 into this one?

src/cmd-buildextend-live

dustymabe

I did a review here but I'm not sure we should really merge this until we get a better idea of what the actual stages will look like (i.e. the manifests and the inputs will change).

src/cmd-buildextend-live

jlebon

Two of the commits have a leading space in their titles.

The last commit is missing a blank line between the title and the body, which messes up git log --oneline.

Agree we should wait until we finalize the stage before adding the pipeline here. Though if you'd like, we can at least get the prep commits in for now in a separate PR.

src/cmd-buildextend-live

dustymabe · 2024-11-18T21:07:42Z

src/osbuild-manifests/platform.live-iso.ipp.yaml

+        options:
+          filename: efiboot.img
+          size:
+            mpp-format-string: '{live_efiboot_img_size_mb * 1024 * 1024}' 


Suggested change

mpp-format-string: '{live_efiboot_img_size_mb * 1024 * 1024}'

mpp-format-string: '{16 * 1024 * 1024}'

We discussed this last week and decided to just hardcode it to 16 here for now and add a comment about it and mention we can parameterize it if we ever need to adjust it in the future.

dustymabe · 2024-11-18T21:09:27Z

src/osbuild-manifests/platform.live-iso.ipp.yaml

+              # XXX: we should rename this pipeline now that we don't use it as
+              # a buildroot
+              - name:build


if #3946 merges we'll need to update this to use the new name.

Suggested change

# XXX: we should rename this pipeline now that we don't use it as

# a buildroot

- name:build

# XXX: we should rename this pipeline now that we don't use it as

# a buildroot

- name:build

dustymabe · 2024-11-18T21:17:00Z

src/cmd-osbuild

@@ -242,6 +246,7 @@ main() {
        "cmd-buildextend-qemu") platforms=(qemu);;
        "cmd-buildextend-qemu-secex") platforms=(qemu-secex);;
        "cmd-buildextend-secex") platforms=(qemu-secex);;
+	"cmd-buildextend-live") platforms=(live-iso);;


we'd need a corresponding entry in the Perform postprocessing case statement in this file to handle the multiple artifacts that are being generated (i.e. not just the ISO should be getting generated now)

src/runvm-osbuild

dustymabe · 2024-11-18T21:22:42Z

Two largish change suggestions:

We consider the "platform" here to be live and not live-iso.

This means platform.live-iso.ipp.yaml -> platform.live.ipp.yaml and that generates the ISO/PXEkernel/initrd/rootfs.

we rename the stage we are proposing to OSBuild to be stages/org.osbuild.coreos.live OR platform.live-artifacts.

dustymabe · 2024-11-18T21:34:57Z

src/osbuild-manifests/platform.live-iso.ipp.yaml

+# This file defines the pipeline for building the live ISO.
+version: '2'
+pipelines:
+  - name: live-iso


since we are copying in the metal.raw and metal4k.raw images here when we copy out of this pipeline it will include those metal.raw and metal4k.raw images which isn't what we want.

what we'll have to do is have one stage for creating the live artifacts and another stage where we copy only the created artifacts, which then gets copied out.

That is... unless we figure out how to do the loopback device stuff inside the stage. If we can do the loopback stuff inside the stage code then we can use an input: from a previous pipeline instead and then set up the loopback device from within the stage code.

I guess another option would be to teach the devices/org.osbuild.loopback code to somehow work on inputs.. but I'm not sure if it can be done easily or if it would be accepted.

The `build` pipeline was renamed to `deployed-tree` in 3a91ec2 and this should have been updated in that commit as well.

We run osbuild inside of supermin, so for building the live ISO we'll need all the tools it needs in there. This includes `genisoimage` and `syslinux-nonlinux`. We don't currently name `squashfs-tools` as a toplevel dependency. It's currently pulled in by libguestfs, but we need it in supermin too, so explicitly list it there.

- Introduce necessary variables for platform.live-iso.ipp.yaml. - Add the platform.live-iso.ipp manifest to support LiveISO creation. - Adapt CoreOS platform manifests Signed-off-by: Renata Ravanelli <[email protected]>

- Introduce LiveISO creation via cmd-osbuild. Signed-off-by: Renata Ravanelli <[email protected]>

- Add checkpoints for metal and metal4k pipelines due LiveISO builds; - Bump cache size from 14G to 20G. Signed-off-by: Renata Ravanelli <[email protected]>

jlebon · 2024-11-21T20:40:01Z

Ran this through the differ #3968 on FCOS x86_64:

We're missing everything in /live in the live ISO. This can be obtained from the container image under /usr/share/coreos-assembler/live (see cmdlib: bake src config's live/ directory in compose #3846).
The osmet filenames in the live rootfs changed from e.g. fedora-coreos-41.20241121.dev.0-1-metal.x86_64.raw.osmet to just metal.osmet and similarly for metal4k. I don't think this breaks anything and we could do without, though having the build ID in the name is a nice cross-check.
/boot on the squashfs is missing everything under /boot/efi. We need to mount the ESP onto /boot/efi when building the squashfs like we do in coreos_gf_run_mount.

We should repeat the exercise on other arches.

ravanelli marked this pull request as draft August 19, 2024 19:15

openshift-ci bot added the do-not-merge/work-in-progress label Aug 19, 2024

ravanelli mentioned this pull request Aug 20, 2024

Build live ISO using osbuild #3847

Closed

jlebon reviewed Sep 25, 2024

View reviewed changes

ravanelli force-pushed the pr/cosa_live branch from 4bbcf0a to 75f616f Compare October 1, 2024 15:16

ravanelli removed the do-not-merge/work-in-progress label Oct 1, 2024

ravanelli marked this pull request as ready for review October 1, 2024 15:17

ravanelli force-pushed the pr/cosa_live branch 2 times, most recently from 954c139 to 543fdf6 Compare October 1, 2024 15:22

ravanelli requested a review from jlebon October 8, 2024 19:34

dustymabe reviewed Oct 17, 2024

View reviewed changes

src/cmd-buildextend-live Outdated Show resolved Hide resolved

src/cmd-buildextend-live Outdated Show resolved Hide resolved

src/cmd-buildextend-live Outdated Show resolved Hide resolved

jlebon reviewed Oct 18, 2024

View reviewed changes

src/cmd-buildextend-live Outdated Show resolved Hide resolved

src/cmd-buildextend-live Outdated Show resolved Hide resolved

src/cmd-buildextend-live Outdated Show resolved Hide resolved

ravanelli force-pushed the pr/cosa_live branch from 543fdf6 to 7e84e2c Compare October 29, 2024 21:53

ravanelli requested review from dustymabe and jlebon October 29, 2024 21:56

ravanelli force-pushed the pr/cosa_live branch from 7e84e2c to ba12dc1 Compare October 30, 2024 00:45

ravanelli force-pushed the pr/cosa_live branch 2 times, most recently from 2cc42e3 to 9abdf48 Compare November 18, 2024 20:55

dustymabe reviewed Nov 18, 2024

View reviewed changes

src/runvm-osbuild Outdated Show resolved Hide resolved

dustymabe reviewed Nov 18, 2024

View reviewed changes

ravanelli changed the title ~~buildextend-live: Add support OSBUILD~~ cmd-osbuild: Add support for LiveISO Nov 19, 2024

ravanelli force-pushed the pr/cosa_live branch 2 times, most recently from 9e1cf94 to 8a0b479 Compare November 20, 2024 00:43

dustymabe and others added 3 commits November 21, 2024 12:32

runvm-osbuild: checkpoint deployed-tree pipeline

83ab13a

The `build` pipeline was renamed to `deployed-tree` in 3a91ec2 and this should have been updated in that commit as well.

runvm-osbuild: Add initial support for LiveISO generation

1e10a9f

- Introduce necessary variables for platform.live-iso.ipp.yaml. - Add the platform.live-iso.ipp manifest to support LiveISO creation. - Adapt CoreOS platform manifests Signed-off-by: Renata Ravanelli <[email protected]>

ravanelli and others added 3 commits November 21, 2024 12:32

cmd-osbuild: Add LiveISO build support

5d0c918

- Introduce LiveISO creation via cmd-osbuild. Signed-off-by: Renata Ravanelli <[email protected]>

runvm-osbuild: Add checkpoints for metal/metal4k and bump cache size

3a85671

- Add checkpoints for metal and metal4k pipelines due LiveISO builds; - Bump cache size from 14G to 20G. Signed-off-by: Renata Ravanelli <[email protected]>

add osbuild patches for live-artifacts

16628f9

dustymabe force-pushed the pr/cosa_live branch from 8a0b479 to 16628f9 Compare November 21, 2024 17:34

jlebon mentioned this pull request Nov 22, 2024

cmdlib: also chown back compose.json in privileged path #3967

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd-osbuild: Add support for LiveISO #3861

cmd-osbuild: Add support for LiveISO #3861

ravanelli commented Aug 19, 2024

jlebon left a comment

dustymabe left a comment

jlebon left a comment

dustymabe Nov 18, 2024

dustymabe Nov 18, 2024

dustymabe Nov 18, 2024 •

edited

Loading

dustymabe commented Nov 18, 2024

dustymabe Nov 18, 2024 •

edited

Loading

jlebon commented Nov 21, 2024 •

edited

Loading

	mpp-format-string: '{live_efiboot_img_size_mb * 1024 * 1024}'
	mpp-format-string: '{16 * 1024 * 1024}'

cmd-osbuild: Add support for LiveISO #3861

Are you sure you want to change the base?

cmd-osbuild: Add support for LiveISO #3861

Conversation

ravanelli commented Aug 19, 2024

jlebon left a comment

Choose a reason for hiding this comment

dustymabe left a comment

Choose a reason for hiding this comment

jlebon left a comment

Choose a reason for hiding this comment

dustymabe Nov 18, 2024

Choose a reason for hiding this comment

dustymabe Nov 18, 2024

Choose a reason for hiding this comment

dustymabe Nov 18, 2024 • edited Loading

Choose a reason for hiding this comment

dustymabe commented Nov 18, 2024

dustymabe Nov 18, 2024 • edited Loading

Choose a reason for hiding this comment

jlebon commented Nov 21, 2024 • edited Loading

dustymabe Nov 18, 2024 •

edited

Loading

dustymabe Nov 18, 2024 •

edited

Loading

jlebon commented Nov 21, 2024 •

edited

Loading