Installing to filesystem: Installing bootloader: Permission denied (os error 13) when targeting arm64

[clodaghwalsh17]

I am building an image for a raspberry pi using this Containerfile. I have tried the following build on a Mac M2 and on a raspberry pi 400 booted from a 128GB USB with the Raspberry Pi OS.

podman run \
--rm \
-it \
--privileged \
--pull=newer \
--security-opt label=type:unconfined_t \
-v ./config.toml:/config.toml:ro \
-v ./output:/output \
-v /var/lib/containers/storage:/var/lib/containers/storage \
quay.io/centos-bootc/bootc-image-builder:latest \
--target-arch arm64 \
--type raw \
--rootfs xfs \
--local \
$(OCI_IMAGE)

In both environments the build fails at the org.osbuild.bootc.install-to-filesystem step of the image pipeline with the error:

org.osbuild.bootc.install-to-filesystem: f93246b75832229fc2a2071dfacd6f7411ee629a07e3269bc6a906e2bb2cdad2 {
"kernel-args": [
"rw",
"console=tty0",
"console=ttyS0"
],
"target-imgref": "quay.io/clwalsh/bootc-pi"
}
device/disk (org.osbuild.loopback): loop0 acquired (locked: False)
mount/- (org.osbuild.xfs): mounting /dev/loop0p3 -> /store/tmp/buildroot-tmp-xdmrvd4u/mounts/
mount/boot (org.osbuild.xfs): mounting /dev/loop0p2 -> /store/tmp/buildroot-tmp-xdmrvd4u/mounts/boot
mount/boot-efi (org.osbuild.fat): mounting /dev/loop0p1 -> /store/tmp/buildroot-tmp-xdmrvd4u/mounts/boot/efi
Mount transient overlayfs for /etc/containers
Creating bind mount for run/osbuild/containers
Installing image: docker://quay.io/clwalsh/bootc-pi
Initializing ostree layout
layers already present: 0; layers needed: 69 (2.1 GB)
Deploying container image...done (2 minutes)
Running bootupctl to install bootloader

bootupctl backend install --write-uuid --device /dev/loop0 /run/osbuild/mounts
ERROR Installing to filesystem: Installing bootloader: Permission denied (os error 13)
Traceback (most recent call last):
File "/run/osbuild/bin/org.osbuild.bootc.install-to-filesystem", line 53, in
r = main(args["options"], args["inputs"], args["paths"])
File "/run/osbuild/bin/org.osbuild.bootc.install-to-filesystem", line 48, in main
subprocess.run(pargs, env=env, check=True)
~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.13/subprocess.py", line 577, in run
raise CalledProcessError(retcode, process.args,
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['bootc', 'install', 'to-filesystem', '--source-imgref', 'containers-storage:[overlay@/run/osbuild/containers/storage+/run/containers/storage]7f07da8e930fddfdcb138659f973fe102eb87c509ab38b6569d2ee9b749691a6', '--skip-fetch-check', '--generic-image', '--karg', 'rw', '--karg', 'console=tty0', '--karg', 'console=ttyS0', '--target-imgref', 'quay.io/clwalsh/bootc-pi', '/run/osbuild/mounts']' returned non-zero exit status 1.
mount/boot-efi (org.osbuild.fat): umount: /store/tmp/buildroot-tmp-xdmrvd4u/mounts/boot/efi unmounted
mount/boot (org.osbuild.xfs): umount: /store/tmp/buildroot-tmp-xdmrvd4u/mounts/boot unmounted
mount/- (org.osbuild.xfs): umount: /store/tmp/buildroot-tmp-xdmrvd4u/mounts/ unmounted

⏱ Duration: 563s
manifest - failed
Failed
2024/11/29 16:59:00 error: cannot run osbuild: running osbuild failed: exit status 1
make: *** [Makefile:16: disk-image] Error 1

[kingsleyzissou]

Thanks Clodagh, are you running this with sudo?

[clodaghwalsh17]

Yes the command is ran as sudo

[mvo5]

Thanks for your bugreport! Could you please check if there a selinux denial when this happens?

[clodaghwalsh17]

There doesnt seem to be any selinux denials in the output of the podman run command. The details of selinux stage are as follows:

org.osbuild.selinux: 23191109ada6bb4c2cabf53f65a57187ef86576a78f93184f8fcf3431d75a5a2 {
  "file_contexts": "etc/selinux/targeted/contexts/files/file_contexts",
  "exclude_paths": [
    "/sysroot"
  ],
  "labels": {
    "/usr/bin/mount": "system_u:object_r:install_exec_t:s0",
    "/usr/bin/ostree": "system_u:object_r:install_exec_t:s0",
    "/usr/bin/umount": "system_u:object_r:install_exec_t:s0"
  }
}

I'm having difficulty execing into the running container to check any log files for selinux denials.

[mvo5]

Thanks! I wonder if this might be a dupe of #645 - sudo journalctl |grep avc: might be a quick way to check.

[clodaghwalsh17]

sudo journalctl |grep avc returns an empty list from the Raspberry Pi. I installed the selinux tools to check the host system selinux policy as mentioned in #645 and confirmed that selinux is disabled on the pi where I am running the build.

[cgwalters]

So bootc has a BOOTC_BOOTLOADER_DEBUG environment variable that one can set, but I don't know offhand if that would make it all the way through if just passed via podman run --env=BOOTC_BOOTLOADER_DEBUG=1 ... quay.io/centos-bootc/bootc-image-builder ....

Raspberry Pi OS.

What I don't know though...is there maybe apparmor involved here?

[clodaghwalsh17]

The apparmor module appears to be loaded on the Pi but it doesnt seem to be active.

pi@raspberrypi:~ $ aa-status
apparmor module is loaded.
apparmor filesystem is not mounted.

pi@raspberrypi:~ $ sudo journalctl | grep apparmor
Nov 19 13:46:18 raspberrypi systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
Dec 02 14:17:04 raspberrypi systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
Dec 02 15:17:05 raspberrypi systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
Dec 02 15:17:06 raspberrypi systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
Dec 05 13:17:04 raspberrypi systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
Dec 05 13:17:05 raspberrypi systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
Dec 05 13:17:06 raspberrypi systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
Dec 05 14:17:03 raspberrypi systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).
Dec 05 14:17:04 raspberrypi systemd[1]: apparmor.service - Load AppArmor profiles was skipped because of an unmet condition check (ConditionSecurity=apparmor).

[clodaghwalsh17]

@cgwalters I ran the build with env=BOOTC_BOOTLOADER_DEBUG=1 but unfortunately it did not provide any extra logs