Switching podman volume on BTRFS broke subvolumes

[marinmo]

Issue Description

Hello,

Let me apologize in advance; this bug report will be hard to pinpoint since it already broke the system it happened on. Podman was run as the user (ie rootless) with networking backend set to slirp4netns in containers.conf, otherwise configuration was default (storage etc).

I was running a BTRFS setup with approximately the following layout;
The root partition was mounted on the BTRFS root mountpoint
As far as I can remember, I only had one subvolume for /home
I had separate partitions for /boot and /boot/EFI.

Container data was held in /home/user/containers/ under which, amongst others, my postgresql directory resided.

Dismayed that bitnami changed their update schedule for non-latest releases, I planned to switch from the bitnami image to the official one (both pulled from docker hub fwiw).

My containers were managed by systemd by units placed in /home/user/.config/containers/systemd.

I first stopped the running postgresql container (bitnami). Unaware of the volume layout of the official image, I found documentation saying that data resided in /var/lib/postgresql/data.

Since the data directory of the bitnami image was located in /home/user/containers/postgresql/16/data I ran the following (the supplied environment file contains the POSTGRES_PASSWORD and PGDATA (set to /var/lib/postgresql/data) variables);
podman run -it --rm --name=pgsql --env-file /home/user/containers/envs/postgresql.env -p 5432:5432 -v /home/user/containers/postgresql/16/data:/var/lib/postgresql/data:z docker.io/library/postgres:16

Here it's worth noting that I had a suspicion that permissions would not be correct for the old data directory (and I was basically hoping it'd use the same directory layout as the bitnami image, which wasn't the case), but figured that I'd solve it once I saw the container running. The container started and initialized (what proved to be a new database). Pleased the container started I CTRL+C the process after which my shell becomes unresponsive (as in starts throwing errors about files not being found) - ie I don't even get a prompt but only errors.

SSHing in to the machine does not work anymore either, so I power cycle manually. On restart the machine won't boot due to switch-root failing. After some back/forward I boot into a live environment, manage to mount the root BTRFS volume to /mnt/root. The root volume now (only) contains the /boot, /dev, /home, /proc, /run, /sys and /tmp directories. The /home directory is somehow still intact (despite being a separate subvolume). That is, the full contents of the /home directory are still accessible from the root volume. I've managed to copy the contents of the containers directory over to another drive.

Running btrfs subvolume list /mnt/root lists the following;

ID 256 gen 2749056 top level 5 path root
ID 259 gen 2749814 top level 256 path root/home/user/containers/postgresql/16/data/pgdata/var/lib/portables
ID 277 gen 2749814 top level 256 path root/home/user/containers/postgresql/16/data/pgdata/var/lib/machines

The portables and machines directories are empty, but it seems /var and /var/lib are from some kind of distribution (var contains subdirectories such as account, adm, cache, crash, db, empty, ftp, games, kerberos etc - guessing esp from permissions of these files that they're from the container populating these directories (despite me not mounting them ...).

I really have no idea what happened, but from what I can tell, podman somehow trashed my filesystem on the host nonetheless. No other process was running that might have affected the filesystem in this way. The subvolumes created also heavily imply something went awry with podman (why'd it create a subvolume there?).

I wish I could give more detail but I don't even know where to start. The system is still in the state described above, but is also a machine I use on my home network and I don't feel like keeping it this way for too long, so please let me know asap what if any more information you'd need.

Steps to reproduce the issue

These are probably very shaky and might not reproduce.

1. Run the bitnami container of pgsql 16 (which will be 16.6) with a persistent data directory
2. Stop the bitnami container
3. Run the official postgresql image with the data directory of the bitnami container as the data volume for the official image
4. Once startup is complete, stop (in my case via CTRL+C) the container running

Describe the results you received

A broken filesystem.

Describe the results you expected

A less, preferably not at all, broken system.

podman info output

Fedora 41 (server), podman was latest (5.4.0)

Podman in a container

No

Privileged Or Rootless

Rootless

Upstream Latest Release

Yes

Additional environment details

No response

Additional information

No response