Using nftables on a Windows WSL machine doesn't work

[l0rd]

Issue Description

After updating to Fedora 41 and configuring the machine in rootful mode, running a container returns a netavark error:

# podman run hello
WARN[0000] Using cgroups-v1 which is deprecated in favor of cgroups-v2 with Podman v5 and will be removed in a future version. Set environment variable `PODMAN_IGNORE_CGROUPSV1_WARNING` to hide this warning.
internal:0:0-0: Error: Could not process rule: No such file or directory

internal:0:0-0: Error: Could not process rule: No such file or directory

Error: netavark: nftables error: nft did not return successfully while applying ruleset

Steps to reproduce the issue

Steps to reproduce the issue

1. Init/Run a rootful machine:

> cd <podman git repo>
> gh pr checkout 26318
> .\winmake.ps1 podman
> .\bin\windows\podman machine init --image docker://quay.io/mloriedo/machine-os-wsl:5.5-next
> .\bin\windows\podman podman machine set --rootful
> .\bin\windows\podman podman machine start
> .\bin\windows\podman podman run hello

podman info output

Podman built from main branch running on Windows 11 with WSL v2.4.9.0 and Kernel v5.15.167.4-1

Additional information

Using a more recent version of the Kernel (v6.6.36) doesn't help.

Configuring podman to use iptables fixes the problem:

[network]
firewall_driver="iptables"

[l0rd]

Tested a rootful WSL machine with a simple nftables ruleset and got no error:

# cat << EOF > /tmp/simplerule
table inet filter {
       chain input {
           type filter hook input priority 0;
           policy drop;

           # Allow established/related connections
           ct state established,related accept

           # Allow HTTP
           tcp dport 80 accept

           # Allow HTTPS
           tcp dport 443 accept
       }
   }
EOF
# nft -f /tmp/simplerule
# nft list ruleset
table inet filter {
        chain input {
                type filter hook input priority filter; policy drop;
                ct state established,related accept
                tcp dport 80 accept
                tcp dport 443 accept
        }
}

However netawark tests in 250-bridge-nftables.bats failed:

# dnf install -y make rust cargo protobuf-compiler bats jq
# cd /mnt/c/Users/<folder where netavark was cloned>
# make
# bats ./test/250-bridge-nftables.bats -f "check nftables driver is in use"
250-bridge-nftables.bats
 ✗ check nftables driver is in use
   (from function `die' in file test/helpers.bash, line 249,
    from function `run_helper' in file test/helpers.bash, line 233,
    from function `run_in_host_netns' in file test/helpers.bash, line 166,
    from function `run_netavark' in file test/helpers.bash, line 136,
    in test file test/250-bridge-nftables.bats, line 12)
     `RUST_LOG=netavark=info run_netavark --file ${TESTSDIR}/testfiles/simplebridge.json setup $(get_container_netns_path)' failed
    nsenter -n -m -w -t 14493 ip link set lo up
    nsenter -n -m -w -t 14493 ./bin/netavark --rootless false --config /tmp/netavark_bats.1qVMcE/config --file /mnt/c/Users/mario/Git/netavark/test/testfiles/simplebridge.json setup /proc/14495/ns/net
   [INFO  netavark::firewall] Using nftables firewall driver
   [INFO  netavark::network::netlink] Adding route (dest: 0.0.0.0/0 ,gw: 10.88.0.1, metric 100)
   [INFO  netavark::firewall::nft] Creating container chain nv_53ce4390_10_88_0_0_nm16
   internal:0:0-0: Error: Could not process rule: No such file or directory

   internal:0:0-0: Error: Could not process rule: No such file or directory

   {"error":"nftables error: nft did not return successfully while applying ruleset"}
   [ rc=1 (** EXPECTED 0 **) ]
   #/vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
   #| FAIL: exit code is 1; expected 0
   #\^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

1 test, 1 failure

[l0rd]

@Luap99 any idea why running nft manually works but netawark fails? Is there a nft ruleset I can test?

[Luap99]

table inet netavark {
	chain INPUT {
		type filter hook input priority filter; policy accept;
		ip saddr 10.88.0.0/16 meta l4proto { tcp, udp } th dport 53 accept
	}

	chain FORWARD {
		type filter hook forward priority filter; policy accept;
		ct state invalid drop
		jump NETAVARK-ISOLATION-1
		ip daddr 10.88.0.0/16 ct state established,related accept
		ip saddr 10.88.0.0/16 accept
	}

	chain POSTROUTING {
		type nat hook postrouting priority srcnat; policy accept;
		meta mark & 0x00002000 == 0x00002000 masquerade
		ip saddr 10.88.0.0/16 jump nv_2f259bab_10_88_0_0_nm16
	}

	chain PREROUTING {
		type nat hook prerouting priority dstnat; policy accept;
		fib daddr type local jump NETAVARK-HOSTPORT-DNAT
	}

	chain OUTPUT {
		type nat hook output priority dstnat; policy accept;
		fib daddr type local jump NETAVARK-HOSTPORT-DNAT
	}

	chain NETAVARK-HOSTPORT-DNAT {
		ip daddr 10.88.0.1 meta l4proto { tcp, udp } th dport 53 dnat ip to 10.88.0.1:54
		tcp dport 80 jump nv_2f259bab_10_88_0_0_nm16_dnat
	}

	chain NETAVARK-HOSTPORT-SETMARK {
		meta mark set meta mark | 0x00002000
	}

	chain NETAVARK-ISOLATION-1 {
	}

	chain NETAVARK-ISOLATION-2 {
	}

	chain NETAVARK-ISOLATION-3 {
		oifname "podman0" drop
		jump NETAVARK-ISOLATION-2
	}

	chain nv_2f259bab_10_88_0_0_nm16 {
		ip daddr 10.88.0.0/16 accept
		ip daddr != 224.0.0.0/4 masquerade
	}

	chain nv_2f259bab_10_88_0_0_nm16_dnat {
		ip saddr 10.88.0.0/16 tcp dport 80 jump NETAVARK-HOSTPORT-SETMARK
		ip saddr 127.0.0.1 tcp dport 80 jump NETAVARK-HOSTPORT-SETMARK
		tcp dport 80 dnat ip to 10.88.0.2:80
	}
}

That is from a simple podman run -d -p 80:80 quay.io/libpod/testimage:20241011 sleep 10

You can also strace -f the podman command, in the netlink messages there are some more nft error messages encoded that are not displayed otherwise.

Looking at this again I think we are using l4proto { tcp, udp } which is a set which per issue microsoft/WSL#6044 is not working

[elhananjair]

Hello @l0rd
I am facing the same issue when executing podman-compose --in-pod=1 --project-name myproject -f ./docker-compose.yml up -d, Have you found a solution for that yet?

[l0rd]

You should not be facing the problem if you are using the default Podman machine. The problem is with nftables and the WSLv2 kernel but we are currently configuring the WSL machine to use iptables rather than the nftables.

[elhananjair]

You should not be facing the problem if you are using the default Podman machine. The problem is with nftables and the WSLv2 kernel but we are currently configuring the WSL machine to use iptables rather than the nftables.

Yeah, I am using WSL2 with the Fedora 42 beta WSL image and facing the issue.

[l0rd]

You can still manually configure podman to use iptables by adding the following lines to the /etc/containers/containers.conf (that's what we currently do automatically when creating the WSL machine):

[network]
firewall_driver="iptables"