Skip to content

ci: let the sandbox canary keep the capabilities its namespace grants - #312

Merged
aojea merged 1 commit into
google:mainfrom
aojea:userns-apparmor
Aug 25, 2026
Merged

ci: let the sandbox canary keep the capabilities its namespace grants#312
aojea merged 1 commit into
google:mainfrom
aojea:userns-apparmor

Conversation

@aojea

@aojea aojea commented Aug 25, 2026

Copy link
Copy Markdown
Collaborator

Canary rollout: the sandbox canary comes up and is a sandbox has failed on every bats run since use normal runners (b86dccf), which moved the suite from ubuntu-22.04-16core to ubuntu-latest. The assertion that fails is the interface list, but the cause is in the agent's log:

set up sandbox network: create tun0: Tuntap IOCTL TUNSETIFF failed [0], errno operation not permitted

Ubuntu 24.04 sets kernel.apparmor_restrict_unprivileged_userns=1. Under it, a process with no AppArmor profile that creates a user namespace is confined to the unprivileged_userns profile, and that profile denies every capability — so --create-namespaces gets its namespace and is then refused the CAP_NET_ADMIN the namespace exists to grant. kind runs its pods unconfined, so every sandbox in that cluster is that process.

Only this suite is affected, which is worth stating because two neighbours look like they should be:

runner outcome
tests/integration (TestSandboxCreatesItsOwnNamespaces) ubuntu-latest passes — runs under sudo make test, and the restriction mediates unprivileged userns creation only
agent_sandbox.bats ubuntu-latest passes — grants --cap-add NET_ADMIN and never takes the namespace route
kind-mesh-e2e.yml ubuntu-22.04 no sandbox, and no such sysctl

The sysctl is cleared on the runner rather than the canary being given a securityContext: GKE, where this canary actually deploys, applies an AppArmor profile to its pods, so the transition never happens there — the manifest under test should stay the manifest that ships.

nano-init's own diagnosis was worse than useless here. It asked for CAP_NET_ADMIN, which the namespace had granted and the host had taken back, so it now reads the sysctl and names it. That is the only thing a contributor running the suite on their own 24.04 workstation has to go on, since no workflow step will fix it for them.

The bats suite moved from ubuntu-22.04-16core to ubuntu-latest, and the canary
stopped being a sandbox:

  create tun0: Tuntap IOCTL TUNSETIFF failed [0], errno operation not permitted

Ubuntu 24.04 sets kernel.apparmor_restrict_unprivileged_userns=1. Under it, a
process with no AppArmor profile that creates a user namespace is confined to
the unprivileged_userns profile, and that profile denies every capability -- so
--create-namespaces gets its namespace and is then refused the CAP_NET_ADMIN
the namespace exists to grant. kind runs its pods unconfined, so every sandbox
in that cluster is that process.

Only this suite is affected. tests/integration drives the same path on the same
runner and passes, because it runs under `sudo make test` and the restriction
mediates unprivileged userns creation only. agent_sandbox.bats grants
--cap-add NET_ADMIN and never takes the namespace route at all.

The sysctl is cleared on the runner rather than the manifest being given a
securityContext: GKE, where this canary deploys, applies an AppArmor profile to
its pods, so the transition never happens there, and the manifest under test
should stay the manifest that ships.

nano-init's own diagnosis was worse than useless here -- it asked for
CAP_NET_ADMIN, which the namespace had granted and the host had taken back --
so it now reads the sysctl and names it. That is the only thing a contributor
running the suite on their own 24.04 workstation has to go on.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds detection for Ubuntu's unprivileged user namespace restriction (/proc/sys/kernel/apparmor_restrict_unprivileged_userns) to improve the error diagnostics when TUN device creation fails. It introduces a helper function userNSCapabilitiesRestricted and updates tunHint to explain when AppArmor restricts capabilities in unprivileged user namespaces. Unit tests in isolation_test.go have also been updated to cover these new cases. There are no review comments, so I have no feedback to provide.

@aojea
aojea merged commit 1f78422 into google:main Aug 25, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant