ci: let the sandbox canary keep the capabilities its namespace grants - #312
Merged
Conversation
The bats suite moved from ubuntu-22.04-16core to ubuntu-latest, and the canary stopped being a sandbox: create tun0: Tuntap IOCTL TUNSETIFF failed [0], errno operation not permitted Ubuntu 24.04 sets kernel.apparmor_restrict_unprivileged_userns=1. Under it, a process with no AppArmor profile that creates a user namespace is confined to the unprivileged_userns profile, and that profile denies every capability -- so --create-namespaces gets its namespace and is then refused the CAP_NET_ADMIN the namespace exists to grant. kind runs its pods unconfined, so every sandbox in that cluster is that process. Only this suite is affected. tests/integration drives the same path on the same runner and passes, because it runs under `sudo make test` and the restriction mediates unprivileged userns creation only. agent_sandbox.bats grants --cap-add NET_ADMIN and never takes the namespace route at all. The sysctl is cleared on the runner rather than the manifest being given a securityContext: GKE, where this canary deploys, applies an AppArmor profile to its pods, so the transition never happens there, and the manifest under test should stay the manifest that ships. nano-init's own diagnosis was worse than useless here -- it asked for CAP_NET_ADMIN, which the namespace had granted and the host had taken back -- so it now reads the sysctl and names it. That is the only thing a contributor running the suite on their own 24.04 workstation has to go on.
Contributor
There was a problem hiding this comment.
Code Review
This pull request adds detection for Ubuntu's unprivileged user namespace restriction (/proc/sys/kernel/apparmor_restrict_unprivileged_userns) to improve the error diagnostics when TUN device creation fails. It introduces a helper function userNSCapabilitiesRestricted and updates tunHint to explain when AppArmor restricts capabilities in unprivileged user namespaces. Unit tests in isolation_test.go have also been updated to cover these new cases. There are no review comments, so I have no feedback to provide.
This was referenced Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Canary rollout: the sandbox canary comes up and is a sandboxhas failed on every bats run sinceuse normal runners(b86dccf), which moved the suite fromubuntu-22.04-16coretoubuntu-latest. The assertion that fails is the interface list, but the cause is in the agent's log:Ubuntu 24.04 sets
kernel.apparmor_restrict_unprivileged_userns=1. Under it, a process with no AppArmor profile that creates a user namespace is confined to theunprivileged_usernsprofile, and that profile denies every capability — so--create-namespacesgets its namespace and is then refused theCAP_NET_ADMINthe namespace exists to grant. kind runs its pods unconfined, so every sandbox in that cluster is that process.Only this suite is affected, which is worth stating because two neighbours look like they should be:
tests/integration(TestSandboxCreatesItsOwnNamespaces)sudo make test, and the restriction mediates unprivileged userns creation onlyagent_sandbox.bats--cap-add NET_ADMINand never takes the namespace routekind-mesh-e2e.ymlThe sysctl is cleared on the runner rather than the canary being given a
securityContext: GKE, where this canary actually deploys, applies an AppArmor profile to its pods, so the transition never happens there — the manifest under test should stay the manifest that ships.nano-init's own diagnosis was worse than useless here. It asked forCAP_NET_ADMIN, which the namespace had granted and the host had taken back, so it now reads the sysctl and names it. That is the only thing a contributor running the suite on their own 24.04 workstation has to go on, since no workflow step will fix it for them.