Add kube networking facilities#28
Draft
c-kruse wants to merge 4 commits into
Draft
Conversation
Allows kube-proxy in iptables mode and other networking tools to use
probabilistic packet matching for load balancing across multiple
endpoints. e.g. `iptables -m statistic ...`
Sets CONFIG_NETFILTER_XT_MATCH_STATISTIC=y
On arm64, enable two Xtables options needed, already set on x86_64
CONFIG_NETFILTER_XT_MATCH_COMMENT=y
CONFIG_NETFILTER_XT_MATCH_MULTIPORT=y
Signed-off-by: Christian Kruse <christian@c-kruse.com>
Allows kube-proxy in nftables mode and other networking tools to use
random number generation for endpoint selection and forwarding
information base lookups for local-address dispatch. e.g. `nft add
rule ... numgen random mod 1 map { ... }`, `nft add rule ... fib
daddr type local`
Sets CONFIG_NFT_NUMGEN=y
Sets CONFIG_NFT_FIB_IPV4=y (selects CONFIG_NFT_FIB=y)
Sets CONFIG_NFT_FIB_IPV6=y
Signed-off-by: Christian Kruse <christian@c-kruse.com>
Allows kindnetd using the kube-network-policies reference implementation, firewalls, and other networking tools to enforce policy via first-packet NFQUEUE inspection with conntrack label caching. e.g. `nft add table inet ...`, `nft add rule ... queue num 101 bypass`, `ct label set 28` Sets CONFIG_NF_TABLES_INET=y Sets CONFIG_NETFILTER_NETLINK_QUEUE=y Sets CONFIG_NETFILTER_NETLINK_GLUE_CT=y Sets CONFIG_NF_CT_NETLINK=y Signed-off-by: Christian Kruse <christian@c-kruse.com>
|
We require contributors to sign our Contributor License Agreement, and we don't have @c-kruse on file. You can sign our CLA at https://e2b.dev/docs/cla . Once you've signed, post a comment here that says '@cla-bot check' |
Author
|
@cla-bot check |
|
The cla-bot has been summoned, and re-checked this pull request! |
Allows kube-proxy in iptables mode and other networking tools to implement session affinity, rate limiting and throttling with xtables recency rules. Sets CONFIG_NETFILTER_XT_MATCH_RECENT=y Signed-off-by: Christian Kruse <christian@c-kruse.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Opening as a conversation about which of these kernel features are appropriate to include in the default Firecracker kernel.
Background
Running a lightweight Kubernetes distribution or developing Kubernetes components in an E2B sandbox currently requires deploying a custom Firecracker kernel.
Kubernetes networking is composed of several independently replaceable components, each with different kernel dependencies. This branch enables a minimal set of additional capabilities intended to support a reasonably complete sandboxed Kubernetes environment without adopting a substantially larger feature set.
kube-proxy
kube-proxy is included in most lightweight k8s distributions to implement Service traffic routing. Alternatives tend to be eBPF based and require a broad set of kernel features. kube-proxy has several modes.
The nftables changes are limited to x86_64 because the current arm configuration does not enable the nf_tables subsystem.
Without at least one of these modes or an alternative Service-routing implementation, the current fc kernel cannot support the networking expected by Kubernetes installations.
NetworkPolicy
NetworkPolicy enforcement is optional, and not every Kubernetes distribution includes an implementation. For this experiment I went with kube-network-policies (included with kindnetd/kind). I judged it to have the smallest surface area, centered around NFQUEUE and conntrack. This is enabled by 1849240.
Other implementations, like Calico, require IP Sets support, several other xtables matches and usually some form of tunneling support - VXLAN, IPIP.
Validation
Tested on x86 with multi Node kind, kube-proxy in iptables, nftables and NetworkPolicy. The arm64 configuration with kube-proxy iptables mode only, no NetworkPolicy.
Recommendation
Not a security professional, but I suspect adding the xtables matches (statistic, recent) to support the iptables mode is pretty uncontroversial and would amount to maybe about 20KB in vmlinux. As for the nftables changes I'm not sure what to suggest - I am a bit confused about the detla between arm and amd64, and they are comparatively larger changes.
Need help on this PR? Tag
/codesmithwith what you need. Autofix is disabled.