Skip to content

[Request]: Enable Cilium's base kernel requirements in the kernel configs #912

Description

@saiyam1814

Feature or enhancement request details

kernel/config-arm64 (and config-x86_64) is the documented way to build a custom kernel for container, and apple/container discussion #1673 asks for Cilium in container k8s clusters and links Cilium's requirements page. Cilium's documented base requirements (https://docs.cilium.io/en/stable/operations/system_requirements/#base-requirements):

CONFIG_BPF=y  CONFIG_BPF_EVENTS=y  CONFIG_BPF_SYSCALL=y  CONFIG_NET_CLS_BPF=y  CONFIG_BPF_JIT=y
CONFIG_NET_CLS_ACT=y  CONFIG_NET_SCH_INGRESS=y  CONFIG_DEBUG_INFO_BTF=y  CONFIG_CRYPTO_SHA1=y
CONFIG_CRYPTO_USER_API_HASH=y  CONFIG_CGROUPS=y  CONFIG_CGROUP_BPF=y  CONFIG_PERF_EVENTS=y  CONFIG_SCHEDSTATS=y

Five are not enabled in config-arm64 on main (the same five in config-x86_64):

Option State today Note
CONFIG_BPF_JIT absent the Cilium agent needs a JIT
CONFIG_SCHEDSTATS # ... is not set one-line flip
CONFIG_CRYPTO_USER_API_HASH # ... is not set one-line flip
CONFIG_BPF_EVENTS absent needs FTRACE and KPROBES/KPROBE_EVENTS, otherwise olddefconfig drops it
CONFIG_DEBUG_INFO_BTF absent (DEBUG_INFO_NONE=y) needs pahole (dwarves) in kernel/image

Everything else on the list, including the tunneling, netfilter, L7 and WireGuard sets, is already =y. CONFIG_NETKIT=y (Cilium's netkit device mode, optional) is also absent. For comparison, the kernel container recommends today (Kata 3.32.0 debug, 6.18.35) has BPF_JIT, BPF_EVENTS and DEBUG_INFO_BTF but not SCHEDSTATS, NETKIT or NETFILTER_XT_MATCH_SOCKET. A Cilium 1.19.4 agent starts on it but keeps failing to install its proxy iptables rules (-m socket --transparent). This request is only about kernel/config-arm64.

Measured. I built linux-6.18.5 (the version kernel/Makefile pins) three times from config-arm64, same toolchain for all three (gcc 13.3 in ubuntu:24.04, not the gcc 9.4 kernel/image toolchain the file was generated with, so absolute sizes will differ slightly from an in-tree build; the deltas should not):

Variant arm64 Image bytes Delta
config-arm64 as-is 29,354,496
+ BPF_JIT, SCHEDSTATS, CRYPTO_USER_API_HASH, NETKIT 29,424,128 +69,632 (+0.24%)
+ the above and BPF_EVENTS with FTRACE, KPROBES, KPROBE_EVENTS 33,769,984 +4,415,488 (+15.0%)

make olddefconfig on the hand-edited small variant adds BPF_JIT_DEFAULT_ON=y and EXECMEM=y and surfaces # CONFIG_BPF_JIT_ALWAYS_ON is not set; nothing else moves relative to the regenerated baseline. The tracing variant additionally pulls in TRACING, EVENT_TRACING, TRACEPOINTS, RING_BUFFER, KRETPROBES, UPROBES/UPROBE_EVENTS, EPROBE_EVENTS, DYNAMIC_EVENTS, NOP_TRACER, CONTEXT_SWITCH_TRACER, GLOB, TRACE_CLOCK (all tracers stay off).

Functional check, both variants, on container CLI 1.0.0: booted with container run --kernel (bpf_jit_enable=1, /proc/schedstat present, an AF_ALG hash socket binds, ip link add ... type netkit succeeds where 6.12.28 and 6.18.35 both say "Unknown device type"), then a two-node kubeadm 1.37 cluster on kindest/node with Cilium 1.19.4 via cilium install --wait: cilium status OK with 2/2 agents, nodes Ready, and from a curl pod ClusterIP, pod IP and cluster DNS answer 200, including pod-to-pod across nodes with cilium-dbg status reporting Routing: Tunnel [vxlan]. The small variant is enough; the tracing variant behaved the same, and its tracing features themselves were not exercised at runtime (size is the only thing measured for it).

Proposal

  1. Enable CONFIG_BPF_JIT, CONFIG_SCHEDSTATS, CONFIG_CRYPTO_USER_API_HASH and CONFIG_NETKIT, hand-edited in place the way Enable CIFS/SMB kernel support in arm64 config #681 and Unset CONFIG_VSOCKETS_LOOPBACK in kernel config #797 were: +68 KB, and enough for the Cilium agent. PR attached, both arches (only arm64 built and tested).
  2. Decide separately on the two heavier items Cilium also lists: CONFIG_BPF_EVENTS (+4.4 MB with its tracing dependencies; needed by kprobe- and tracepoint-based BPF programs such as Tetragon) and CONFIG_DEBUG_INFO_BTF (needs pahole in kernel/image and flips DEBUG_INFO_NONE; needed by CO-RE tooling like bpftool and libbpf programs). Neither was needed for Cilium 1.19.4 here, so they fit a follow-up with measured boot-time cost, or a documented custom-build fragment.

Code of Conduct

  • I agree to follow this project's Code of Conduct

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions