You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
kernel/config-arm64 (and config-x86_64) is the documented way to build a custom kernel for container, and apple/container discussion #1673 asks for Cilium in container k8s clusters and links Cilium's requirements page. Cilium's documented base requirements (https://docs.cilium.io/en/stable/operations/system_requirements/#base-requirements):
Five are not enabled in config-arm64 on main (the same five in config-x86_64):
Option
State today
Note
CONFIG_BPF_JIT
absent
the Cilium agent needs a JIT
CONFIG_SCHEDSTATS
# ... is not set
one-line flip
CONFIG_CRYPTO_USER_API_HASH
# ... is not set
one-line flip
CONFIG_BPF_EVENTS
absent
needs FTRACE and KPROBES/KPROBE_EVENTS, otherwise olddefconfig drops it
CONFIG_DEBUG_INFO_BTF
absent (DEBUG_INFO_NONE=y)
needs pahole (dwarves) in kernel/image
Everything else on the list, including the tunneling, netfilter, L7 and WireGuard sets, is already =y. CONFIG_NETKIT=y (Cilium's netkit device mode, optional) is also absent. For comparison, the kernel container recommends today (Kata 3.32.0 debug, 6.18.35) has BPF_JIT, BPF_EVENTS and DEBUG_INFO_BTF but not SCHEDSTATS, NETKIT or NETFILTER_XT_MATCH_SOCKET. A Cilium 1.19.4 agent starts on it but keeps failing to install its proxy iptables rules (-m socket --transparent). This request is only about kernel/config-arm64.
Measured. I built linux-6.18.5 (the version kernel/Makefile pins) three times from config-arm64, same toolchain for all three (gcc 13.3 in ubuntu:24.04, not the gcc 9.4 kernel/image toolchain the file was generated with, so absolute sizes will differ slightly from an in-tree build; the deltas should not):
+ the above and BPF_EVENTS with FTRACE, KPROBES, KPROBE_EVENTS
33,769,984
+4,415,488 (+15.0%)
make olddefconfig on the hand-edited small variant adds BPF_JIT_DEFAULT_ON=y and EXECMEM=y and surfaces # CONFIG_BPF_JIT_ALWAYS_ON is not set; nothing else moves relative to the regenerated baseline. The tracing variant additionally pulls in TRACING, EVENT_TRACING, TRACEPOINTS, RING_BUFFER, KRETPROBES, UPROBES/UPROBE_EVENTS, EPROBE_EVENTS, DYNAMIC_EVENTS, NOP_TRACER, CONTEXT_SWITCH_TRACER, GLOB, TRACE_CLOCK (all tracers stay off).
Functional check, both variants, on container CLI 1.0.0: booted with container run --kernel (bpf_jit_enable=1, /proc/schedstat present, an AF_ALG hash socket binds, ip link add ... type netkit succeeds where 6.12.28 and 6.18.35 both say "Unknown device type"), then a two-node kubeadm 1.37 cluster on kindest/node with Cilium 1.19.4 via cilium install --wait: cilium status OK with 2/2 agents, nodes Ready, and from a curl pod ClusterIP, pod IP and cluster DNS answer 200, including pod-to-pod across nodes with cilium-dbg status reporting Routing: Tunnel [vxlan]. The small variant is enough; the tracing variant behaved the same, and its tracing features themselves were not exercised at runtime (size is the only thing measured for it).
Decide separately on the two heavier items Cilium also lists: CONFIG_BPF_EVENTS (+4.4 MB with its tracing dependencies; needed by kprobe- and tracepoint-based BPF programs such as Tetragon) and CONFIG_DEBUG_INFO_BTF (needs pahole in kernel/image and flips DEBUG_INFO_NONE; needed by CO-RE tooling like bpftool and libbpf programs). Neither was needed for Cilium 1.19.4 here, so they fit a follow-up with measured boot-time cost, or a documented custom-build fragment.
Feature or enhancement request details
kernel/config-arm64(andconfig-x86_64) is the documented way to build a custom kernel forcontainer, and apple/container discussion #1673 asks for Cilium incontainer k8sclusters and links Cilium's requirements page. Cilium's documented base requirements (https://docs.cilium.io/en/stable/operations/system_requirements/#base-requirements):Five are not enabled in
config-arm64onmain(the same five inconfig-x86_64):CONFIG_BPF_JITCONFIG_SCHEDSTATS# ... is not setCONFIG_CRYPTO_USER_API_HASH# ... is not setCONFIG_BPF_EVENTSFTRACEandKPROBES/KPROBE_EVENTS, otherwise olddefconfig drops itCONFIG_DEBUG_INFO_BTFDEBUG_INFO_NONE=y)pahole(dwarves) inkernel/imageEverything else on the list, including the tunneling, netfilter, L7 and WireGuard sets, is already
=y.CONFIG_NETKIT=y(Cilium's netkit device mode, optional) is also absent. For comparison, the kernelcontainerrecommends today (Kata 3.32.0 debug, 6.18.35) hasBPF_JIT,BPF_EVENTSandDEBUG_INFO_BTFbut notSCHEDSTATS,NETKITorNETFILTER_XT_MATCH_SOCKET. A Cilium 1.19.4 agent starts on it but keeps failing to install its proxy iptables rules (-m socket --transparent). This request is only aboutkernel/config-arm64.Measured. I built linux-6.18.5 (the version
kernel/Makefilepins) three times fromconfig-arm64, same toolchain for all three (gcc 13.3 in ubuntu:24.04, not the gcc 9.4kernel/imagetoolchain the file was generated with, so absolute sizes will differ slightly from an in-tree build; the deltas should not):Imagebytesconfig-arm64as-isBPF_JIT,SCHEDSTATS,CRYPTO_USER_API_HASH,NETKITBPF_EVENTSwithFTRACE,KPROBES,KPROBE_EVENTSmake olddefconfigon the hand-edited small variant addsBPF_JIT_DEFAULT_ON=yandEXECMEM=yand surfaces# CONFIG_BPF_JIT_ALWAYS_ON is not set; nothing else moves relative to the regenerated baseline. The tracing variant additionally pulls inTRACING,EVENT_TRACING,TRACEPOINTS,RING_BUFFER,KRETPROBES,UPROBES/UPROBE_EVENTS,EPROBE_EVENTS,DYNAMIC_EVENTS,NOP_TRACER,CONTEXT_SWITCH_TRACER,GLOB,TRACE_CLOCK(all tracers stay off).Functional check, both variants, on container CLI 1.0.0: booted with
container run --kernel(bpf_jit_enable=1,/proc/schedstatpresent, anAF_ALGhash socket binds,ip link add ... type netkitsucceeds where 6.12.28 and 6.18.35 both say "Unknown device type"), then a two-node kubeadm 1.37 cluster on kindest/node with Cilium 1.19.4 viacilium install --wait:cilium statusOK with 2/2 agents, nodes Ready, and from a curl pod ClusterIP, pod IP and cluster DNS answer 200, including pod-to-pod across nodes withcilium-dbg statusreportingRouting: Tunnel [vxlan]. The small variant is enough; the tracing variant behaved the same, and its tracing features themselves were not exercised at runtime (size is the only thing measured for it).Proposal
CONFIG_BPF_JIT,CONFIG_SCHEDSTATS,CONFIG_CRYPTO_USER_API_HASHandCONFIG_NETKIT, hand-edited in place the way Enable CIFS/SMB kernel support in arm64 config #681 and Unset CONFIG_VSOCKETS_LOOPBACK in kernel config #797 were: +68 KB, and enough for the Cilium agent. PR attached, both arches (only arm64 built and tested).CONFIG_BPF_EVENTS(+4.4 MB with its tracing dependencies; needed by kprobe- and tracepoint-based BPF programs such as Tetragon) andCONFIG_DEBUG_INFO_BTF(needspaholeinkernel/imageand flipsDEBUG_INFO_NONE; needed by CO-RE tooling likebpftooland libbpf programs). Neither was needed for Cilium 1.19.4 here, so they fit a follow-up with measured boot-time cost, or a documented custom-build fragment.Code of Conduct