The kube-flannel pod on the worker node stays in CrashLoopBackOff status.

[lengcangche-gituhub]

I use kubeadm join to add a worker node. However, the flannel pod on the worker node stays in CrashLoopBackOff status.

Expected Behavior

The flannel pod becomes Running

Current Behavior

The flannel pod on the worker node stays in CrashLoopBackOff status

Steps to Reproduce (for bugs)

1.On the master node:

root@NPU-Atlas-2:/home/lincom# kubeadm init --pod-network-cidr=100.100.0.0/16 --image-repository=registry.aliyuncs.com/google_containers --apiserver-advertise-address=192.168.1.122
root@NPU-Atlas-2:/home/lincom# mkdir -p $HOME/.kube
root@NPU-Atlas-2:/home/lincom# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
root@NPU-Atlas-2:/home/lincom# sudo chown $(id -u):$(id -g) $HOME/.kube/config
root@NPU-Atlas-2:/home/lincom# kubectl apply -f kube-flannel.yml

2.On the worker node:

kubeadm join 192.168.1.122:6443 --token 2ydxw7.y64x3rl3d2g4fsxh --discovery-token-ca-cert-hash sha256:9e3a2259e1c0d2a3bf0abcd6e344c5f65c7324cb58900d251f2305d4d16e7273

3.On the master node:

root@NPU-Atlas-2:/home/lincom# kubectl get pods --all-namespaces
NAMESPACE      NAME                                   READY   STATUS              RESTARTS        AGE
default        kubernetes-bootcamp-666cf565fc-97sbb   0/1     ContainerCreating   0               4m39s
kube-flannel   kube-flannel-ds-2tszk                  0/1     CrashLoopBackOff    5 (2m44s ago)   5m46s
kube-flannel   kube-flannel-ds-mkhst                  1/1     Running             0               9m54s
...................
root@NPU-Atlas-2:/home/lincom# kubectl describe pods/kube-flannel-ds-2tszk -n kube-flannel
.............................
Events:
  Type     Reason   Age                    From     Message

  Warning  BackOff  32m (x416 over 122m)   kubelet  Back-off restarting failed container kube-flannel in pod kube-flannel-ds-2tszk_kube-flannel(b97155bd-b848-4272-88d4-0e5fa2f89706)
  Normal   Pulled   28m                    kubelet  Container image "swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/flannel/flannel-cni-plugin:v1.4.1-flannel1-linuxarm64" already present on machine
  Normal   Created  28m                    kubelet  Created container install-cni-plugin
  Normal   Started  28m                    kubelet  Started container install-cni-plugin
  Normal   Pulled   28m                    kubelet  Container image "swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/flannel/flannel:v0.25.1-linuxarm64" already present on machine
  Normal   Created  28m                    kubelet  Created container install-cni
  Normal   Started  28m                    kubelet  Started container install-cni
  Normal   Pulled   27m (x4 over 28m)      kubelet  Container image "swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/flannel/flannel:v0.25.1-linuxarm64" already present on machine
  Normal   Created  27m (x4 over 28m)      kubelet  Created container kube-flannel
  Normal   Started  27m (x4 over 28m)      kubelet  Started container kube-flannel
  Warning  BackOff  3m31s (x115 over 28m)  kubelet  Back-off restarting failed container kube-flannel in pod kube-flannel-ds-2tszk_kube-flannel(b97155bd-b848-4272-88d4-0e5fa2f89706)

root@NPU-Atlas-2:/home/lincom# kubectl -n kube-flannel logs kube-flannel-ds-2tszk
Defaulted container "kube-flannel" out of: kube-flannel, install-cni-plugin (init), install-cni (init)

Context

I need a working pod network.

Your Environment

• Flannel version:
  flannel:v0.25.1-linuxarm64
  flannel-cni-plugin:v1.4.1-flannel1-linuxarm64
• Backend used (e.g. vxlan or udp): vxlan
• Etcd version:
• Kubernetes version (if used):
  Client Version: v1.28.14
  Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
  Server Version: v1.28.14
• Operating System and version:
  $ cat /etc/os-release
  NAME="Ubuntu"
  VERSION="20.04 LTS (Focal Fossa)"
  ID=ubuntu
  ID_LIKE=debian
  PRETTY_NAME="Ubuntu 20.04 LTS"
  VERSION_ID="20.04"
  HOME_URL="https://www.ubuntu.com/"
  SUPPORT_URL="https://help.ubuntu.com/"
  BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
  PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
  VERSION_CODENAME=focal
  UBUNTU_CODENAME=focal
  $ uname -a
  Linux FT-D2000-2-1 4.19.15 Create a subnet allocator mode #5 SMP Tue Sep 24 09:29:10 CST 2024 aarch64 aarch64 aarch64 GNU/Linux

[rbrtbnfgl]

I saw that you are using ARM64. Is only the failing node that is using arm or both?

[flucas1]

I have to use this during the cluster initialization

---
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
controllerManager:
  extraArgs:
    - name: allocate-node-cidrs
      value: "true"
    - name: cluster-cidr
      value: "10.244.0.0/16"
    - name: node-cidr-mask-size
      value: "24"
kubernetesVersion: stable
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16
scheduler: {}

kubeadm init --v=6 --config=<config.yaml>

[lengcangche-gituhub]

I saw that you are using ARM64. Is only the failing node that is using arm or both?

Both are ARM64

[lengcangche-gituhub]

I have to use this during the cluster initialization

---
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
controllerManager:
  extraArgs:
    - name: allocate-node-cidrs
      value: "true"
    - name: cluster-cidr
      value: "10.244.0.0/16"
    - name: node-cidr-mask-size
      value: "24"
kubernetesVersion: stable
networking:
  dnsDomain: cluster.local
  podSubnet: 10.244.0.0/16
scheduler: {}

kubeadm init --v=6 --config=<config.yaml>

I think your configuration is the same as mine.

[rbrtbnfgl]

Are you sure that you don't have any logs from the failing pod?

[Clifforid]

I have the same problem.

[root@k8s-master-01 ~]# kubectl logs -f kube-flannel-ds-42k6p -c install-cni -n kube-flannel
Error from server (BadRequest): container "install-cni" in pod "kube-flannel-ds-42k6p" is waiting to start: PodInitializing
[root@k8s-master-01 ~]# kubectl logs kube-flannel-ds-42k6p -c install-cni-plugin -n kube-flannel
exec /bin/cp: exec format error
[root@k8s-master-01 ~]# kubectl get pod --all-namespaces
NAMESPACE      NAME                                    READY   STATUS                  RESTARTS        AGE
kube-flannel   kube-flannel-ds-42k6p                   0/1     Init:CrashLoopBackOff   9 (2m44s ago)   23m
kube-flannel   kube-flannel-ds-9z7s5                   0/1     Init:CrashLoopBackOff   9 (2m38s ago)   23m
kube-flannel   kube-flannel-ds-n74gg                   0/1     Init:CrashLoopBackOff   9 (2m56s ago)   23m