Flannel from VXLAN to host-gw

[gallart]

Environmental Info:
K3s Version:
k3s version v1.21.0+k3s1 (2705431)
go version go1.16.2

Node(s) CPU architecture, OS, and Version:
Centos 7 x64

Cluster Configuration:
3 servers, 0 agents

Describe the bug:
After boostraping a 3 nodes cluster and deploying some workload, I want to change flannel backend from default VXLAN to host-gw.

I followed instructions from #538 and restarted k3s service on all nodes after changing systemd but I have the feeling that this is not working.

How can I check that host-gw is working fine?

Steps To Reproduce:

• Installed K3s:
• restart service with --flannel-backend=host-gw

Expected behavior:
Flannel should switch to host-gw

Actual behavior:
Unknown

[brandond]

In general it's not a good idea to switch out major bits of your cluster networking configuration (CNI plugin, CNI backend, subnets, etc) once the cluster is already up and running. You may be able to resolve this by rebooting all the nodes in order to clear the old interfaces and routes, but if you continue to run into problems after rebooting all the nodes with the new flag set, you're probably best off rebuilding the cluster.

[cchangwen]

curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" INSTALL_K3S_EXEC="server --disable=traefik --node-ip 10.136.252.221 --flannel-backend=host-gw" sh -

TOKEN=$(ssh 10.136.252.221 cat /var/lib/rancher/k3s/server/node-token)
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="agent --server https://10.136.252.221:6443 --token $TOKEN --node-ip 10.136.252.219" sh -

TOKEN=$(ssh 10.136.252.221 cat /var/lib/rancher/k3s/server/node-token)
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="agent --server https://10.136.252.221:6443 --token $TOKEN --node-ip 10.136.252.220" sh -

New Installation. Why is k3s not managing the routes?
I need to manually add some routes to make the network work correctly.

ip route add 10.42.0.0/24 via 10.136.252.221
ip route add 10.42.1.0/24 via 10.136.252.219
ip route add 10.42.2.0/24 via 10.136.252.220

[brandond]

Flannel should do that for you. However, a 3 year old discussion thread on an unrelated topic is not the correct place to ask this question. If you are having a problem, please open a new issue.

[cchangwen]

In some cases (Servers on DigitalOcean, whitch has Two network interfaces), k3s/flannel may not determine the correct interface, it chooses a public IP as the node ip and causes other problems.

[cchangwen]

with --flannel-iface

curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE="644" INSTALL_K3S_EXEC="server --disable=traefik --flannel-iface eth1 --flannel-backend=host-gw" sh -

TOKEN=$(ssh 10.136.252.221 cat /var/lib/rancher/k3s/server/node-token)
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="agent --server https://10.136.252.221:6443 --token $TOKEN --flannel-iface eth1" sh -