fix: add kube-vip toleration to match node taint in harvester-cloud-provider

[Yu-Jack]

Description

After setting up guest cluster with this configuration (control-plane + ETCD separate from worker in different pool) mentioned in Test Plan.

When you create load balancer type service with harvester cloud provider annotation, the load balancer type service EXTERNAL_IP is stuck in pending status

After discussion in harvester/harvester#4678, the reason is kube-vip toleration doesn't matched any node taint.

Solution

Add one more toleration in kube-vip DaemonSet. It should works in all cases listed in Test Plan.

Bump PR is #203

Issue

harvester/harvester#4678

Test Plan

Common set up

1. We choose harvester cloud provider.
   
2. Set up cloud-config in each node for later usage because we need to reinstall harvester-cloud-provider manually

write_files:
  - encoding: b64
    content: {harvester's kube config, the cluster namespace should be same as the pool you created (base64 enconded)}
    owner: root:root
    path: /etc/kubernetes/cloud-config
    permission: '0644'

3. After pools are created, we remove the harvester-cloud-provider in Apps > Installed Apps (kube-system namesapce).
   
4. We add new charts in Apps > Repositories.
   Use https://charts.harvesterhci.io to install and select 0.2.3.
5. Go to Apps > Charts to install harvester-cloud-provider

Different Guest Cluster Pool

1. control-plane, ETCD and worker in same pool
2. control-plane and ETCD in A pool, worker in B pool
3. control-plane in A pool, ETCD in B pool and worker in C pool

After install harvester-cloud-provider, DaemonSet kube-system/kube-vip should run in above three cases in guest cluster. Then create following load balance type service to check EXTERNAL_IP is assigned or not.

p.s. check helloworldserver-lb service in the guest cluster, GUI and command are both okay.

# example svc
apiVersion: v1
kind: Service
metadata:
  name: helloworldserver-lb
  annotations:
    cloudprovider.harvesterhci.io/ipam: dhcp
    cloudprovider.harvesterhci.io/healthcheck-port: "80"
spec:
  type: LoadBalancer
  selector:
    app: web
  ports:
    - name: test
      protocol: TCP
      port: 8088
      targetPort: 80

[Yu-Jack]

If I need to bump version, that means I need to open another PR to release branch?

[bk201]

Can we skip the tarball? I believe the content is the same, just timestamp is different.

[Yu-Jack]

@bk201 Actually, the tarball is different from previous version. You could extract the tar to compare. Only difference is the DaemonSet toleration.

[bk201]

Thanks, If the tarball content is different, should we bump its version?

[Yu-Jack]

Thanks, If the tarball content is different, should we bump its version?

@bk201 Yeah, after checking, I think we need to bump its version. But, I'm not sure how do you update the gh-pages branch index.html. I run helm repo index harvester-cloud-provider, however, its result seems different from gh-pages branch index.html. Did I miss something?

Okay, it uses chart-releaser-action to release.

[Yu-Jack]

BTW, it seems I need to open another PR to release branch for bumping the harvester-cloud-provider version to 0.2.3, right?

[bk201]

LGTM, thanks!

[bk201]

Let's try overriding kube-vip's toleration in cloud-provider's value file:

charts/charts/harvester-cloud-provider/values.yaml

Line 73 in eaf3568

kube-vip:

With this way, I guess we don't need to bump the dependency chart.

[Yu-Jack]

@bk201 After re-run template with only modifying the value.yaml of parents, and it seems well.

And test case is successful.

[Yu-Jack]

override references: Helm | Subcharts and Global Values

[w13915984028]

Just planned to fix this issue, but #202 and #203 have already been ready for merge. thanks @Yu-Jack

@bk201 Let's merge those 2 PR, then I will rebase rancher/charts#3316 with v0.2.3

[w13915984028]

LGTM, thanks.