Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Unable to disable kube-vip in harvester-cloud-provider chart #6527

Open
adonskoy opened this issue Sep 9, 2024 · 8 comments · May be fixed by harvester/charts#279
Open

[BUG] Unable to disable kube-vip in harvester-cloud-provider chart #6527

adonskoy opened this issue Sep 9, 2024 · 8 comments · May be fixed by harvester/charts#279
Assignees
Labels
kind/bug Issues that are defects reported by users or that we know have reached a real release reproduce/needed Reminder to add a reproduce label and to remove this one severity/needed Reminder to add a severity label and to remove this one

Comments

@adonskoy
Copy link

adonskoy commented Sep 9, 2024

Describe the bug

The chart has the kube-vip.enabled option in values, but when the value is changed to false, nothing happens, kube-vip remains deployed

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'

Expected behavior

kube-vip should not deploy when value is false

Support bundle

Environment

  • Harvester ISO version: v1.3.1
  • Underlying Infrastructure (e.g. Baremetal with Dell PowerEdge R630):

Additional context

@adonskoy adonskoy added kind/bug Issues that are defects reported by users or that we know have reached a real release reproduce/needed Reminder to add a reproduce label and to remove this one severity/needed Reminder to add a severity label and to remove this one labels Sep 9, 2024
@bk201
Copy link
Member

bk201 commented Sep 9, 2024

cc @starbops @w13915984028

@brandboat
Copy link
Contributor

Hi @adonskoy, thank you for reporting this issue. I'm curious, could you share the reason why you needed to disable kube-vip in the harvester-cloud-provider?

@adonskoy
Copy link
Author

Hi @brandboat, we encountered the following problems:

  • Stability of integration with kube-vip. In cases where the etcd leader changes in the guest cluster, kube-apiserver is restarted, then harvester-cloud-provider restarted, and for some reason it restarts kube-vip too. After which traffic to the target service stops coming until the pods of this service are recreated

  • Inability to schedule kube-vip to the worker node - changed selectors and affinity rules are ignored (also probably a problem in the chart, but I have not yet looked at what exactly the problem is). I would like to use externalTrafficPolicy: Local in services, but with kube-vip I will have to schedule the workload to the master nodes for this

At the moment we have deployed metallb, removed address pools for the built-in balancer from the harvester and everything works, but kube-vip still performs unnecessary actions, trying to reconcile these services

Also, it seems that metallb's performance is slightly higher in this setup than kube-vip's. But this is a subjective opinion, I haven't done any measurements yet.

@w13915984028
Copy link
Member

Back to the issue itself, the enabled option is not handled in the embedded kube-vip chart, we will add a fix.

https://github.com/harvester/charts/blob/4cc0524d6cbf231cecc174baa388829ed66b1796/charts/harvester-cloud-provider/values.yaml#L74

@adonskoy Thanks for reporting this issue.

@bk201
Copy link
Member

bk201 commented Sep 10, 2024

@w13915984028
@adonskoy has a PR: harvester/charts#279

@starbops
Copy link
Member

Stability of integration with kube-vip. In cases where the etcd leader changes in the guest cluster, kube-apiserver is restarted, then harvester-cloud-provider restarted, and for some reason it restarts kube-vip too. After which traffic to the target service stops coming until the pods of this service are recreated

Hi @adonskoy Regarding this one, do you have more information? We'd like to check if it's reproducible.

  • Rancher version
  • Guest cluster version
  • VM OS

Thank you.

@adonskoy
Copy link
Author

Hi @adonskoy Regarding this one, do you have more information? We'd like to check if it's reproducible.

  • Rancher version
  • Guest cluster version
  • VM OS

Thank you.

Hi @starbops, sure

  • Rancher version: 2.8.5-2.9.1
  • Guest cluster version: rke2 from 1.28 to 1.31
  • VM OS: Ubuntu 22.04 and 24.04

We had this problem on all these versions

@w13915984028 w13915984028 self-assigned this Sep 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Issues that are defects reported by users or that we know have reached a real release reproduce/needed Reminder to add a reproduce label and to remove this one severity/needed Reminder to add a severity label and to remove this one
Projects
Development

Successfully merging a pull request may close this issue.

5 participants