Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "vm-operator-ca" #1213

Closed
AurimasNav opened this issue Jan 8, 2025 · 1 comment

Comments

@AurimasNav
Copy link

Hi,
trying to deploy vm helm chart victoria-metrics-k8s-stack 0.33.2 via kustomize + argocd

A lot of resources are failing to apply with errors similar to this:

Internal error occurred: failed calling webhook "vmagent.victoriametrics.com": failed to call webhook: Post "https://victoria-metrics-k8s-stack-victoria-metrics-operator.victoriametrics.svc:9443/validate-operator-victoriametrics-com-v1beta1-vmagent?timeout=10s": tls: failed to verify certificate: x509: certificate signed by unknown authority (possibly because of "crypto/rsa: verification error" while trying to verify candidate authority certificate "vm-operator-ca")

log in operator pod:

{"level":"error","ts":"2025-01-08T10:05:57Z","logger":"manager","msg":"Reconciler error","controller":"vmsingle","controllerGroup":"operator.victoriametrics.com","controllerKind":"VMSingle","VMSingle":{"name":"victoria-metrics-k8s-stack","namespace":"victoriametrics"},"namespace":"victoriametrics","name":"victoria-metrics-k8s-stack","reconcileID":"a09a216a-1524-455b-9b32-06a99ef8d10d","error":"cannot patch finalizers for object=\"operator.victoriametrics.com/v1beta1, Kind=VMSingle\" with name=\"victoria-metrics-k8s-stack\": Internal error occurred: failed calling webhook \"vmsingle.victoriametrics.com\": failed to call webhook: Post \"https://victoria-metrics-k8s-stack-victoria-metrics-operator.victoriametrics.svc:9443/validate-operator-victoriametrics-com-v1beta1-vmsingle?timeout=10s\": tls: failed to verify certificate: x509: certificate signed by unknown authority (possibly because of \"crypto/rsa: verification error\" while trying to verify candidate authority certificate \"vm-operator-ca\")"}
2025/01/08 10:06:33 http: TLS handshake error from 10.42.0.148:40954: remote error: tls: bad certificate

reproduce:

deploy helm chart with default values (except for grafana dependency which should be irrelevant)

kustomization.yaml

namespace: victoriametrics

helmGlobals:
  chartHome: charts
helmCharts:
- name: victoria-metrics-k8s-stack
  version: 0.33.2
  repo: https://victoriametrics.github.io/helm-charts
  releaseName: victoria-metrics-k8s-stack
  includeCRDs: true
  namespace: victoriametrics
  valuesFile: values.yaml
@AurimasNav
Copy link
Author

I've nuked the deployment, forcing deletion of all resources by removing finalizers and it seems to have deployed fine now.
Must note though that it seems it always requires some manual intervention with removing finalizers whenever new helm version is being deployed in argocd scenario at least.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant