Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

wrong host entry crashes the whole external-dns pod #1668

Closed
Nilegfx opened this issue Jul 11, 2020 · 4 comments · Fixed by #1756 · May be fixed by #4885
Closed

wrong host entry crashes the whole external-dns pod #1668

Nilegfx opened this issue Jul 11, 2020 · 4 comments · Fixed by #1756 · May be fixed by #4885
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@Nilegfx
Copy link

Nilegfx commented Jul 11, 2020 •
edited
Loading

What happened:
the whole external-dns pod fails and keep restarting if one ingress has an invalid host

error

time="2020-07-11T01:39:46Z" level=info msg="config: {Master: KubeConfig: RequestTimeout:30s IstioIngressGatewayServices:[] ContourLoadBalancerService:heptio-contour/contour SkipperRouteGroupVersion:zalando.org/v1 Sources:[ingress] Namespace: AnnotationFilter: FQDNTemplate: CombineFQDNAndAnnotation:false IgnoreHostnameAnnotation:false Compatibility: PublishInternal:false PublishHostIP:false AlwaysPublishNotReadyAddresses:false ConnectorSourceServer:localhost:8080 Provider:cloudflare GoogleProject: GoogleBatchChangeSize:1000 GoogleBatchChangeInterval:1s DomainFilter:[] ExcludeDomains:[] ZoneIDFilter:[] AlibabaCloudConfigFile:/etc/kubernetes/alibaba-cloud.json AlibabaCloudZoneType: AWSZoneType: AWSZoneTagFilter:[] AWSAssumeRole: AWSBatchChangeSize:1000 AWSBatchChangeInterval:1s AWSEvaluateTargetHealth:true AWSAPIRetries:3 AWSPreferCNAME:false AzureConfigFile:/etc/kubernetes/azure.json AzureResourceGroup: AzureSubscriptionID: AzureUserAssignedIdentityClientID: CloudflareProxied:true CloudflareZonesPerPage:50 CoreDNSPrefix:/skydns/ RcodezeroTXTEncrypt:false AkamaiServiceConsumerDomain: AkamaiClientToken: AkamaiClientSecret: AkamaiAccessToken: InfobloxGridHost: InfobloxWapiPort:443 InfobloxWapiUsername:admin InfobloxWapiPassword: InfobloxWapiVersion:2.3.1 InfobloxSSLVerify:true InfobloxView: InfobloxMaxResults:0 DynCustomerName: DynUsername: DynPassword: DynMinTTLSeconds:0 OCIConfigFile:/etc/kubernetes/oci.yaml InMemoryZones:[] OVHEndpoint:ovh-eu PDNSServer:http://localhost:8081 PDNSAPIKey: PDNSTLSEnabled:false TLSCA: TLSClientCert: TLSClientCertKey: Policy:sync Registry:txt TXTOwnerID:playground TXTPrefix: TXTSuffix: Interval:1m0s Once:false DryRun:false UpdateEvents:false LogFormat:text MetricsAddress::7979 LogLevel:info TXTCacheInterval:0s ExoscaleEndpoint:https://api.exoscale.ch/dns ExoscaleAPIKey: ExoscaleAPISecret: CRDSourceAPIVersion:externaldns.k8s.io/v1alpha1 CRDSourceKind:DNSEndpoint ServiceTypeFilter:[] CFAPIEndpoint: CFUsername: CFPassword: RFC2136Host: RFC2136Port:0 RFC2136Zone: RFC2136Insecure:false RFC2136TSIGKeyName: RFC2136TSIGSecret: RFC2136TSIGSecretAlg: RFC2136TAXFR:false RFC2136MinTTL:0s NS1Endpoint: NS1IgnoreSSL:false TransIPAccountName: TransIPPrivateKeyFile:}"
time="2020-07-11T01:39:46Z" level=info msg="Instantiating new Kubernetes client"
time="2020-07-11T01:39:46Z" level=info msg="Using inCluster-config based on serviceaccount-token"
time="2020-07-11T01:39:46Z" level=info msg="Created Kubernetes client https://10.96.0.1:443"
panic: runtime error: index out of range [1] with length 1

goroutine 1 [running]:
sigs.k8s.io/external-dns/registry.affixNameMapper.toTXTName(0x0, 0x0, 0x0, 0x0, 0xc0005aa260, 0x12, 0x1f63f01, 0xc000245680)
	/bitnami/blacksmith-sandox/external-dns-0.7.2/src/github.com/kubernetes-incubator/external-dns/registry/txt.go:241 +0x127
sigs.k8s.io/external-dns/registry.(*TXTRegistry).ApplyChanges(0xc0004e80e0, 0x28a7ae0, 0xc0003fd4d0, 0xc000114c60, 0x0, 0x0)
	/bitnami/blacksmith-sandox/external-dns-0.7.2/src/github.com/kubernetes-incubator/external-dns/registry/txt.go:141 +0x381
sigs.k8s.io/external-dns/controller.(*Controller).RunOnce(0xc0003d5710, 0x28a7ae0, 0xc0003fd4d0, 0x3b38da0, 0x2252e01)
	/bitnami/blacksmith-sandox/external-dns-0.7.2/src/github.com/kubernetes-incubator/external-dns/controller/controller.go:152 +0x38a
sigs.k8s.io/external-dns/controller.(*Controller).Run(0xc0003d5710, 0x28a7a20, 0xc0004ed4c0)
	/bitnami/blacksmith-sandox/external-dns-0.7.2/src/github.com/kubernetes-incubator/external-dns/controller/controller.go:189 +0x205
main.main()
	/bitnami/blacksmith-sandox/external-dns-0.7.2/src/github.com/kubernetes-incubator/external-dns/main.go:341 +0xd4f

What you expected to happen:
single dns creation failure

How to reproduce it (as minimally and precisely as possible):
create ingress with host an invalid host, like, subdomain-without-FQDN

Anything else we need to know?:
I solved the problem by correcting the hostname, then I am reporting it here.

Environment:

  • External-DNS version (use external-dns --version):v0.7.2-dirty
  • DNS provider: cloudflare
@Nilegfx Nilegfx added the kind/bug Categorizes issue or PR as related to a bug. label Jul 11, 2020
@jgrumboe
Copy link
Contributor

Ouch...seems to be something in my last change with adding txt-suffix feature.
I'm currently not able to debug/correct this before August.

@chemasan chemasan mentioned this issue Aug 26, 2020
Merged
1 task
@titilambert
Copy link
Contributor

Hello !
I got the same issue with a DNSendpoint but only when the record is already created:
So externalDNs create the record, then it crashes at the second run when it sees/reads the records.
I'm using external-dns v0.14.2 with --provider=webhook --txt-suffix=-%{record_type --policy=sync --registry=txt --txt-owner-id=externaldns
here is an example of what is crashing

apiVersion: externaldns.k8s.io/v1alpha1
kind: DNSEndpoint
metadata:
  creationTimestamp: "2024-08-02T01:33:09Z"
  generation: 1
  name: unifi-android-tv
  namespace: system
  resourceVersion: "3121222"
  uid: 056d3e35-dee9-4369-83c4-89f39625e3d6
spec:
  endpoints:
  - dnsName: android-tv
    recordTTL: 180tb.lt
    recordType: A
    targets:
    - 192.168.32.69
  - dnsName: android-tv.home
    recordTTL: 180
    recordType: A
    targets:
    - 192.168.32.69
status:
  observedGeneration: 1

here is the log:

time="2024-08-03T01:11:02Z" level=info msg="Instantiating new Kubernetes client"
time="2024-08-03T01:11:02Z" level=info msg="Using inCluster-config based on serviceaccount-token"
time="2024-08-03T01:11:02Z" level=info msg="Created Kubernetes client https://10.43.0.1:443"
panic: runtime error: index out of range [1] with length 1

goroutine 1 [running]:
sigs.k8s.io/external-dns/registry.affixNameMapper.toEndpointName({{0x0, 0x0}, {0x7ffd503a9766, 0xf}, {0x0, 0x0}}, {0xc001710c77?, 0x0?})
      sigs.k8s.io/external-dns/registry/txt.go:409 +0x285
sigs.k8s.io/external-dns/registry.(*TXTRegistry).Records(0xc0015fa340, {0x44c9488?, 0xc001694050?})
      sigs.k8s.io/external-dns/registry/txt.go:145 +0x414
sigs.k8s.io/external-dns/controller.(*Controller).RunOnce(0xc0015fa410, {0x44c9488, 0xc001694050})
      sigs.k8s.io/external-dns/controller/controller.go:206 +0x72
sigs.k8s.io/external-dns/controller.(*Controller).Run(0xc0015fa410, {0x44c9488, 0xc001694050})
      sigs.k8s.io/external-dns/controller/controller.go:334 +0xca
main.main()
      sigs.k8s.io/external-dns/main.go:489 +0x4ccf

@titilambert
Copy link
Contributor

CC @jgrumboe

@jgrumboe
Copy link
Contributor

@titilambert Sorry, I can't help here. I've been out of this space and using the external-dns project for 4+ years.

@titilambert titilambert mentioned this issue Nov 19, 2024
Open
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Milestone
No milestone
3 participants
@titilambert @Nilegfx @jgrumboe