Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Azure EventHub receiver is failing #37385

Open
zojovano opened this issue Jan 21, 2025 · 3 comments
Open

Azure EventHub receiver is failing #37385

zojovano opened this issue Jan 21, 2025 · 3 comments
Labels
bug Something isn't working receiver/azureeventhub

Comments

@zojovano
Copy link

zojovano commented Jan 21, 2025 •
edited by atoulme
Loading

Component(s)

receiver/azureeventhub

What happened?

Description

I am receiving following error after collector is started:

error   [email protected]/eventhubhandler.go:150   Error reported by event hub     {"kind": "receiver", "name": "azureeventhub", "data_type": "metrics", "error": "context canceled"}
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/azureeventhubreceiver.(*eventhubHandler).setUpOnePartition.func1    
        github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/eventhubhandler.go:150

EventHub has 10 partitions.

Steps to Reproduce

Configure collector to use eventhub receiver and azure data explorer exporter

Expected Result

Collector reading logs and metrics from EventHub and sending to ADX via the exporter.

Actual Result

Receiving errors in the Collector logs and no logs or metrics are sent to ADX.

Collector version

0.17

Environment information

Environment

OS:Windows

OpenTelemetry Collector configuration

# To limit exposure to denial of service attacks, change the host in endpoints below from 0.0.0.0 to a specific network interface.
# See https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks

extensions:
  #health_check:
  #pprof:
  #  endpoint: 0.0.0.0:1777


receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317

  azureeventhub:
    connection: "Endpoint=sb://demo-observabilty.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=;EntityPath=maindiagnosticcollection"
    partition: ""
    group: $Default
    offset: ""
    format: "azure"
    # optional
    time_formats:
      # All supported time format. Default is empty string array, which means using the current iso8601 parser. The format is based on https://pkg.go.dev/time#Layout. If no time-zone info, will use UTC time.
      # logs: ["01/02/2006 15:04:05","2006-01-02 15:04:05","2006-01-02T15:04:05Z07:00"]
      # metrics: ["01/02/2006 15:04:05"]
      logs: ["2025-01-21T16:37:39.7526336Z"]  # Example time format
      metrics: ["2025-01-21T16:37:39.7526336Z"]  # Example time format


processors:
  batch:

exporters:
  debug:
    verbosity: basic
  azuredataexplorer:
    cluster_uri: "https://trd-sxwndfr8sm0vy6844c.z5.kusto.fabric.microsoft.com"
    application_id: "c84761b4-8a31-4cd9-baf9-bd6752190365"
    application_key: "<key>"
    tenant_id: "539d8bb1-bbd5-4f9d-836d-223c3e6d1e43"
    db_name: "OTELEventHouse"
    metrics_table_name: "OTELMetrics"
    logs_table_name: "OTELLogs"
    traces_table_name: "OTELTraces"
    ingestion_type : "managed"

service:

  pipelines:

    traces:
      #receivers: [otlp,azureeventhub]
      receivers: [otlp]
      processors: [batch]
      exporters: [debug,azuredataexplorer]

    metrics:
      receivers: [otlp,azureeventhub]
      #receivers: [otlp]
      processors: [batch]
      exporters: [debug,azuredataexplorer]

    logs:
      receivers: [otlp,azureeventhub]
      #receivers: [otlp]
      processors: [batch]
      exporters: [debug,azuredataexplorer]

Log output

2025-01-21T18:19:16.281+0100    error   [email protected]/eventhubhandler.go:150   Error reported by event hub     {"kind": "receiver", "name": "azureeventhub", "data_type": "metrics", "error": "context canceled"}
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/azureeventhubreceiver.(*eventhubHandler).setUpOnePartition.func1    
        github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/eventhubhandler.go:150       
2025-01-21T18:19:16.310+0100    warn    [email protected]/azureresourcemetrics_unmarshaler.go:103  Unhandled Time Grain  {"kind": "receiver", "name": "azureeventhub", "data_type": "metrics", "timegrain": ""}
2025-01-21T18:19:16.520+0100    error   [email protected]/eventhubhandler.go:150   Error reported by event hub     {"kind": "receiver", "name": "azureeventhub", "data_type": "metrics", "error": "context canceled"}
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/azureeventhubreceiver.(*eventhubHandler).setUpOnePartition.func1    
        github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/eventhubhandler.go:150       
2025-01-21T18:19:16.681+0100    error   [email protected]/eventhubhandler.go:150   Error reported by event hub     {"kind": "receiver", "name": "azureeventhub", "data_type": "metrics", "error": "context canceled"}
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/azureeventhubreceiver.(*eventhubHandler).setUpOnePartition.func1    
        github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/eventhubhandler.go:150       
2025-01-21T18:19:16.702+0100    warn    [email protected]/azureresourcemetrics_unmarshaler.go:103  Unhandled Time Grain  {"kind": "receiver", "name": "azureeventhub", "data_type": "metrics", "timegrain": ""}
2025-01-21T18:19:16.902+0100    error   [email protected]/eventhubhandler.go:150   Error reported by event hub     {"kind": "receiver", "name": "azureeventhub", "data_type": "metrics", "error": "context canceled"}
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/azureeventhubreceiver.(*eventhubHandler).setUpOnePartition.func1    
        github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/eventhubhandler.go:150       
2025-01-21T18:19:17.094+0100    error   [email protected]/eventhubhandler.go:150   Error reported by event hub     {"kind": "receiver", "name": "azureeventhub", "data_type": "metrics", "error": "context canceled"}
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/azureeventhubreceiver.(*eventhubHandler).setUpOnePartition.func1    
        github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/eventhubhandler.go:150       
2025-01-21T18:19:17.318+0100    error   [email protected]/eventhubhandler.go:150   Error reported by event hub     {"kind": "receiver", "name": "azureeventhub", "data_type": "metrics", "error": "context canceled"}
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/azureeventhubreceiver.(*eventhubHandler).setUpOnePartition.func1    
        github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/eventhubhandler.go:150       
2025-01-21T18:19:17.511+0100    error   [email protected]/eventhubhandler.go:150   Error reported by event hub     {"kind": "receiver", "name": "azureeventhub", "data_type": "metrics", "error": "context canceled"}
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/azureeventhubreceiver.(*eventhubHandler).setUpOnePartition.func1    
        github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/eventhubhandler.go:150       
2025-01-21T18:19:17.718+0100    error   [email protected]/eventhubhandler.go:150   Error reported by event hub     {"kind": "receiver", "name": "azureeventhub", "data_type": "metrics", "error": "context canceled"}
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/azureeventhubreceiver.(*eventhubHandler).setUpOnePartition.func1    
        github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/eventhubhandler.go:150       
2025-01-21T18:19:17.885+0100    error   [email protected]/eventhubhandler.go:150   Error reported by event hub     {"kind": "receiver", "name": "azureeventhub", "data_type": "metrics", "error": "context canceled"}
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/azureeventhubreceiver.(*eventhubHandler).setUpOnePartition.func1    
        github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/eventhubhandler.go:150       
2025-01-21T18:19:18.118+0100    error   [email protected]/eventhubhandler.go:150   Error reported by event hub     {"kind": "receiver", "name": "azureeventhub", "data_type": "metrics", "error": "context canceled"}
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/azureeventhubreceiver.(*eventhubHandler).setUpOnePartition.func1    
        github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/eventhubhandler.go:150

Additional context

No response
@zojovano zojovano added bug Something isn't working needs triage New item requiring triage labels Jan 21, 2025
@github-actions GitHub Actions
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@atoulme atoulme removed the needs triage New item requiring triage label Jan 21, 2025
@atoulme
Copy link
Contributor

atoulme commented Jan 21, 2025

Thank you for the detailed report. I will take a quick look to triage further.

@atoulme
Copy link
Contributor

atoulme commented Jan 21, 2025

The error emanates from the eventhub lib when it tries to read the current checkpoint of the eventhub.

Before we go any further, are you sure the name of the event hub is correct?
Your endpoint says:

demo-observabilty.servicebus.windows.net

Shouldn't it be:

demo-observability.servicebus.windows.net

Further questions for you:

  • did this work previously and stopped working?
  • Did this work or does this work with a different version of the collector?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working receiver/azureeventhub
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@atoulme @zojovano