Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardize exporter error count metric in exporterhelper #11960

Open
harsh8398 opened this issue Dec 19, 2024 · 0 comments
Open

Standardize exporter error count metric in exporterhelper #11960

harsh8398 opened this issue Dec 19, 2024 · 0 comments

Comments

@harsh8398
Copy link

Is your feature request related to a problem? Please describe.

The exporterhelper module in the otel collector does not currently expose metrics that provide visibility into errors encountered during data export. This limits observability into exporter health and can make it challenging to diagnose and address issues in production environments.

Today the exporter health we are interested in are specific to HTTP. For example:

  • If the exporter is not able to reach the destination, it can be classified as a HostNotFound error.
  • If there is HTTP error status code in response, it can be mapped to the most appropriate error class. Such as,
    • 400 -> InvalidRequest
    • 403 -> ConnectionForbidden
    • 401 -> NotAuthenticated or NotAuthorized
    • 404 -> ResourceNotFound

Describe the solution you'd like

Here is our stack setup:

  • We use persistent queue with exporterhelper retry
    • We never drop data when exporters export with errors
    • Retries are indefinite until it succeeds
  • We do not use batch processor
  • We have different receivers coming in with data and different exporters going out with data

We want to expose a set of common HTTP errors via exporterhelper so each exporter implementation can choose which one to return. Then exporterhelper would instrument when it receives these errors.

Expose error types from exporterhelper packages:

  • HostNotFound
  • ConnectionForbidden
  • NotAuthenticated
  • ResourceNotFound
  • NotAuthorized
  • InvalidRequest
  • Unclassified

Introduce a new metric in exporterhelper with the following specification:

exporter_error_count
"""Shows the number of errors that have occurred when exporting data to a data destination"""

Customizable dimensions: (instance, destinationId)

Dimension: errorType
Possible Values (Can be extended):
1. HostNotFound
2. ConnectionForbidden
3. NotAuthenticated
4. ResourceNotFound
5. NotAuthorized
6. InvalidRequest
7. Unclassified

This metric will help users monitor exporter health, identify failure patterns, and address issues proactively.

Describe alternatives you've considered

Alternative is each exporter does its own thing before error is returned. This approach introduces a lot of duplication codes across http based exporters. Errors are also non-standardized.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant