Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: facing challenges with recently changed prometheus metrics naming convention #5601

Closed
2 tasks done
tayalrishabh96 opened this issue Jul 31, 2024 · 1 comment · Fixed by #5608
Closed
2 tasks done
Assignees
Labels
bug Something isn't working pager-duty Bugs / Issues found while on pager duty SRE Reported SRE Reported issues

Comments

@tayalrishabh96
Copy link
Contributor

📜 Description

Not receiving any alerts due to recent change in prometheus metrics naming convention.
earlier : nats_event_consumption_time_count
now : Nats_Event_Consumption_Time_count
Due to this change we are not receiving any alerts and also this is changed only for all streams except image-scanner due to which we will need to create segregated alerts for image-scanner NATS metrics.

Affected areas

Other CRITICAL functionality

Additional affected areas

Other CRITICAL functionality

Prod/Non-prod environments?

Prod

Is User unblocked?

No

How was the user un-blocked?

None

Impact on Enterprise

Unable to track anomalies in NATS like nats event consumption time etc.

👟 Steps to replicate the Issue

if you plot a graph in grafana over nats_event_consumption_time_count then you will see the data stopped coming after a certain time duration.

👍 Expected behavior

metrics naming convention should be same across all exposed metrics.

👎 Actual Behavior

Currently all metrics naming convention is lower case except for NATs which is camel case.

☸ Kubernetes version

EKS 1.30

Cloud provider

AWS

🌍 Browser

Chrome

✅ Proposed Solution

NA

👀 Have you spent some time to check if this issue has been raised before?

  • I checked and didn't find any similar issue

🏢 Have you read the Code of Conduct?

@tayalrishabh96 tayalrishabh96 added bug Something isn't working pager-duty Bugs / Issues found while on pager duty labels Jul 31, 2024
Copy link

Final Score: 240

@github-actions github-actions bot removed the pager-duty Bugs / Issues found while on pager duty label Jul 31, 2024
@tayalrishabh96 tayalrishabh96 added the pager-duty Bugs / Issues found while on pager duty label Jul 31, 2024
@prakarsh-dt prakarsh-dt added the SRE Reported SRE Reported issues label Aug 2, 2024
@prkhrkat prkhrkat reopened this Aug 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working pager-duty Bugs / Issues found while on pager duty SRE Reported SRE Reported issues
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants