Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rules from Amazon Managed Service for Prometheus data source do not show up in Grafana alerting view, like they do for Prometheus core data source #251

Open
VermaPriyanka opened this issue Oct 2, 2024 · 6 comments

Comments

@VermaPriyanka
Copy link

What happened:
When an Amazon Managed Service for Prometheus endpoint is configured as a data source using Prometheus core plugin, the rules/alerts defined in Amazon Managed Service for Prometheus ruler configuration are displayed in Grafana Alerting "Alert Rules" page. However, same rules/alerts do not show up when the data source is configured using the new Amazon Managed Service for Prometheus plugin.

What you expected to happen:
When "Manage alerts via Alerting UI" is toggled on, the rules/alerts from an Amazon Managed Service for Prometheus data source (using this plugin) are displayed in the Grafana alerting UI.

How to reproduce it (as minimally and precisely as possible):

  1. Configure an Amazon Managed Service for Prometheus data source using Prometheus core plugin, toggle "Manage alerts via Alerting UI". Go to Alerting UI, and check for alerts/rules from this data source.
  2. Configure an Amazon Managed Service for Prometheus data source using Amazon Managed Service for Prometheus plugin, toggle "Manage alerts via Alerting UI". Go to Alerting UI, and check for alerts/rules from this data source.
  3. Alerts show up from 1 and not from 2.

Environment:

  • Grafana version: 11.2
  • Plugin version: 1.0.1
@kevinwcyu
Copy link
Contributor

Hi @VermaPriyanka, I was able to create an alerting rule using the Amazon Managed Service for Prometheus plugin with the "Manage alerts via Alerting UI" toggle enabled, and view it from the "Alert rules" page (see the screenshot below).

Image

I ran a fresh copy of Grafana with docker using the command docker run --rm -p 3000:3000 -e GF_INSTALL_PLUGINS=grafana-amazonprometheus-datasource grafana/grafana-enterprise:11.2.0. Is there some other configuration that is being set?

@kevinwcyu kevinwcyu moved this from Incoming to Waiting in AWS Datasources Oct 8, 2024
@VermaPriyanka
Copy link
Author

The snapshot above shows a Grafana managed alert against an Amazon Managed Service for Prometheus data source configured with the Amazon Managed Service for Prometheus plugin. This issue is for data-source managed alerts, alert rules defined in Amazon Managed Service for Prometheus and viewable in Grafana. These are viewable with the Prometheus core plugin.

Using Prometheus core plugin:
Image

Using Amazon Managed Service for Prometheus plugin:
Image

@aholovin-smtc
Copy link

Hello @kevinwcyu,
We seem to be facing a similar issue and noticed that the PR with the proposed fix was closed due to a lack of activity or reviews. I was wondering if there are any plans to revisit or resume work on it?
Thank you for your time and consideration!

@kevinwcyu
Copy link
Contributor

Hi @aholovin-smtc, we plan to revisit the approach for it. There are instances where we need to be able treat the Prometheus and Amazon Prometheus data source interchangeably in a generic way.

@aholovin-smtc
Copy link

Thank you for the response, @kevinwcyu.
Are there any predictions on when this functionality might be integrated?
We are currently using the Prometheus core data source in Grafana Cloud as a workaround, but there is a deprecation notice for the SigV4 authentication method.
At some point, it could stop working, and we might lose visibility of alert rules in Grafana.

@kevinwcyu
Copy link
Contributor

Hi @aholovin-smtc, we currently don't have a timeline for this yet. The SigV4 auth in the core Prometheus data source won't be removed before we have this functionality working with the Amazon Managed Prometheus data source though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Waiting
3 participants