Add Alertmanager datasource tips

[seaurching]

I started mimir by command:

./mimir --target=all,alertmanager

And the service's status is running

Alertmanager's Ring Status is:

I load the alertmanager.conf with mimirtool, show configs,like this

./Downloads/mimirtool-darwin-amd64 alertmanager get --id=miyamoto.local --address=http://127.0.0.1:8080
 cat alertmanager.conf
global:
  resolve_timeout: 2m
  smtp_smarthost: '[email protected]:587'
  smtp_from: '[email protected]'
  smtp_require_tls: true
  smtp_auth_username: '[email protected]'
  smtp_auth_password: 'password'
route:
  group_by: ['alertname', 'cluster', 'service']
  group_wait: 30s
  group_interval: 5m
  repeat_interval: 3h
  receiver: team-prometheus-mails
receivers:
- name: 'team-prometheus-mails'
  email_configs:
  - to: '[email protected]'
---
# template
{{ define "wechat.down.message" }}
{{ range .Alerts }}
= Instance or Sevice Down =
Level: {{ .Labels.severity }}
Type: {{ .Labels.alertname }}
Host: {{ .Labels.instance }}
Service: {{ .Labels.name }}
Details: {{ .Annotations.description }}
----------------------------
{{ end }}
{{ end }}

When add an datasource for alertmanager, show this message. For more information see below

It looks like you have chosen a Mimir or Cortex implementation, but detected a Prometheus endpoint. Please update implementation selection and try again.

[dimitarvdimitrov]

What URL are you using for the datasource? I see you are reaching the alertmanager on localhost. Is that the same for the datasource? Try omitting /alertmanager from the datasource URL.

[seaurching]

@dimitarvdimitrov I updated the discuss. Could you give me some sugguestions?

[dimitarvdimitrov]

looking at the code that does these checks, I think we've confused grafana. If you have marked the implementation as Mimir, it expects that /alertmanager/api/v2/status works and /api/v2/status doesn't work. But you have also added the /alertmanager suffix. So when it tries the /api/v2/status endpoint, the actual request goes to https://192.168.3.56:8080/alertmanager/api/v2/status

Did you also try without the /alertmanager suffix? Reading the code, that should mitigate it.

[gillesdemey]

I've also had another look at the implementation and when using Mimir's Alertmanager, omitting the /alertmanager suffix makes Grafana accept the AM data source – tested with a local Mimir setup.

[seaurching]

@gillesdemey Are your settings working on?

[seaurching]

@gillesdemey @dimitarvdimitrov why delete alertmanager's directory when load alertmanager.yaml by mimirtool

level=info ts=2022-08-24T09:27:59.609581907Z caller=state_replication.go:202 component=MultiTenantAlertmanager user=WEB-06 msg="Waiting for notification and silences to settle..."
level=info ts=2022-08-24T09:27:59.609658419Z caller=state_replication.go:206 component=MultiTenantAlertmanager user=WEB-06 msg="skipping settling (no replicas)"
level=info ts=2022-08-24T09:27:59.683617028Z caller=multitenant.go:1048 component=MultiTenantAlertmanager msg="deleted local directory for user" dir=/home/soft/mimir/data/alertmanager/alertmanager user=alertmanager
level=info ts=2022-08-24T09:28:14.606656574Z caller=multitenant.go:484 component=MultiTenantAlertmanager msg="synchronizing alertmanager configs for users"
level=info ts=2022-08-24T09:28:14.607470119Z caller=multitenant.go:608 component=MultiTenantAlertmanager msg="deactivating per-tenant alertmanager" user=WEB-06
level=info ts=2022-08-24T09:28:14.628151738Z caller=multitenant.go:610 component=MultiTenantAlertmanager msg="deactivated per-tenant alertmanager" user=WEB-06

[dimitarvdimitrov]

Are you seeing any problems with the alertmanager tenant? Is it possible that its config was deleted in the object store manually by you?

Also possible that the alertmanager directory was not an actual tenant and was found in the local storage and just cleaned up because it doesn't belong to an active tenant

[seaurching]

Are you seeing any problems with the alertmanager tenant? Is it possible that its config was deleted in the object store manually by you?

Also possible that the alertmanager directory was not an actual tenant and was found in the local storage and just cleaned up because it doesn't belong to an active tenant

No, the config is auto delete by itself, not by me

[seaurching]

#2698 Here is my mimir configurations

[pracucci]

I think the configuration is conflicting. Please see my comment in the other discussion:
#2698 (reply in thread)