Skip to content

feat(helm): add a ServiceMonitor for the controller metrics endpoint - #2401

Open
younsl wants to merge 1 commit into
kagent-dev:mainfrom
younsl:feat/controller-servicemonitor
Open

feat(helm): add a ServiceMonitor for the controller metrics endpoint#2401
younsl wants to merge 1 commit into
kagent-dev:mainfrom
younsl:feat/controller-servicemonitor

Conversation

@younsl

@younsl younsl commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Description

This adds native ServiceMonitor support to the kagent Helm chart so controller metrics are collected by Prometheus Operator out of the box.

This is the follow up that #1803 deferred. That PR landed the metrics Service and the scrape RBAC and closed #1369, but left the ServiceMonitor out of scope. #1802 carried one and was closed as a duplicate of #1803, so the ServiceMonitor half of #1369 was never reviewed on its own and no template for it exists in the chart today. This PR is additive on top of what #1803 merged and changes none of it.

Today controller.metrics.enabled provisions the metrics Service and the ClusterRoles for authenticated scrapes, but not the resource that makes Prometheus actually scrape them. Every user running kube-prometheus-stack has to hand write a ServiceMonitor outside the chart and keep its port name and namespace in sync with the chart by hand. The bundled kagent-tools subchart already ships one, so this closes the gap for the controller.

New values live under controller.metrics.serviceMonitor, disabled by default and gated on controller.metrics.enabled. Interval, scrape timeout, honorLabels, relabelings, metric relabelings, extra labels, annotations and a namespace override are exposed. With secure serving on, the endpoint defaults to https with the ServiceAccount token and insecureSkipVerify, which matches the self signed certificate the controller serves, and both are overridable.

The metrics Service port name moved into a shared helper used by both the Service and the ServiceMonitor endpoint, so the two cannot drift when secure serving is flipped. Rendered output for existing installs is unchanged.

Changelog

Added an optional Prometheus Operator ServiceMonitor for the controller metrics endpoint, configurable under controller.metrics.serviceMonitor.

Testing

  1. helm unittest helm/kagent passes with 298 tests, 15 of them new in helm/kagent/tests/controller-servicemonitor_test.yaml.
  2. Nothing renders unless both controller.metrics.enabled and controller.metrics.serviceMonitor.enabled are set, and the bindAddress disable sentinel still suppresses it.
  3. Rendered the secure and plaintext variants with helm template, then validated them against a live cluster with kubectl apply --dry-run=server so the manifest is checked against the real ServiceMonitor CRD rather than only the chart.

Additional Notes

Rendering the ServiceMonitor does not authorize the scrape on its own. With secure serving, the metrics reader ClusterRole still has to be bound to the Prometheus ServiceAccount, which the values comment calls out.

The chart does not gate on the monitoring.coreos.com API being present, matching what kagent-tools does. The explicit opt in flag is the gate, so a missing CRD fails loudly at install time instead of silently dropping the resource during helm template.

controller.metrics.enabled provisions the metrics Service and the
ClusterRoles for authenticated scrapes, but stops short of the resource
that makes Prometheus Operator actually scrape it, so every user running
kube-prometheus-stack has to hand-write a ServiceMonitor outside the
chart. The bundled kagent-tools subchart already ships one.

Adds controller.metrics.serviceMonitor, off by default and gated on
controller.metrics.enabled. Scrape settings (interval, scrapeTimeout,
honorLabels, relabelings, metricRelabelings), extra labels/annotations
and a namespace override are exposed; the endpoint defaults to https
with the ServiceAccount token and insecureSkipVerify when secureServing
is on, matching the self-signed certificate the controller serves, and
both are overridable.

The Service port name moves into a kagent.controller.metricsPortName
helper shared with the ServiceMonitor endpoint so the two cannot drift
when secureServing is flipped. Rendered output is unchanged.

Signed-off-by: younsl <cysl@kakao.com>
@younsl
younsl requested a review from a team as a code owner August 7, 2026 12:44
@github-actions github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Helm Chart: Add support for enabling metrics endpoint and optional ServiceMonitor

1 participant