metrics: expose cluster identity via cluster_info gauge

[tmgstevens]

Motivation

Nothing in a Prometheus scrape uniquely identifies which cluster a broker belongs to. Users who want to distinguish clusters in dashboards, recording rules, or federated setups must inject external_labels in every scrape config, which is error-prone and often skipped. The bootstrap cluster UUID (immutable, created once at cluster formation, not user-editable — unlike the cluster_id config) is the trustworthy identity, so expose it directly.

What this does

Adds a constant info-style gauge to both metrics endpoints, following the kube_state_metrics *_info convention (join via group_left in PromQL):

redpanda_cluster_info{cluster_uuid="<bootstrap uuid>"} 1     # /public_metrics
vectorized_cluster_info{cluster_uuid="<bootstrap uuid>"} 1   # /metrics

An info metric was chosen over stamping the UUID onto every series: per-series labels would change the identity of every time series on upgrade (breaking dashboards and recording rules), bloat scrape payloads, and fight with external labels that observability pipelines already add.

Design notes

• The cluster UUID is not yet known when setup_metrics() runs (it is loaded from the kvstore / replicated via the bootstrap command only after storage starts), so registration happens in a background fiber that waits on storage::api::wait_for_cluster_uuid(): immediate on restart, right after bootstrap on a brand-new cluster. Until then the metric is simply absent — a node that has not joined a cluster has no cluster identity.
• Follows the metrics/instance_metrics.cc precedent: metrics::all_metrics_groups for dual-endpoint registration (honors both disable flags), ssx::spawn_with_gate for the fiber, gate closed via _deferred at shutdown (application::shutdown() breaks the UUID waiters as its first step, so the fiber always unblocks).

Bug fix (first commit, independently backportable)

storage::api::set_cluster_uuid() used condition_variable::signal(), which wakes exactly one waiter. Multiple independent fibers can be blocked in wait_for_cluster_uuid() on the same shard — the cloud metadata uploader (cluster/cloud_metadata/uploader.cc) and the datalake iceberg committer (datalake/coordinator/iceberg_file_committer.cc) — so on a freshly bootstrapped cluster the losing fiber slept forever (e.g. cluster metadata uploads silently never starting for the life of the process). This is a latent pre-existing race that the new waiter here would have made more likely; fixed with broadcast().

Testing

• New ducktape test ClusterIdentityMetricsTest.test_cluster_info_metric: 3-broker cluster, asserts the metric appears on both endpoints of every broker and the label matches the admin API's GET /v1/cluster/uuid.
• MetricsTest.test_aggregate_metrics now excludes cluster_info from its line counts, since the async registration could otherwise change the count between scrapes.
• Developed on a machine without Bazel: relying on CI for build, clang-format, and the rptest type checker. Python formatted/linted with the repo-pinned ruff 0.12.10.

Backports Required

• none - not a bug fix
• none - this is a backport
• none - issue does not exist in previous branches
• none - papercut/not impactful enough to backport
• v26.1.x (first commit only — the signal()→broadcast() fix)
• v25.3.x (first commit only — the signal()→broadcast() fix)
• v25.2.x

Release Notes

Improvements

• The bootstrap cluster UUID is now exposed on both metrics endpoints as a constant cluster_info gauge with a cluster_uuid label, so Prometheus scrapes can uniquely identify the cluster without external labels.

🤖 Generated with Claude Code

https://claude.ai/code/session_01E7sDG1E1hom3tZ2nd3FH2e

[tmgstevens]

/ci-repeat 1