Adding test utils to reset metrics

[tillprochaska]

I have started instrumenting a Python app I’m maintaining. To ensure correctness of the collected metrics I’ve added a few basic unit tests.

As metrics are typically defined as module globals, testing them in isolation requires resetting them before each test. I’ve been using the following as part of the setup hook in my test cases which seems to work as expected:

collectors = REGISTRY._collector_to_names.keys()
for collector in collectors:
    if isinstance(collector, MetricWrapperBase):
        collector._metrics.clear()
        collector._metric_init()

The only alternative I’ve come across so far seems to be storing the initial value of a metric at the beginning of a test, then asserting the difference is as expected.

class MyTestCase(TestCase):
    def test_worker_task_success(self):
        labels = {queue: "my_queue"}
        counter_before = REGISTRY.get_sample_value("worker_task_succeeded_total", labels)
        # Init worker and process tasks
        counter_after = REGISTRY.get_sample_value("worker_task_succeeded_total", labels)
    
        assert (counter_after - counter_before) == 1

This can get quite verbose and complex, especially when you need to test multiple metrics or different permutations of label values. Ideally, I’d like to be able to do something like this:

from prometheus_client import REGISTRY
from prometheus_client.test_utils import reset_registry

class MyTestCase(TestCase):
    def setup_method(self, _):
        reset_registry(REGISTRY)

    def test_worker_task_success(self):
        # Init worker and process tasks
        counter = REGISTRY.get_sample_value("worker_task_succeeded_total", {"queue": "my_queue"})
        assert counter == 1

    def test_worker_task_failure(self):
        # ...

There are multiple GitHub issues discussing similar features. I understand that adding methods to reset metrics to the core API would probably add confusion and might result in people using these methods outside of unit tests, violating Prometheus conventions.

But would you be open to add test helpers like this to a separate, clearly named module as a convenience? I’d be happy to open a PR to contribute this.

For starters, I think methods like reset_registry(registry: CollectorRegistry) and reset_metric(metric: MetricWrapperBase) would be helpful.