Converting sample histogram to floathistogram

[rishabhkumar92]

Hi Team,

I am working on a use case where I need to translate sample histogram which is returned via client_golang response to floathistogram. Is there a way to achieve that? Sample histogram is part of response of v1 query/query_range APIs.

cc: @beorn7

[beorn7]

I don't think the model.SampleHistogram is used in Prometheus's v1 query/query_range APIs. Where do you see that?

The floathistogram link above points to low level chunk structures of the TSDB. Are you sure you want to create those?

[beorn7]

Now I have realized that model.SampleHistogram is used in the API client.

OK, so you are using that API client to retrieve a native histogram. What are you doing with it then? As said, it would be rather weird if you are going to create a TSDB histogram chunk with it.

[rishabhkumar92]

Thanks @beorn7 for taking a look. I am using a open source project Promxy to query multiple mimir clusters and aggregate the results. Since Promxy is running prom engine itself so it needs the reverse conversion (SampleHistogram -> FloatHistogram)

I have implemented translation here.

Tl;DR I am deriving schema, spans and bucket values from HistogramBuckets using the following methodology:

• Schema can be figured out from jump in the bucket
• ZeroThreshold and ZeroCount can be figure out from the bucket which has boundaries on either side of zero
• Span and Bucket values can be figured out from placement of buckets and their corresponding Count

[beorn7]

Gotcha… Yeah, so I would say that is actually a quite hard task. The Prometheus query API intentionally "denormalizes" the histogram to just return explicit buckets and no internals like schemas and spans.

With the introduction of non-exponential schemas (like NHCB, which is currently being worked on), it will become even harder to reverse-engineer the original schema.

I don't think that this kind of conversion has been done before. (Maybe the query-frontend component in systems like Mimir, Thanos, or Cortex is doing something similar. But maybe they don't go all the way back to the PromQL engine for further aggregation.)

I would recommend to talk to the Promxy developers to find the best course of action.