Repository navigation
feat(registry): add Prometheus protobuf exposition - #860
ethanolchik wants to merge 1 commit into
Conversation
Signed-off-by: Ethan Olchik <eitan.olchik@gmail.com>
krajorama
left a comment
There was a problem hiding this comment.
looking good, a few comments
| | OpenMetricsContentType | ||
| | PrometheusProtobufContentType; | ||
|
|
||
| export type RegistryMetrics<T extends RegistryContentType> = |
There was a problem hiding this comment.
Changing the constuctor seemed suspicious, I asked LLM and LLM says this and the constructor change is breaking the API, suggests this fix and text:
diff --git a/index.d.ts b/index.d.ts
@@ -36,14 +36,19 @@ export type RegistryContentType =
| OpenMetricsContentType
| PrometheusProtobufContentType;
-export type RegistryMetrics<T extends RegistryContentType> =
- T extends PrometheusProtobufContentType ? Uint8Array : string;
+// Not distributive, so a Registry whose content type is not known statically
+// keeps returning a string, as it did before protobuf was supported.
+export type RegistryMetrics<T extends RegistryContentType> = [T] extends [
+ PrometheusProtobufContentType,
+]
+ ? Uint8Array
+ : string;
/**
* Container for all registered metrics
*/
export class Registry<
- BoundRegistryContentType extends RegistryContentType = PrometheusContentType,
+ BoundRegistryContentType extends RegistryContentType = RegistryContentType,
> {
diff --git a/test/protobufTypes.ts b/test/protobufTypes.ts
@@ -52,7 +52,11 @@ const openMetrics: Promise<string> = new Registry(
Registry.OPENMETRICS_CONTENT_TYPE,
).metrics();
const unknownFormat: Registry<RegistryContentType> = registry;
-const unknownBody: Promise<string | Uint8Array> = unknownFormat.metrics();
+const unknownBody: Promise<string> = unknownFormat.metrics();
+const plainOpenMetrics: Registry = new Registry(
+ Registry.OPENMETRICS_CONTENT_TYPE,
+);
+const plainProtobuf: Registry = registry;
void [
binary,
oneText,
@@ -63,6 +67,8 @@ void [
text,
openMetrics,
unknownBody,
+ plainOpenMetrics,
+ plainProtobuf,
];
|
|
||
| // Classic histogram aggregation can produce fractional counts. Protobuf has | ||
| // dedicated double fields for them; uint64 encoding would truncate them. | ||
| function encodeHistogramCount(message, field) { |
There was a problem hiding this comment.
This is technically correct and the parser on Prometheus side can process the output which contains a mix of integer and float (bucket) counts.
There is one subtle problem when it comes to aggregated histograms: if the output flip-flops between integer only and not integer only histograms, the NHCB conversion on the Prometheus side may flip-flop between integer and float native histograms with custom buckets, which would in turn cut new chunk on every flip and flop, which can be quite bad for storage. It would be better if the aggregation decided up front what type to use and for example stick to floats for average. Probably a niche problem, but we should add an issue once this PR lands to keep track.
Note: similarly PromQL always produces float histograms, since it avoids having this problem in recording rules - and makes it simpler for the end-user to process the results.
| !Number.isInteger(group.summary.sampleCount) | ||
| ) { | ||
| throw new TypeError( | ||
| `Prometheus protobuf requires an integer sample count for summary ${metric.name}`, |
There was a problem hiding this comment.
This fails the whole scrape , right? So using average aggregation is basically useless :(
Well, actually the average is wrong for the quantiles anyway see #225.
Could we consider rounding the result and logging that we did this instead of failing? WDYT @jdmarshall ?
I'll ask the Prometheus team if they would be ok adding a float version of the count field like we have for histograms.
| }, | ||
| "dependencies": { | ||
| "@opentelemetry/api": "^1.4.0", | ||
| "protobufjs": "^8.8.0", |
There was a problem hiding this comment.
This is plus 3.9MB, but not all of it is used. Is this a problem @jdmarshall ?
One could extract encoding/deconfig from this lib possibly?
| const key = JSON.stringify(labelPairs(labels)); | ||
| let group = groups.get(key); | ||
| if (!group) { | ||
| group = { label: labelPairs(labels), [metric.type]: {} }; |
There was a problem hiding this comment.
labelPairs is called twice, can we reduce?
Summary
Add length-delimited Prometheus protobuf exposition for the existing metric types.
This adds:
Registry.PROMETHEUS_PROTOBUF_CONTENT_TYPEand the exported content-type constant.Registry.metrics()output for protobuf registries while keeping theAsStringmethods text-only.This intentionally does not add native histogram collection; that is stacked separately in #856.
Testing