Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support the quantile for response time #257

Open
u5surf opened this issue Jan 4, 2023 · 4 comments
Open

Support the quantile for response time #257

u5surf opened this issue Jan 4, 2023 · 4 comments
Labels

Comments

@u5surf
Copy link
Collaborator

u5surf commented Jan 4, 2023

It should be important for the module users to fetch the response time quantile for various use cases.
I consider that it is suitable for using libgkc which is the library within the high performance http server h2o
https://github.com/h2o/h2o/tree/master/deps/libgkc

According this code, it follows the algorithm which is written in this paper. Thus it is efficiency spatial complexity for that calculate.
https://www.cis.upenn.edu/~sanjeev/papers/sigmod01_quantiles.pdf

@u5surf u5surf changed the title for response time Support the quantile for response time Jan 4, 2023
@sboisson
Copy link

There are other histogram implementations/structures that seems worth considering:

@u5surf
Copy link
Collaborator Author

u5surf commented Jan 14, 2024

@sboisson
Thanks advice so much.
I consider that HdrHistogram is more understandability about its algorithm.
But we guess it is suitable the pseudo-quantile algorithm which is called by e-approximate p-Quantile (e.g. ddsketch, CKMS, etc...) if we're willing to save more space of the memory.
At least, GK seems to be less accuracy than other pseudo-quantile algorithms.

CKMS:
https://grafana.com/blog/2022/03/01/how-summary-metrics-work-in-prometheus/

@cohalz
Copy link
Contributor

cohalz commented Jan 22, 2024

If you are using Prometheus, you can calculate quantiles by implementing Prometheus' Native Histogram (OpenTelemetry's Exponential Histogram).
In addition to the advantage of compressing the amount of data, there are almost no settings for the user.

@SuperQ
Copy link
Collaborator

SuperQ commented Jan 22, 2024

Agreed, you definitely want to use something like Prometheus native histograms. Directly exposing quantiles like the older Prometheus "Summary" metrics is not useful as the data can't be aggregated over multiple instances.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants