Percentile calculations seem to be off

[fivenp]

Describe the bug

Having these 6 execution times:

5.34
5.75
5.93
6.16
6.56
11.56

Testkube tells me my P95 is 6.56

Whereas the actual P95 (even P85!) is actually 11.56
Multiple percentile calculators throw out the same result

The real issue here is the message/assumption Testkube is sending to users "Everything is fine" – where in reality there is really an issue with the executions because P95 actually is DOUBLE P50

[scott-david-walker]

I just looked a little bit into this and found that small data sets on the quantile package seems to be inaccurate (never actually queries the percentile, just returns the value at an index of the samples.

I've made an issue about this on the repo but based on the last open issues and pr being ignored I wonder if they're maintaining it.

As a workaround, we could force a flush of the stream therefore forcing the internal func query (the actual percentile logic) to be called by doing

    q.Merge(q.Samples())

which will give more accuracy but sacrificing performance of the library (according to the comments in stream.go).

Unit tests confirm that this works but it also feels a little hacky.

Would appreciate advice before going forward.

Cheers

[vsukhin]

hey @scott-david-walker Feel free to open PR with porposed changes. We will check it out. Or might be you can sugest to switch to more accurate library?