why could the promql function "rate" be parallelized?

[wanghaao]

example: sum(rate(metric[1m]))

The result of running this function at different shards should be different from the result of running at one node.

[pracucci]

sum(rate(metric[1m])) is the sum of the rate of all series matching the labels matcher metric (which translates to {__name__="metric"}. The sum() can be parallelised, so it gets parallelised as (example with 3 shards):

sum(
  sum(rate(metric{shard="1"}[1m]))
  sum(rate(metric{shard="2"}[1m]))
  sum(rate(metric{shard="3"}[1m]))
)

You can learn more here:
https://grafana.com/docs/mimir/latest/operators-guide/architecture/query-sharding/

[wanghaao]

Thanks!
Does query-sharding be based on shuffle-sharding?
In my opinion, when all time-series of one timeline are stored in one specific shard, the sharding operation of sum(rate(metric[1m])) is correct.

[pracucci]

Does query-sharding be based on shuffle-sharding?

Query sharding and shuffle sharding unrelated. They can both co-exist, but they're a different thing. Query sharding is about sharding the execution of a query, partitioning the workload. Shuffle sharding is about deciding which replica (in a Mimir cluster) should run the workload for a given tenant.

From query sharding perspective, doesn't make any difference which replica runs the query or hold series in ingester (except by the fact that to execute the query, the querier will have to lookup the right ingesters and store-gateways, according to how shuffle sharding is configured).

[wanghaao]

For the distributor component, the sharding process will distribute different time-series to different ingesters and tenant blocks.
Take metric{ instance="a" } for example, this time-line will be stored into different blocks which represent the different retention periods.
From the query perspective, sum(rate(metric[1m])) will be parallelised as shard="1", shard="2", shard="3". However, for the time-line of metric{ instance="a" }, the data maybe hold by different shards, then the result of parallelized execution on different shard is not correct. The correct execution of irate function must be based on the complete data of one time-line.

[pracucci]

I'm not sure I can follow you. If the question is whether query sharding returns correct results for the sum(rate(metric[1m])), then yes, query sharding returns correct results :)

[wanghaao]

Thank you.