chore: add codspeed for benchmark

[dqhl76]

Which issue does this PR close?

Closes #4937 .

Rationale for this change

Trying to add a benchmark workflow to track and compare performance

What changes are included in this PR?

A new workflow that runs on every PR and the default main branch.
The criterion dependency is changed to codspeed-criterion-compat.
This should not affect cargo bench

Are there any user-facing changes?

No

---

Wait for approval from infra team https://issues.apache.org/jira/projects/INFRA/issues/INFRA-26030?filter=allissues

[Xuanwo]

Please load from 1password.

[dqhl76]

Will use 1password after I get the token. Thanks for the review.

[Xuanwo]

Hi, @adriencaccia, OpenDAL has a benchmark suite that can run tests over different services based on the environment. Does Codspeed support our case? For example, running Cargo Codspeed multiple times can display their performance in groups based on service.

[dqhl76]

[adriencaccia]

Hi, @adriencaccia, OpenDAL has a benchmark suite that can run tests over different services based on the environment. Does Codspeed support our case? For example, running Cargo Codspeed multiple times can display their performance in groups based on service.

Hey, at the moment there is no support for grouping benchmarks.
What I would recommend in the meantime, if relevant, is to manually change the name of the benchmarks depending on the service.
For example, for a benchmark myBench in the codebase, you could make its name depending on the service, so that multiple benchmarks are tracked on CodSpeed: service1-myBench, service2-myBench, service3-myBench...

[Xuanwo]

Hi, @adriencaccia, there are some concerns from the ASF INFRA, do you have comments?

[adriencaccia]

Hi, @adriencaccia, there are some concerns from the ASF INFRA, do you have comments?

Hey, I just answered in apache/arrow-rs#6150 (comment) 😉

[Xuanwo]

Hey, I just answered in apache/arrow-rs#6150 (comment) 😉

Thanks!