[query] Benchmark configurations

[ehigham]

Change Description

Added benchmark parameters to all benchmark tests in the Hail codebase. Each test now specifies:

• Minimum detectable slowdown (mds)
• Number of instances (ie number of batch jobs)
• Number of iterations (how many timed trials of the benchmark per instance)
• Number of burn-in iterations (number of untimed trials run before timing)

These parameters allow for more precise and controlled benchmark measurements by explicitly defining the statistical requirements and execution parameters for each test.

Security Assessment

This change has no security impact

Impact Description

Low-level test configuration changes only - modifies benchmark test parameters without changing any functional code or test logic.

[ehigham]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• [query] Benchmark configurations #14762 👈 (View in Graphite)
• [benchmark] notebook for interactively computing benchmark config #14757
• [query] Various Benchmark Suite Improvements #14761
• [query] make local and remote tmp settable on backend #14748
• [query] Lowering + Optimisation with implict timing context #14731 : 1 other dependent PR (#14732 )
• [query] Lift backend state into {Service|Py4J}BackendApi #14698
• [query] Move is.hail.check to test code #14801 : 1 other dependent PR (#14693 )
• [query] Move LoweredTableReaderCoercer into ExecuteContext #14696
• [query] Remove persistedIr from Backend interface #14692
• [query] Remove lookupOrCompileCachedFunction from Backend interface #14691
• [query] Remove BlockMatrix persist from Backend interface #14690
• [query] Expose references via ExecuteContext #14686
• [query] Extract Backend Methods called from Python into Py4JBackendExtensions #14767
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.