[benchmark] notebook for interactively computing benchmark config

[ehigham]

Change Description

Adds tools for analyzing benchmark variability and detecting minimal slowdowns when running benchmarks on Hail Batch. The analysis is based on methods from Laaber et al.'s paper on software microbenchmarking in the cloud.

Key additions:

• New Jupyter notebook for exploring benchmark variability
• Statistical analysis tools for computing confidence intervals and detecting performance changes
• Functions for determining minimal detectable slowdowns using different testing strategies
• Improved benchmark data import/export capabilities
• Enhanced visualization tools for benchmark results

Security Assessment

This change has no security impact

Impact Description

Analysis and visualization code that operates on benchmark data only, with no access to sensitive information or systems.

[ehigham]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• [query] Benchmark configurations #14762
• [benchmark] notebook for interactively computing benchmark config #14757 👈 (View in Graphite)
• [query] Various Benchmark Suite Improvements #14761
• [query] make local and remote tmp settable on backend #14748
• [query] Lowering + Optimisation with implict timing context #14731 : 1 other dependent PR (#14732 )
• [query] Lift backend state into {Service|Py4J}BackendApi #14698
• [query] Move is.hail.check to test code #14801 : 1 other dependent PR (#14693 )
• [query] Move LoweredTableReaderCoercer into ExecuteContext #14696
• [query] Remove persistedIr from Backend interface #14692
• [query] Remove lookupOrCompileCachedFunction from Backend interface #14691
• [query] Remove BlockMatrix persist from Backend interface #14690
• [query] Expose references via ExecuteContext #14686
• [query] Extract Backend Methods called from Python into Py4JBackendExtensions #14767
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.