Benchmarks with strange behavior

[LilithHafner]

This thread is for realistic benchmarks where the "right answer" is unclear, highly system dependent, very different for benchmarktools vs. chairmarks, and/or has other strange behaviors. I'm gathering examples of benchmark results that baffle me. This does not include obvious bugs.

• Discourse (15/3/24) @be (Δp=1, Δa=10) _.Δp/2sind(_.Δa/2) strangest behavior reported here, though plain chairmarks-benchmarktools discrepancy is also present on some systems.

• benchmark results oscillate #79 (9/3/24) v = zero(SVector{14,Int16}); @b $v == $v has weird statefull behavior, effects both Chairmarks and BenchmarkTools

• Does not suppress allocation & GC time as much as BenchmarkTools #78 (9/3/24) @b rand(100, 10000, 100) has GC-mediated statefull behavior.