-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Suggestion to Follow BenchmarkTools More Closely #100
Comments
Thanks for bringing this up! All of these departures are intentional, the rest of this comment is both a response to you and a draft for inclusion in https://chairmarks.lilithhafner.com/stable/explanations for future reference. I'm open to discussion on all of these points. When there are conflicts between compatibility/alignment with Below, I'll list some specific design departures and why I made them
Chairmarks uses the abbreviated macros
If you are writing a script that computes some values and want to display those values to the user, you generally have to call display. Chairmarks in not an exception. If it were possible, I would consider special-casing
No Julia objects that I know of The fact that BenchmarkTools.jl's short display mode ( julia> @b 1+1
1.644 ns
julia> @btime 1+1;
1.083 ns (0 allocations: 0 bytes)
julia> @b rand(10)
110.621 ns (2 allocs: 144 bytes)
julia> @btime rand(10);
45.037 ns (2 allocations: 144 bytes)
julia> @b rand(10_000_000)
11.445 ms (3 allocs: 76.294 MiB, 5.43% gc time)
julia> @btime rand(10_000_000);
8.843 ms (3 allocations: 76.29 MiB)
julia> @b @eval 1+1
88.375 μs (51 allocs: 1.828 KiB)
julia> @btime @eval 1+1;
34.501 μs (47 allocations: 1.72 KiB)
julia> @b @eval begin f(x) = x+1; f(1) end
1.328 ms (863 allocs: 38.766 KiB, 56.93% compile time)
julia> @btime @eval begin f(x) = x+1; f(1) end;
1.402 ms (859 allocations: 38.66 KiB)
julia> @time @b sleep(1)
1.269034 seconds (38.04 k allocations: 1.944 MiB, 20.59% compilation time)
1.002 s (4 allocs: 112 bytes, without a warmup)
julia> @time @btime sleep(1)
1.002 s (4 allocations: 112 bytes)
16.438018 seconds (27.50 k allocations: 1.341 MiB, 24.21% gc time, 2.16% compilation time) It would be a loss restrict ourselves to only runtime and allocations, it would be distracting to include "0% compilation time" in all of these outputs, and it would be inconsistent to make some fields (e.g. allocation count and amount) always display while others are only displayed when non-zero. Sparse display is the compromise I've chosen to get the best of both worlds. |
Thanks for your quick response! I completely understand there are trade offs, that's why I labelled them as suggestions. I'd still insist on the importance of adding "0 allocations". If you see a line like 11.445 ms could be easily confused with some option that only provides the time, without indicating any other information. Many thanks for such a great package! It's saving me tons of time in my workflow. |
I prefer following Nevertheless, I'm going to solicit other opinions on this matter of opinion :) |
FWIW, I initially agreed with 1., 2., and 3. with @alfaromartino, reconsidering after the great response of @LilithHafner I totally dropped 1. and 2., but I actually need to say that |
What sort of longer names do folks want? A. I prefer B or C I would implement this by adding long names as an option and then maybe deprecating the use of short names in scripts. I do not plan to deprecate the use of short names at the REPL. |
I prefer the human generated C. and I also like the pun ;) |
Didn't know that specific names/macros could be exclusive to REPL. Keeping In that case, I think that F. would be a good option (or Some other suggestions with the help of AI would In any case, I think C or F are fine. |
Just a couple of suggestions for this superb blazingly fast package! In summary, I propose following
BenchmarkTools
more closely regarding layout.Thanks for your work!
@btime
and is also consistent when there's at least one allocation with @b. ExampleIt might seem trivial, but it becomes important when you have several lines of results, you're new to the package, and for teaching.
display
by default as BenchmarkTools to reduce boilerplate code. Right now, to get the same results when you execute a whole code snippet, we need to explicitly adddisplay
to see the result. Example:@b
for a more explicit name? I can imagine that reading code with@b
without any context calls for confusion.The text was updated successfully, but these errors were encountered: