Skip to content

Commit

Permalink
clean up api
Browse files Browse the repository at this point in the history
  • Loading branch information
schurhammer committed Nov 3, 2023
1 parent a696be3 commit c38c1e1
Show file tree
Hide file tree
Showing 3 changed files with 54 additions and 59 deletions.
34 changes: 14 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,45 +8,39 @@ A library for benchmarking gleam code.
# How To

```rust
import gleamy_bench.{Bench, BenchTime, Function, IPS, Input, Min, P, run, table}

// ..

Bench(
bench.run(
[
bench.Input("pre-sorted list", list.range(1, 100_000)),
bench.Input("reversed list", list.range(1, 100_000) |> list.reverse),
],
[
Input("n=5", 5),
Input("n=10", 10),
Input("n=15", 15),
bench.Function("list.sort()", sort_int)
],
[
Function("fib1", fib1),
Function("fib2", fib2),
bench.Duration(1000),
bench.Warmup(100)
],
)
|> run([BenchTime(500)])
|> table([IPS, Min, P(99)])
)
|> bench.table([bench.IPS, bench.Min, bench.P(99)])
|> io.println()

```

A benchmark is defined by giving a list of inputs and a list of functions to run on those inputs. Each input + function combination will be timed.

The inputs should all be the same type, and the functions should all accept that type as the only argument. The return type of the function does not matter, only that they all return the same type.
The given inputs should all be the same type, and the functions should all accept that type as the only argument.

The `run` function actually runs the benchmark and collects the results. It accepts a list of options to change default behaviour, for example `BenchTime(100)` can be used to change how long each function is run repeatedly when collecting results (in milliseconds).
The `run` function actually runs the benchmark and collects the results. It also accepts a list of options to change default behaviour, for example `Duration(1000)` is used to change how long each function is run repeatedly when collecting results (in milliseconds). This list is optional and can be empty if you have no need to change the defaults.

The `table` function makes a table out of the results. You can choose the list of statistics you would like to include in the table.

The output for this example looks like the following.

```
Input Function IPS Min P99
n=5 fib1 2236277.3002 0.0002 0.0006
n=5 fib2 2493122.7461 0.0002 0.0006
n=10 fib1 750561.7961 0.0010 0.0022
n=10 fib2 2755751.7477 0.0002 0.0005
n=15 fib1 80833.4127 0.0102 0.0184
n=15 fib2 2139409.1371 0.0003 0.0007
pre-sorted list list.sort() 37.8532 22.4190 31.3593
reversed list list.sort() 34.0101 27.0734 31.0618
```

## Contributing
Expand Down
39 changes: 23 additions & 16 deletions src/gleamy_bench.gleam
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,6 @@ pub type Set {
Set(input: String, function: String, reps: List(Float))
}

pub type Bench(a, b) {
Bench(inputs: List(Input(a)), functions: List(Function(a, b)))
}

pub type Stat {
P(Int)
IPS
Expand Down Expand Up @@ -112,38 +108,49 @@ fn repeat_until(duration: Float, value: a, fun: fn(a) -> b) {
}

pub type Option {
WarmupTime(ms: Int)
BenchTime(ms: Int)
Warmup(ms: Int)
Duration(ms: Int)
Quiet
}

type Options {
Options(warmup_time: Int, bench_time: Int)
Options(warmup: Int, duration: Int, quiet: Bool)
}

fn default_options() -> Options {
Options(warmup_time: 50, bench_time: 500)
Options(warmup: 50, duration: 500, quiet: False)
}

fn apply_options(default: Options, options: List(Option)) -> Options {
case options {
[] -> default
[x, ..xs] ->
case x {
WarmupTime(ms) -> apply_options(Options(..default, warmup_time: ms), xs)
BenchTime(ms) -> apply_options(Options(..default, bench_time: ms), xs)
Warmup(ms) -> apply_options(Options(..default, warmup: ms), xs)
Duration(ms) -> apply_options(Options(..default, duration: ms), xs)
Quiet -> apply_options(Options(..default, quiet: True), xs)
}
}
}

pub fn run(bench: Bench(a, b), options: List(Option)) -> List(Set) {
pub fn run(
inputs: List(Input(a)),
functions: List(Function(a, b)),
options: List(Option),
) -> List(Set) {
let options = apply_options(default_options(), options)
use Input(input_label, input) <- list.flat_map(bench.inputs)
use function <- list.map(bench.functions)
use Input(input_label, input) <- list.flat_map(inputs)
use function <- list.map(functions)
case function {
Function(fun_label, fun) -> {
io.println("benching set " <> input_label <> " " <> fun_label)
let _warmup = repeat_until(int.to_float(options.warmup_time), input, fun)
let timings = repeat_until(int.to_float(options.bench_time), input, fun)
case options.quiet {
True -> Nil
False -> {
io.println("benching set " <> input_label <> " " <> fun_label)
}
}
let _warmup = repeat_until(int.to_float(options.warmup), input, fun)
let timings = repeat_until(int.to_float(options.duration), input, fun)
Set(input_label, fun_label, timings)
}
}
Expand Down
40 changes: 17 additions & 23 deletions src/gleamy_bench_example.gleam
Original file line number Diff line number Diff line change
@@ -1,31 +1,25 @@
import gleamy_bench.{Bench, BenchTime, Function, IPS, Input, Min, P, run, table}
import gleamy_bench as bench
import gleam/io
import gleam/int
import gleam/list

fn fib1(n: Int) -> Int {
case n {
0 -> 0
1 -> 1
n -> fib1(n - 1) + fib1(n - 2)
}
}

fn do_fib2(a, b, n) {
case n {
0 -> a
_ -> do_fib2(b, a + b, n - 1)
}
}

fn fib2(n: Int) -> Int {
do_fib2(0, 1, n)
fn sort_int(data) {
list.sort(data, int.compare)
}

pub fn main() {
Bench(
[Input("n=5", 5), Input("n=10", 10), Input("n=15", 15)],
[Function("fib1", fib1), Function("fib2", fib2)],
bench.run(
[
bench.Input("pre-sorted list", list.range(1, 100_000)),
bench.Input(
"reversed list",
list.range(1, 100_000)
|> list.reverse,
),
],
[bench.Function("list.sort()", sort_int)],
[bench.Duration(1000), bench.Warmup(100)],
)
|> run([BenchTime(100)])
|> table([IPS, Min, P(99)])
|> bench.table([bench.IPS, bench.Min, bench.P(99)])
|> io.println()
}

0 comments on commit c38c1e1

Please sign in to comment.