Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

loser-tree: add sequence abstraction #376

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from
Draft

Conversation

bboreham
Copy link
Contributor

@bboreham bboreham commented Sep 5, 2023

What this PR does:

Give the loser tree package a sequence abstraction.
Having the implementation limited to slices limits the extent to which this package can be re-used.
This PR is one step to unifying the implementation between dskit and Loki.

#281, where loser tree was added to dskit, stated that this was to achieve a 15% performance improvement, but benchmarks on this change give a mixed picture, with some going 5% faster and some 7% slower.

Profiling shows that on my machine 60% of the time goes in initializing random numbers, which could be cached.

Checklist

  • Tests updated
  • [?] CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]
Benchmark results
goos: darwin
goarch: arm64
pkg: github.com/grafana/dskit/ring
                                                                         │    before    │               after               │
                                                                         │    sec/op    │   sec/op     vs base              │
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_1,_shard_size_=_3-8      16.46µ ±  4%   17.45µ ± 1%  +6.02% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_1,_shard_size_=_10-8     48.02µ ±  5%   48.86µ ± 1%       ~ (p=0.310 n=6)
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_1,_shard_size_=_30-8     151.4µ ± 10%   145.7µ ± 0%  -3.73% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_3,_shard_size_=_3-8      41.57µ ±  8%   42.08µ ± 1%       ~ (p=0.093 n=6)
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_3,_shard_size_=_10-8     81.09µ ±  4%   83.44µ ± 1%       ~ (p=0.240 n=6)
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_3,_shard_size_=_30-8     193.7µ ±  6%   188.8µ ± 2%  -2.52% (p=0.041 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_1,_shard_size_=_3-8     16.09µ ±  1%   17.04µ ± 1%  +5.90% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_1,_shard_size_=_10-8    47.80µ ±  4%   48.01µ ± 1%       ~ (p=0.485 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_1,_shard_size_=_30-8    149.1µ ±  4%   144.5µ ± 1%  -3.03% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_3,_shard_size_=_3-8     41.63µ ±  4%   41.22µ ± 0%       ~ (p=0.065 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_3,_shard_size_=_10-8    77.90µ ±  1%   82.28µ ± 5%  +5.62% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_3,_shard_size_=_30-8    186.4µ ±  6%   185.8µ ± 1%       ~ (p=0.240 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_1,_shard_size_=_3-8    16.26µ ±  7%   16.44µ ± 0%       ~ (p=0.084 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_1,_shard_size_=_10-8   48.59µ ±  4%   46.82µ ± 1%  -3.64% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_1,_shard_size_=_30-8   151.7µ ±  1%   142.7µ ± 1%  -5.87% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_3,_shard_size_=_3-8    41.25µ ±  7%   39.95µ ± 0%  -3.15% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_3,_shard_size_=_10-8   79.29µ ±  2%   79.82µ ± 0%  +0.68% (p=0.009 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_3,_shard_size_=_30-8   189.2µ ±  1%   183.5µ ± 1%  -3.01% (p=0.002 n=6)
geomean                                                                    64.76µ         64.70µ       -0.09%

                                                                         │    before    │               after                │
                                                                         │     B/op     │     B/op      vs base              │
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_1,_shard_size_=_3-8      9.391Ki ± 0%   9.367Ki ± 0%  -0.25% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_1,_shard_size_=_10-8     15.51Ki ± 0%   15.34Ki ± 0%  -1.10% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_1,_shard_size_=_30-8     34.21Ki ± 0%   33.79Ki ± 0%  -1.24% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_3,_shard_size_=_3-8      21.55Ki ± 0%   21.53Ki ± 0%  -0.11% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_3,_shard_size_=_10-8     33.03Ki ± 0%   32.94Ki ± 0%  -0.28% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_3,_shard_size_=_30-8     61.42Ki ± 0%   60.91Ki ± 0%  -0.84% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_1,_shard_size_=_3-8     9.391Ki ± 0%   9.367Ki ± 0%  -0.25% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_1,_shard_size_=_10-8    15.51Ki ± 0%   15.34Ki ± 0%  -1.11% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_1,_shard_size_=_30-8    34.21Ki ± 0%   33.79Ki ± 0%  -1.23% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_3,_shard_size_=_3-8     21.55Ki ± 0%   21.53Ki ± 0%  -0.11% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_3,_shard_size_=_10-8    33.03Ki ± 0%   32.94Ki ± 0%  -0.28% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_3,_shard_size_=_30-8    61.42Ki ± 0%   60.91Ki ± 0%  -0.84% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_1,_shard_size_=_3-8    9.391Ki ± 0%   9.367Ki ± 0%  -0.25% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_1,_shard_size_=_10-8   15.51Ki ± 0%   15.34Ki ± 0%  -1.10% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_1,_shard_size_=_30-8   34.21Ki ± 0%   33.79Ki ± 0%  -1.23% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_3,_shard_size_=_3-8    21.55Ki ± 0%   21.53Ki ± 0%  -0.11% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_3,_shard_size_=_10-8   33.03Ki ± 0%   32.94Ki ± 0%  -0.28% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_3,_shard_size_=_30-8   61.42Ki ± 0%   60.90Ki ± 0%  -0.84% (p=0.002 n=6)
geomean                                                                    24.53Ki        24.37Ki       -0.64%

                                                                         │   before   │               after                │
                                                                         │ allocs/op  │  allocs/op   vs base               │
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_1,_shard_size_=_3-8      22.00 ± 0%    25.00 ± 0%  +13.64% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_1,_shard_size_=_10-8     31.00 ± 0%    41.00 ± 0%  +32.26% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_1,_shard_size_=_30-8     52.00 ± 0%    82.00 ± 0%  +57.69% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_3,_shard_size_=_3-8      37.00 ± 0%    40.00 ± 0%   +8.11% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_3,_shard_size_=_10-8     52.00 ± 0%    64.00 ± 0%  +23.08% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_50,_num_zones_=_3,_shard_size_=_30-8     76.00 ± 0%   106.00 ± 0%  +39.47% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_1,_shard_size_=_3-8     22.00 ± 0%    25.00 ± 0%  +13.64% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_1,_shard_size_=_10-8    31.00 ± 0%    41.00 ± 0%  +32.26% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_1,_shard_size_=_30-8    52.00 ± 0%    82.00 ± 0%  +57.69% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_3,_shard_size_=_3-8     37.00 ± 0%    40.00 ± 0%   +8.11% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_3,_shard_size_=_10-8    52.00 ± 0%    64.00 ± 0%  +23.08% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_100,_num_zones_=_3,_shard_size_=_30-8    76.00 ± 0%   106.00 ± 0%  +39.47% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_1,_shard_size_=_3-8    22.00 ± 0%    25.00 ± 0%  +13.64% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_1,_shard_size_=_10-8   31.00 ± 0%    41.00 ± 0%  +32.26% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_1,_shard_size_=_30-8   52.00 ± 0%    82.00 ± 0%  +57.69% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_3,_shard_size_=_3-8    37.00 ± 0%    40.00 ± 0%   +8.11% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_3,_shard_size_=_10-8   52.00 ± 0%    64.00 ± 0%  +23.08% (p=0.002 n=6)
Ring_ShuffleShard/num_instances_=_1000,_num_zones_=_3,_shard_size_=_30-8   76.00 ± 0%   106.00 ± 0%  +39.47% (p=0.002 n=6)
geomean                                                                    41.60         53.25       +28.00%

Having the implementation limited to slices limits the extent to which
this package can be re-used.

The PR introducing loser tree stated that this was to achieve a 15%
performance improvement, but benchmarks on this change give a mixed
picture, with some going 5% faster and some 7% slower.
Copy link
Contributor

@charleskorn charleskorn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please include BenchmarkRing_ShuffleShard_LargeShardSize in your benchmark comparison?

If that shows a similarly low absolute difference in performance, then this change makes sense to me.

@@ -565,6 +565,7 @@ func TestDoUntilQuorumWithoutSuccessfulContextCancellation_PartialZoneFailure(t
}

func TestDoUntilQuorumWithoutSuccessfulContextCancellation_CancelsEntireZoneImmediatelyOnSingleFailure(t *testing.T) {
t.Skip()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you intend to include this?

}

return out
}

// Wrapper over a slice that implements the loser.Sequence API
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth including a version of this in the loser package? I imagine this won't be the only place that needs to work with slices.

Having said that, I'm not sure whether we'd want a generic slice sequence type, or specialised types - would be interesting to benchmark and see if the use of generics makes any substantial performance difference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants