Skip to content

Commit

Permalink
increase block cache size for stakedb
Browse files Browse the repository at this point in the history
  • Loading branch information
@ukane-philemon
ukane-philemon committed Dec 24, 2022
1 parent 7e3a521 commit 1b474c0
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions stakedb/ticketpool.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -172,6 +172,7 @@ func dbVersion(db *badger.DB) (uint32, error) {
func badgerOptions(badgerDbPath string) badger.Options {
opts := badger.DefaultOptions(badgerDbPath)
opts.DetectConflicts = false
opts.BlockCacheSize = 1 << 29

This comment has been minimized.

Sign in to view

Copy link
@chappjc

chappjc Dec 24, 2022

Member

So this will use 512 MiB of RAM just for the ticket pool? Doesn't sound great. What did v1 use, if hidden?
Also, the way to test performance is not via the http api requests, but by traversing the ticket pool in Go. I think there are functions and methods for that. A Go benchmark with different cache sizes would be the way to make an informed decision on this parameter.

This comment has been minimized.

Sign in to view

Copy link
@ukane-philemon

ukane-philemon Dec 26, 2022 •
edited
Loading

Author Collaborator

A Go benchmark with different cache sizes would be the way to make an informed decision on this parameter.

Yeah, just wrote some benchmark tests and am getting good results. Will share when I am done.

Turns out Badger v3 is faster than Badger v1, with a performance improvement of around 5x. But to make badger stop printing out the block size insufficient log maybe 300MiB is okay?

Benchmark test results ``` Inserting 100000 diffs with 8 In and Out hashes each.
 goos: darwin goarch: amd64 pkg: github.com/decred/dcrdata/v8/stakedb cpu: Intel(R) Core(TM) i7-3720QM CPU @ 2.60GHz
 BenchmarkBadgerV1-8 1 20340623706 ns/op BenchmarkBadgerV3-8 1 3781469131 ns/op // default block cache size

BenchmarkBadgerV1-8 1 19579837526 ns/op
BenchmarkBadgerV3-8 1 3620374836 ns/op // 300 mib block cache size


BenchmarkBadgerV1-8 1 19482899867 ns/op
BenchmarkBadgerV3-8 1 3865946773 ns/op // 350 mib block cache size



BenchmarkBadgerV1-8 1 19568935566 ns/op
BenchmarkBadgerV3-8 1 3919804691 ns/op

--- BENCH: BenchmarkBadgerV1-8
ticketpool_test.go:523: Loading 100000 diffs took 1 ms
--- BENCH: BenchmarkBadgerV3-8
ticketpool_test.go:547: Loading 100000 diffs took 100 ms


</details>

This comment has been minimized.

Sign in to view

Copy link
@ukane-philemon

ukane-philemon Dec 26, 2022

Author Collaborator

What did v1 use, if hidden?

AFAICT, this wasn't a thing then. But I think 300MiB will do just fine, the default is 256.
opts.CompactL0OnClose = true
opts.MetricsEnabled = false
opts.Logger = &badgerLogger{log}
Expand Down

0 comments on commit 1b474c0

Please sign in to comment.