Various optimizations

[joamaki]

Various optimizations:

• Switch storing of tables and indexes from iradix to simple arrays.
• Drop callerPackage as it was very costly and use a named handle instead (we can easily inject the module name as part of cell.Module using a module provider).
• Rework the benchmarks to use constant data set size to avoid using too much memory.

Before:

 goos: linux
 goarch: amd64
 pkg: github.com/cilium/statedb
 cpu: Intel(R) Core(TM) i5-10210U CPU @ 1.60GHz
 BenchmarkDB_WriteTxn_1-8                                          132523              9268 ns/op            107899 objects/sec
 BenchmarkDB_WriteTxn_10-8                                         367114              3044 ns/op            328531 objects/sec
 BenchmarkDB_WriteTxn_100-8                                        430207              2459 ns/op            406751 objects/sec
 BenchmarkDB_WriteTxn_100_SecondaryIndex-8                         503071              2358 ns/op            424111 objects/sec
 BenchmarkDB_RandomInsert-8                                           945           1288636 ns/op            776014 objects/sec
 BenchmarkDB_RandomReplace-8                                          226           5256999 ns/op            190223 objects/sec
 BenchmarkDB_SequentialInsert-8                                       370           3580799 ns/op            279267 objects/sec
 ...
 BenchmarkDB_DeleteTracker_Baseline-8                                 356           3314962 ns/op            301663 objects/sec
 BenchmarkDB_DeleteTracker-8                                          181           6609844 ns/op            151290 objects/sec
 BenchmarkDB_RandomLookup-8                                          3289            354713 ns/op           2819181 objects/sec
 BenchmarkDB_SequentialLookup-8                                      3519            334955 ns/op           2985479 objects/sec
 BenchmarkDB_FullIteration_All-8                                    88718             12817 ns/op          78022832 objects/sec
 BenchmarkDB_FullIteration_Get-8                                    71965             15974 ns/op          62599999 objects/sec
 BenchmarkDB_PropagationDelay-8                                    188659              6398 ns/op
 55.00 50th_µs           74.00 90th_µs          277.0 99th_µs
 PASS
 ok      github.com/cilium/statedb       33.132s 

After:

 goos: linux
 goarch: amd64
 pkg: github.com/cilium/statedb
 cpu: Intel(R) Core(TM) i5-10210U CPU @ 1.60GHz
 BenchmarkDB_WriteTxn_1-8                                          310290              3885 ns/op            257388 objects/sec
 BenchmarkDB_WriteTxn_10-8                                         523450              2441 ns/op            409679 objects/sec
 BenchmarkDB_WriteTxn_100-8                                        538578              2219 ns/op            450628 objects/sec
 BenchmarkDB_WriteTxn_100_SecondaryIndex-8                         515170              2156 ns/op            463816 objects/sec
 BenchmarkDB_RandomInsert-8                                          1110           1081693 ns/op            924477 objects/sec
 BenchmarkDB_RandomReplace-8                                          237           5034048 ns/op            198647 objects/sec
 BenchmarkDB_SequentialInsert-8                                       380           3048134 ns/op            328070 objects/sec
 ...
 BenchmarkDB_DeleteTracker_Baseline-8                                 396           3066078 ns/op            326150 objects/sec
 BenchmarkDB_DeleteTracker-8                                          169           7019558 ns/op            142459 objects/sec
 BenchmarkDB_RandomLookup-8                                          8839            137467 ns/op           7274474 objects/sec
 BenchmarkDB_SequentialLookup-8                                      8958            124483 ns/op           8033258 objects/sec
 BenchmarkDB_FullIteration_All-8                                    97218             11356 ns/op          88057271 objects/sec
 BenchmarkDB_FullIteration_Get-8                                    78102             14373 ns/op          69577187 objects/sec
 BenchmarkDB_PropagationDelay-8                                    245020              4727 ns/op
 42.00 50th_µs           48.00 90th_µs          210.0 99th_µs
 PASS
 ok      github.com/cilium/statedb       31.520s

Result highlights:

• One insert per one WriteTxn went from 100k/s to 260k/s
• Random insert of a new object went from 776k/s to 924k/s

[bimmlerd]

lgtm

[bimmlerd]

Why is it okay to modify this root?

EDIT: aha. was reviewing commit by commit - I guess the next line should be in an earlier commit

[joamaki]

I'm not following. How would you change this?

[bimmlerd]

in 308c157 we do

root := *db.root.Load()
for pos, table := range txn.modifiedTables {
	if table != nil {
		root[pos] = *table
	}
}

which seems to modify the root in place as part of Commit, which I was confused as to why it's safe (can't other read transactions have the same root slice pointer?).

But then I saw that the next commit adds the Clone: 1858d6b#diff-0e4a5bf98fe17be3b8b206da2133e0a481d6c82fb40e8c0155f40479e26ace07R446 so I think we're good, but it seems like this line should be part of the earlier commit.