Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix NumObjects and Initialized with WriteTxn #42

Merged
merged 2 commits into from
Aug 8, 2024

Conversation

joamaki
Copy link
Contributor

@joamaki joamaki commented Aug 8, 2024

Both NumObjects() and Initialized() returned the committed state and not the currently modified state:

txn := db.WriteTxn(tbl)
tbl.NumObjects(txn) == 0
tbl.Insert(txn, Foo{})
tbl.NumObjects(txn) == 0 !!!

init := tbl.RegisterInitializer("test")
tbl.Initialized(txn) == false

init(txn)
tbl.Initialized(txn) == false !!!

With a WriteTxn both NumObjects and Initialized did not return the new mutated
state, but rather the previously committed state.

Signed-off-by: Jussi Maki <[email protected]>
These methods did not look into the "modifiedTables" nor into the
currently open write transactions into the indexes but rather returned
the previously committed state. 

Signed-off-by: Jussi Maki <[email protected]>
Copy link

github-actions bot commented Aug 8, 2024

$ make test
go: downloading github.com/mitchellh/mapstructure v1.5.0
go: downloading golang.org/x/term v0.16.0
go: downloading github.com/spf13/cast v1.6.0
go: downloading github.com/fsnotify/fsnotify v1.7.0
go: downloading github.com/sagikazarmark/slog-shim v0.1.0
go: downloading github.com/spf13/afero v1.11.0
go: downloading golang.org/x/sys v0.17.0
go: downloading github.com/subosito/gotenv v1.6.0
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading gopkg.in/ini.v1 v1.67.0
go: downloading github.com/magiconair/properties v1.8.7
go: downloading github.com/pelletier/go-toml/v2 v2.1.0
go: downloading golang.org/x/text v0.14.0
	github.com/cilium/statedb/reconciler/benchmark		coverage: 0.0% of statements
	github.com/cilium/statedb/reconciler/example		coverage: 0.0% of statements
ok  	github.com/cilium/statedb	5.264s	coverage: 88.8% of statements
ok  	github.com/cilium/statedb/index	0.006s	coverage: 25.5% of statements
ok  	github.com/cilium/statedb/internal	0.008s	coverage: 93.3% of statements
ok  	github.com/cilium/statedb/part	1.788s	coverage: 82.5% of statements
ok  	github.com/cilium/statedb/reconciler	2.297s	coverage: 92.0% of statements
-----
$ make bench
go test ./... -bench . -benchmem -test.run xxx
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkDB_WriteTxn_1-4                    	  448350	      2568 ns/op	    389355 objects/sec	    2720 B/op	      32 allocs/op
BenchmarkDB_WriteTxn_10-4                   	 1248512	       953.7 ns/op	   1048525 objects/sec	     715 B/op	      10 allocs/op
BenchmarkDB_WriteTxn_100-4                  	 1601576	       778.9 ns/op	   1283832 objects/sec	     582 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_1000-4                 	 1524307	       785.4 ns/op	   1273158 objects/sec	     533 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_10000-4                	 1263010	       985.0 ns/op	   1015264 objects/sec	     554 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_100_SecondaryIndex-4   	  535526	      2263 ns/op	    441874 objects/sec	    1671 B/op	      30 allocs/op
BenchmarkDB_RandomInsert-4                  	    2416	    479935 ns/op	   2083615 objects/sec	  386103 B/op	    7100 allocs/op
BenchmarkDB_RandomReplace-4                 	     427	   2802489 ns/op	    356826 objects/sec	 1847344 B/op	   30564 allocs/op
BenchmarkDB_SequentialInsert-4              	    1562	    783347 ns/op	   1276574 objects/sec	  533720 B/op	    7288 allocs/op
BenchmarkDB_Changes_Baseline-4              	    1256	    947163 ns/op	   1055786 objects/sec	  536188 B/op	   10246 allocs/op
BenchmarkDB_Changes-4                       	     702	   1706062 ns/op	    586145 objects/sec	  964342 B/op	   14541 allocs/op
BenchmarkDB_RandomLookup-4                  	   20188	     57240 ns/op	  17470545 objects/sec	     144 B/op	       1 allocs/op
BenchmarkDB_SequentialLookup-4              	   24882	     47292 ns/op	  21145392 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_FullIteration_All-4             	     813	   1450652 ns/op	  68934630 objects/sec	     360 B/op	       7 allocs/op
BenchmarkDB_FullIteration_Get-4             	     207	   7648686 ns/op	  13074162 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_PropagationDelay-4              	  494841	      2251 ns/op	        20.00 50th_µs	        22.00 90th_µs	        94.00 99th_µs	    1514 B/op	      24 allocs/op
PASS
ok  	github.com/cilium/statedb	26.481s
PASS
ok  	github.com/cilium/statedb/index	0.004s
PASS
ok  	github.com/cilium/statedb/internal	0.003s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/part
cpu: AMD EPYC 7763 64-Core Processor                
Benchmark_Insert_RootOnlyWatch-4    	    9786	    121790 ns/op	   8210886 objects/sec	  104118 B/op	    2041 allocs/op
Benchmark_Insert-4                  	    6427	    170851 ns/op	   5853038 objects/sec	  202681 B/op	    3064 allocs/op
Benchmark_Replace-4                 	29615572	        40.59 ns/op	  24634151 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Replace_RootOnlyWatch-4   	29527713	        40.59 ns/op	  24638268 objects/sec	       0 B/op	       0 allocs/op
Benchmark_txn_1-4                   	 2972832	       396.3 ns/op	   2523385 objects/sec	     432 B/op	       7 allocs/op
Benchmark_txn_10-4                  	 7793312	       151.9 ns/op	   6585318 objects/sec	     152 B/op	       2 allocs/op
Benchmark_txn_100-4                 	 7508161	       173.9 ns/op	   5750672 objects/sec	     223 B/op	       2 allocs/op
Benchmark_txn_1000-4                	 7668471	       152.7 ns/op	   6547189 objects/sec	     215 B/op	       2 allocs/op
Benchmark_txn_10000-4               	 5412721	       226.6 ns/op	   4413570 objects/sec	     237 B/op	       2 allocs/op
Benchmark_txn_100000-4              	 4325589	       286.8 ns/op	   3487321 objects/sec	     343 B/op	       2 allocs/op
Benchmark_txn_delete_1-4            	 3235819	       367.5 ns/op	   2721019 objects/sec	     840 B/op	       6 allocs/op
Benchmark_txn_delete_10-4           	 8671322	       135.5 ns/op	   7381561 objects/sec	     131 B/op	       1 allocs/op
Benchmark_txn_delete_100-4          	10564479	       110.5 ns/op	   9050374 objects/sec	      60 B/op	       1 allocs/op
Benchmark_txn_delete_1000-4         	12047515	        99.21 ns/op	  10079253 objects/sec	      26 B/op	       1 allocs/op
Benchmark_txn_delete_10000-4        	10159572	       116.8 ns/op	   8558873 objects/sec	      25 B/op	       1 allocs/op
Benchmark_txn_delete_100000-4       	 9182602	       130.4 ns/op	   7669181 objects/sec	      26 B/op	       1 allocs/op
Benchmark_Get-4                     	   38839	     30478 ns/op	  32810339 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Iterate-4                 	  166304	      7338 ns/op	 136277490 objects/sec	      80 B/op	       3 allocs/op
Benchmark_Hashmap_Insert-4          	   15562	     77354 ns/op	  12927599 objects/sec	   86556 B/op	      64 allocs/op
Benchmark_Hashmap_Get_Uint64-4      	  156871	      7809 ns/op	 128055308 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Hashmap_Get_Bytes-4       	  153370	      7772 ns/op	 128669252 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Uint64Map_Random-4        	    1260	    930516 ns/op	   1074673 items/sec	 2689528 B/op	    9025 allocs/op
Benchmark_Uint64Map_Sequential-4    	    1436	    872737 ns/op	   1145822 items/sec	 2476394 B/op	    9749 allocs/op
PASS
ok  	github.com/cilium/statedb/part	32.697s
PASS
ok  	github.com/cilium/statedb/reconciler	0.004s
?   	github.com/cilium/statedb/reconciler/benchmark	[no test files]
?   	github.com/cilium/statedb/reconciler/example	[no test files]
go run ./reconciler/benchmark -quiet
1000000 objects reconciled in 2.83 seconds (batch size 1000)
Throughput 353266.44 objects per second
Allocated 6011435 objects, 409095kB bytes, 535024kB bytes still in use

@joamaki joamaki merged commit 480663e into main Aug 8, 2024
1 check passed
@joamaki joamaki deleted the pr/joamaki/fix-initialized-numobjects-writetxn branch August 8, 2024 13:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants