Skip to content

Commit e470758

Browse files
authored
statement-store performance benchmarks (#9763)
# Description Adds benchmarks to measure the performance of the statement-store: - Message Exchange Scenario: interaction with one or many nodes. - Memory Stress Test Scenario. ## Results **Key improvements made to improve the performance:** - [Fixed a deadlock](#9868) - [Increased statements limits](#9894) - [Improved gossiping](#9912) **Hardware** All benchmarks were run on a MacBook Pro M2 ### 1. Message Exchange Scenario **Test Configuration** - Total participants: 49_998 - Group size: 6 participants per group - Total groups: 8_333 - Statement payload size: 512 KB - Propagation delay: 2 seconds (empirically determined) - Parachain network tested with: 2, 3, 6, 12 nodes **Network Topologies** We tested two distribution patterns: 1. **To one RPC:** All participants send statements to a single RPC node 2. **To all RPC:** Participants distribute statements across all nodes (slower due to gossiping overhead) **Participant Flow** - Sends a statement with their key for an exchange session (1 sent) - Waits 2 seconds for statement propagation - Receives session keys from other members in the group (5 received) - Sends statements containing a 512KB message to each member in the group (5 sent) - Waits 2 seconds for statement propagation - Receives messages from other members (5 received) - Total: 6 sent, 10 received. **Results** | Collators | Avg time | Max time | Memory | | -------------- | -------- | -------- | ------ | | **To one RPC** | | | | | 2 | 35s | 35s | 2.1GB | | 6 | 48s | 50s | 1.7GB | | **To all RPC** | | | | | 3 | 41s | 51s | 1.9GB | | 6 | 61s | 71s | 1.4GB | | 12 | 94s | 119s | 1.9GB | **Observations** - Sending to one RPC node is faster but creates a bottleneck - Distributing across all nodes takes longer due to gossiping overhead - More collators increase gossiping load, resulting in slower completion times - Memory usage per node remains around 2GB. ### 2. Memory Stress Test Scenario **Test Configuration** We prepared one more scenario to check how much memory nodes use with a full store after we increased the limits. To maximize memory usage for index usage, we submitted statements with all unique topics; other fields (e.g., proofs) were not used. - Total tasks: 100,000 concurrent - Statement size: 1 KB per statement - Network size: 6 collators **Test Flow** 1. Spawn 100,000 concurrent tasks 2. Each task sends statements with 1KB payload to one node until store is full 3. Statements are gossiped across the network to the other 5 collators 4. Test completes when all collator stores are full **Results** During the tests, each node used up to 4.5GB of memory.
1 parent 2ae8298 commit e470758

File tree

5 files changed

+917
-0
lines changed

5 files changed

+917
-0
lines changed

Cargo.lock

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

cumulus/zombienet/zombienet-sdk/Cargo.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,10 @@ zombienet-orchestrator = { workspace = true }
2121
zombienet-configuration = { workspace = true }
2222
cumulus-zombienet-sdk-helpers = { workspace = true }
2323
sp-statement-store = { workspace = true, default-features = true }
24+
sc-statement-store = { workspace = true, default-features = true }
2425
sp-keyring = { workspace = true, default-features = true }
2526
sp-core = { workspace = true, default-features = true }
27+
codec = { workspace = true }
2628
rstest = { workspace = true }
2729

2830
[features]

cumulus/zombienet/zombienet-sdk/tests/zombie_ci/mod.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,4 +11,5 @@ mod pov_recovery;
1111
mod rpc_collator_build_blocks;
1212
mod runtime_upgrade;
1313
mod statement_store;
14+
mod statement_store_bench;
1415
mod sync_blocks;

cumulus/zombienet/zombienet-sdk/tests/zombie_ci/people-rococo-spec.json

Lines changed: 148 additions & 0 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)