perf: concurrent txpool fetcher with separate IPC connection#906
Open
vbuilder69420 wants to merge 2 commits intoflashbots:developfrom
Open
perf: concurrent txpool fetcher with separate IPC connection#906vbuilder69420 wants to merge 2 commits intoflashbots:developfrom
vbuilder69420 wants to merge 2 commits intoflashbots:developfrom
Conversation
…klist check Remove the AccessListInspector entirely from RBuilderEVMInspector. Replace the per-opcode blocklist tracking with a post-execution check against ResultAndState.state (EvmState = HashMap<Address, Account>), which already contains every address touched during EVM execution. The AccessListInspector called step() on every EVM opcode to build an access list, solely used to check addresses against the blocklist. Profiling showed this inspector overhead consumed ~52% of CPU time. The EVM execution result already contains the same information in its state diff, making the inspector entirely redundant. Changes: - order_commit.rs: Use create_evm() (NoOpInspector) when no used_state_tracer is needed. Check blocklist via res.state.keys() after execution instead of via access list. - evm_inspector.rs: Remove AccessListInspector from RBuilderEVMInspector. The inspector now only wraps the optional UsedStateEVMInspector (used by parallel builder / EVM caching). This optimization works regardless of whether a blocklist is configured. Benchmark (builder-lab, 100 TPS, seed=42, 60s profiling window): | Metric | Before | After | Change | |---------------------|----------|----------|--------| | Block fill p50 | 96.8ms | 58.9ms | -39% | | Block fill p95 | 129.2ms | 87.1ms | -33% | | E2E latency p50 | 98ms | 61ms | -38% | | E2E latency p95 | 134ms | 92ms | -31% | | Blocks submitted | 255 | 342 | +34% | | Txs included | 17,882 | 23,449 | +31% | Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The txpool fetcher previously processed tx hash notifications sequentially:
for each hash, it made a blocking get_raw_transaction_by_hash RPC call over
the same IPC connection before consuming the next notification. At high TPS,
this caused the subscription to back up and the IPC connection to die.
Three changes:
1. Use a SEPARATE IPC connection for RPC calls, so fetches don't block
the subscription stream consumer
2. Fetch transactions CONCURRENTLY using tokio::spawn with a semaphore
(64 concurrent fetches) instead of sequential per-hash blocking
3. Increase the subscription broadcast channel from 16 to 16384 items
to prevent silent notification drops under load
Before: IPC subscription dies after ~5 blocks at 500 TPS
After: IPC subscription survives the full run (46+ blocks at 500 TPS,
5+ minutes at 2000 TPS)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix the txpool subscription dying under high transaction load by:
get_raw_transaction_by_hashRPC callsRoot Cause
The previous implementation had three cascading bottlenecks:
Sequential per-tx RPC calls block the subscription consumer — For each tx hash notification, rbuilder made a blocking
get_raw_transaction_by_hashcall over the same IPC connection before reading the next notification. At 500+ TPS, the subscription produced hashes faster than rbuilder could fetch full txs one-by-one.Broadcast channel of 16 items — The alloy
PubSubFrontenddefaults the broadcast channel to 16 items. At 500+ TPS, this overflows instantly and notifications are silently dropped (SubscriptionStreamignoresLaggederrors).No backpressure — The unbounded channels between the IPC backend and PubSub service meant the kernel socket buffer could fill up, causing reth to reset the connection.
Before / After
Design
Test plan
test_fetcher_retrieves_transactionspreserved🤖 Generated with Claude Code