Repository navigation
perf(debug-trace-server): build the witness execution form once per block - #185
Merged
Merged
Conversation
… cz/chore/upgrade-revm-40)
reth v2.3.0's V2 sparse tries merge extension nodes into their child branches and silently drop extensions whose child branch is absent from the witness — a legal minimal exclusion proof that the frozen MegaETH witness format produces, and whose split-on-insert never needs the child. Vendors the proven v1.6.0 serial sparse trie into stateless-core, reveals the storage trie directly (dropping the synthesized account-leaf wrapper reveal_witness required), and adds a regression test for the extension-split shape.
…update docs - Drop the returnValue 0x-strip shim: alloy-rpc-types-trace 2.x serializes the prefix on both mega-reth and this server, so emitting the frame as-is is what now matches. - rust-version 1.94 (mega-evm and the optimism monorepo crates require it). - Document the vendored withdrawal sparse trie and erc7562Tracer handling.
The reth section now holds only paradigmxyz/reth tag v2.3.0 crates; alloy-op-evm, alloy-op-hardforks, and reth-optimism-chainspec move next to the op-alloy crates they share a source identity (and pin comment) with.
Replaces the vendored reth-1.6 serial sparse trie (~1.9k lines) with a ~250-line linearization of the witnessed storage trie into alloy_trie::HashBuilder — the primitive reth's own eth_getProof generator is built on. Surviving and updated leaves stream as add_leaf (leaf paths are the hashed slots and never change); unwitnessed subtrees stream as add_branch (position and hash invariant under edits elsewhere); every structural change — extension splits, branch collapses, new branches — falls out of the stream shape inside the builder, so the module carries no trie mutation code at all. A pre-pass with no updates must reproduce storage_root exactly, self-checking the walk on every witness before its post-state root is trusted. Updates the witness cannot prove (descent into an unwitnessed subtree, a collapse adopting an unwitnessed node) fail closed, matching reth-1.6 behavior. Drops the reth-trie-common, smallvec, and auto_impl dependencies; adds tests for collapse merging, unwitnessed-survivor collapse, unwitnessed-region descent, and diverging-removal no-ops.
- withdrawals: skip the post-pass when a block carries no withdrawal updates; factor the branch-child partition into child_regions/ChildRegion shared by emission and survival walks; drop the dead empty-region arm; slice-pattern the sole-survivor check; add_insert_leaves helper for the four insert-only loops; flatten withdrawals/ back to withdrawals.rs; dedupe test fixtures (TrieNode::rlp, SLOT_B, ext_with_absent_branch). - debug-trace-server: reject the unsupported erc7562Tracer at the RequestShape::classify gate (-32602 before any block data is fetched), the executor arms staying as unreachable backstops; drop per-tx EvmState clones the revm-40 API made avoidable (block/tx prestate, mux); take tracing inspectors by mem::replace instead of cloning their recorded trace arenas. - executor: project slot_num from header.slot_number like every other BlockEnv field instead of pinning 0.
Beyond the textual Cargo.toml conflict (alloy family stays 2.0.5; #172's new alloy-rpc-client added at 2.0.5): - workspace reqwest 0.12 -> 0.13: #172 hands our tuned client to alloy's transport, which rides reqwest 0.13 in the 2.x era; stateless-r2 keeps its own 0.12 pin (API-coupled to mega-reth, no type crossing). - #169's builtin_tracer_kind gains the alloy-2.x Erc7562Tracer arm in its not-routed-here group. - mux rejection wording: since alloy 2.x an unknown tracer key in a mux config parses as a JS tracer name and is rejected by MuxInspector construction rather than serde; the error context stays 'invalid muxTracer tracerConfig' so #169's defense-in-depth test keeps its contract.
mega-reth's upstream/reth workspace — the branch this upgrade wave targets — is on reqwest 0.13, so the API-coupled pin follows (rustls-tls feature is named rustls since 0.13; http2 note stays valid, the feature is opt-in now). The lockfile drops to a single reqwest 0.13.4.
The no_std build (--no-default-features) has no prelude format!; import it explicitly like evm_database.rs does. Fixes the no-std and coverage CI jobs, which both compile stateless-core without default features.
- tracing_executor: extract the four TracerKind frame builders into one extract_trace_frame! macro shared by the block and tx paths (the twin matches this PR had to edit in lockstep six times); the FlatCall take now really is the per-tx reset (guarded tail); mux inspectors reset from a pristine clone instead of re-cloning + re-parsing the config each tx (also removes the only dispatcher unwrap); erc7562 rejection has one home in builtin_tracer_kind and the dispatchers route it there; preceding-tx replay in the tx-level and parity paths seeds a none-config inspector instead of recording every step just to discard it; drop a dead duplicate binding. - rpc_service: classify_and_gate dedups its rejection tail behind an exhaustive match (a future RequestShape variant must make an explicit gate decision); new test pins the -32602 for unsupported tracers. - response_cache: the label-conformance test now covers the Unsupported variant. - withdrawals: the leaf arm reuses sortedness instead of collect+sort; the split_at_prefix upper bound is a second partition_point.
…lock Every trace request deep-cloned the block's LightWitness (multi-hundred-KB kvs/levels maps) out of the shared Arc<BlockData> and then rebuilt the LightWitnessExecutor direct-lookup table from scratch — paid per debug_traceTransaction (never response-cached) and per block-level cache miss/bypass. BlockData now stores the LightWitnessExecutor, built once where the BlockData is assembled (DB load, remote fetch, fixtures), and the trace entry points borrow it; the block-data cache weigher charges the lookup table alongside the witness payload.
Contributor
Claude review status
✅ Review clean Last reviewed: head New this round: 0 finding(s), 0 question(s) · Resolved this round: 0 · Open questions: 0 |
Codecov Report❌ Patch coverage is
☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Troublor
approved these changes
Aug 11, 2026
vincent-k2026
approved these changes
Aug 12, 2026
PR #184 was squash-merged, so this branch (forked from an intermediate copy of #184) conflicted with main across unrelated files. Resolve by taking main's tree, which carries the final #184, and re-applying this PR's change on top of it instead of reconciling the intermediate #184 history. - light_witness.rs: main dropped the `impl LightWitnessExecutor` block (`kvs()` was unused); re-add it with only `light_witness()` and `lookup_table_memory_bytes()`. - tracing_executor.rs: keep main's `BlockExecutionEnv` plumbing in `TracingEnv` and borrow the shared `&LightWitnessExecutor`. - Adapt tests added on main (single-tx vs block trace parity, DB-tier drift) to the executor-typed `BlockData::witness`. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…tidy witness naming Simplify pass over the shared-executor change: - Add `light_witness_executor_memory_bytes` beside `light_witness_memory_bytes` so the block-data weigher and its test share one definition of an executor's footprint, with a test pinning the lookup-table term. - Name the witness `witness` across the trace entry points and the `TracingEnv` field instead of the stale `light_witness`. - Split `test_support::fixture_parts` out of `fixture_block_data` so tests that need a plain `LightWitness` stop round-tripping through the executor. - Shorten the `BlockData.witness` doc to what it is. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…fetch pipeline and cache weight - do_fetch_block_data: list the execution-form build as the final pipeline step, so a reader profiling slow cold fetches sees where the lookup table is built. - README: say the block-data cache charges a witness as its payload plus the lookup table, so an entry weighs more than its witness maps alone. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Troublor
force-pushed
the
liquan/perf/share-light-witness-executor
branch
from
October 19, 2026 13:13
00a7b9c to
4f0c128
Compare
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
…ss footprint math - Add `BlockData::new`, the one place the witness is wrapped in its execution form. - Keep the lookup-table footprint in `witness_size` (`LOOKUP_ENTRY_BYTES`); core exposes only `plain_keys()`. - Let `TracingEnv` own its `WitnessDatabase`, dropping `create_witness_db` and the redundant fields. - Make the executor-footprint test assert the table is charged instead of restating the formula. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every trace request deep-cloned the block's
LightWitnessout of the sharedArc<BlockData>(data.witness.clone()— the multi-hundred-KB-to-MBkvs/levelsmaps) and then rebuilt theLightWitnessExecutordirect-lookup table from scratch, two O(witness) passes paid on everydebug_traceTransaction(transaction responses are never response-cached) and on every block-level response-cache miss or bypass (JS, mux, struct-logger-with-flags). Surfaced by the review pass on #184 as the largest remaining hot-path allocation; pre-existing, so fixed here separately.Change
BlockData.witnessnow stores theLightWitnessExecutor— witness plus lookup table — built once where theBlockDatais assembled (DB load, remote fetch, test fixtures) and shared by every request served from that entry, including repeated traces of a hot block. The trace entry points (trace_block,trace_transaction,parity_trace_block,parity_trace_transaction) andTracingEnvborrow it (&LightWitnessExecutor); no per-request clone or table rebuild remains.LightWitnessExecutorgainslight_witness()(forextract_code_hashesand witness-size accounting) andlookup_table_memory_bytes(), which the block-data cache weigher now charges alongside the witness payload so the cache budget reflects the stored table.Testing
cargo test --workspace: 374 passed / 0 failed; clippy / fmt / cargo-sort clean.Notes
Based on #184 (
liquan/chore/upgrade-reth-2.3) — merge after it lands.