trie_prefetcher: alternate structure #666
Draft
+283
−390
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
darioush
commented
Oct 1, 2024
Comment on lines
+87
to
+94
| defer func() { | ||
| type closer interface { | ||
| Close() | ||
| } | ||
| if closer, ok := p.db.(closer); ok { | ||
| closer.Close() | ||
| } | ||
| }() |
There was a problem hiding this comment.
added compared to upstream
darioush
commented
Oct 1, 2024
Comment on lines
+313
to
+314
| if trie, ok := sf.trie.(waiter); ok { | ||
| trie.Wait() |
There was a problem hiding this comment.
added compared to upstream
darioush
commented
Oct 1, 2024
| @@ -59,7 +59,8 @@ func filledStateDB() *StateDB { | |||
|
|
|||
| func TestCopyAndClose(t *testing.T) { | |||
| db := filledStateDB() | |||
| prefetcher := newTriePrefetcher(db.db, db.originalRoot, "", maxConcurrency) | |||
| prefetchDb := newPrefetcherDatabase(db.db, maxConcurrency) | |||
There was a problem hiding this comment.
to preserve existing UT behavior for now
darioush
force-pushed
the
trie-prefetcher-alt
branch
from
3d6f372
to
f3612cd
Compare
darioush
commented
Oct 1, 2024
| sf.dups++ | ||
| } else { | ||
| if len(task) == common.AddressLength { | ||
| sf.trie.GetAccount(common.BytesToAddress(task)) |
There was a problem hiding this comment.
we could do something more explicit here instead of wrapping GetAccount at the cost of more modifications, if desired.
darioush
commented
Oct 1, 2024
Signed-off-by: Darioush Jalali <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why this should be merged
The current trie_prefetcher is modified significantly compared to upstream, which makes it difficult to merge changes from upstream.
This PR aims to preserve the existing behavior but only with minor inline modifications to upstream code.
I would be open to other structures with the same idea.
How this works
Uses a wrapper for DB to track the central worker pool.
Adds a couple hooks for stopping the background processes.
How this was tested
CI