Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

state_object, trie: update storage trie of an account parallel #352

Closed
wants to merge 1 commit into from

Conversation

minh-bq
Copy link
Contributor

@minh-bq minh-bq commented Sep 7, 2023

We observe the high latency when validating block, which finalises all storage changes from statedb to trie layer. We want to parallel the trie operations but parallel means some synchronization methods are need and they cost some overhead. In this commit, we don't parallel all trie operations, we just parallel the leaf value node update of the storage trie of an account. This operation does not change the trie structure, which is simpler to synchronize comparing to delete or insert a new node.

When updating the leaf node, we need to take care of the hash node resolve, 2 updates cannot resolve the same hash node parallel. We introduce read-write mutex into the full node and short node to solve this problem. When the child of these node are hash node, a hash node resolve must happens inside the lock critical section.

Quick benchmark with spamming batch transfer Axies:
image

devnet-8 is the machine with this parallel storage trie update

@minh-bq minh-bq force-pushed the feat/state-obj-update branch 2 times, most recently from 65393bc to daccfbc Compare January 4, 2024 09:39
We observe the high latency when validating block, which finalises all storage
changes from statedb to trie layer. We want to parallel the trie operations but
parallel means some synchronization methods are need and they cost some
overhead. In this commit, we don't parallel all trie operations, we just
parallel the leaf value node update of the storage trie of an account. This
operation does not change the trie structure, which is simpler to synchronize
comparing to delete or insert a new node.

When updating the leaf node, we need to take care of the hash node resolve, 2
updates cannot resolve the same hash node parallel. We introduce read-write
mutex into the full node and short node to solve this problem. When the child of
these node are hash node, a hash node resolve must happens inside the lock
critical section.
@minh-bq minh-bq closed this Mar 4, 2024
@minh-bq minh-bq deleted the feat/state-obj-update branch March 4, 2024 09:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant