state_object, trie: update storage trie of an account parallel #352
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We observe the high latency when validating block, which finalises all storage changes from statedb to trie layer. We want to parallel the trie operations but parallel means some synchronization methods are need and they cost some overhead. In this commit, we don't parallel all trie operations, we just parallel the leaf value node update of the storage trie of an account. This operation does not change the trie structure, which is simpler to synchronize comparing to delete or insert a new node.
When updating the leaf node, we need to take care of the hash node resolve, 2 updates cannot resolve the same hash node parallel. We introduce read-write mutex into the full node and short node to solve this problem. When the child of these node are hash node, a hash node resolve must happens inside the lock critical section.
Quick benchmark with spamming batch transfer Axies:
devnet-8 is the machine with this parallel storage trie update