Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NEP-568: Resharding V3 #568

Open
wants to merge 30 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
7ef6857
template
wacban Oct 24, 2024
55bf0b3
rename
wacban Oct 24, 2024
2ac4b05
metadata
wacban Oct 24, 2024
5ef6b9d
summary
wacban Oct 24, 2024
62f127c
specification
wacban Oct 24, 2024
bec1ae6
formatting template texts
wacban Oct 24, 2024
d72d509
fix lints
wacban Oct 28, 2024
47f22f9
Resharding V3 - added future possibilities (#569)
wacban Oct 29, 2024
8d8e761
Add flat state specs to resharding NEP (#570)
Trisfald Oct 29, 2024
1ee5a74
fix lints
wacban Oct 29, 2024
6c98441
State Storage - State (#571)
staffik Nov 6, 2024
08b1834
fix lint
staffik Nov 6, 2024
6e431ff
fix lint
staffik Nov 6, 2024
5368857
ShardId Semantics (#572)
wacban Nov 7, 2024
a7bbee7
ReshardingV3 memtrie (#574)
shreyan-gupta Nov 14, 2024
65ece27
lint
shreyan-gupta Nov 14, 2024
615a92f
Add reference implementation for Flat state to resharding NEP (#575)
Trisfald Nov 15, 2024
e488308
fix lint
Trisfald Nov 15, 2024
12fd994
motivation (#576)
wacban Nov 18, 2024
03eeeea
move some summary to motivation
wacban Nov 19, 2024
080ac3c
Resharding V3 - add a few state sync details (#573)
marcelo-gonzalez Nov 20, 2024
910cf6c
Resharding V3 - state witness, implementation (#577)
Longarithm Nov 22, 2024
8ab6927
[resharding] Add sections for receipt handling (#578)
shreyan-gupta Nov 25, 2024
09dc287
cross shard traffic
wacban Dec 4, 2024
d889993
state sync: reword to make clear it's not just for convenience (#579)
marcelo-gonzalez Dec 13, 2024
0ae4721
resharding: cold storage (#580)
staffik Dec 13, 2024
cf3174f
lint
wacban Dec 13, 2024
c4f5e30
Add section on buffered receipt handling
shreyan-gupta Dec 16, 2024
1efd6cf
[resharding] ChatGPT modulation (#581)
shreyan-gupta Dec 16, 2024
74301d0
fix(resharding): update pseudocode (#583)
staffik Jan 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions neps/nep-0568.md
wacban marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,46 @@ The change we propose is to move the initial state download point to one in the

### Cross Shard Traffic

When the shard layout changes, it is crucial to handle cross-shard traffic
correctly, especially in the presence of missing chunks. Care must be taken to
ensure that no receipt is lost or duplicated. There are two important receipt
types that need to be considered - the outgoing receipts and the incoming
receipts.

Note - this proposal reuses the approach taken by ReshardingV2.

#### Outgoing Receipts
wacban marked this conversation as resolved.
Show resolved Hide resolved

Each new chunk in a shard contains a list of outgoing receipts generated during
the processing of the previous chunk in that shard.

In cases where chunks are missing at the resharding boundary, both child shards
could theoretically include the outgoing receipts from their shared ancestor
chunk. However, this naive approach would lead to the duplication of receipts,
which must be avoided.

The proposed solution is to reassign the outgoing receipts from the parent chunk
to only one of the child shards. Specifically, the child shard with the lower
shard ID will claim all outgoing receipts from the parent, while the other child
will receive none. This ensures that all receipts are processed exactly once.

#### Incoming Receipts
wacban marked this conversation as resolved.
Show resolved Hide resolved

To process a chunk in a shard, it is necessary to gather all outgoing receipts
from other shards that are targeted at this shard. These receipts must then be
included as incoming receipts.

In the presence of missing chunks, the new chunk must collect receipts from all
previous blocks, spanning the period since the last new chunk in this shard.
This range may cross the resharding boundary.

When this occurs, the chunk must also consider receipts that were previously
targeted at its parent shard. However, it must filter these receipts to include
only those where the recipient lies within the current shard, discarding those
where the recipient belongs to the sibling shard in the new shard layout. This
filtering process ensures that every receipt is processed exactly once and in
the correct shard.

### Delayed Receipt Handling

The delayed receipts queue contains all incoming receipts that could not be executed as part of a block due to resource constraints like compute cost, gas limits etc. The entries in the delayed receipt queue can belong to any of the accounts as part of the shard. During a resharding event, we ideally need to split the delayed receipts across both the child shards according to the associated account_id with the receipt.
Expand Down
Loading