From 6fb347977e55a787e56bfbbd58e1c0f24f9cf731 Mon Sep 17 00:00:00 2001 From: John Adler Date: Mon, 14 Jun 2021 09:37:25 -0400 Subject: [PATCH 1/7] Add fork choice rule under DAS doc. --- src/README.md | 1 + src/SUMMARY.md | 1 + src/rationale/README.md | 1 + src/rationale/fork_choice_das.md | 39 ++++++++++++++++++++++++++++++++ 4 files changed, 42 insertions(+) create mode 100644 src/rationale/fork_choice_das.md diff --git a/src/README.md b/src/README.md index 6caaa29..a85bbc7 100644 --- a/src/README.md +++ b/src/README.md @@ -11,4 +11,5 @@ - [Rationale](./rationale/README.md) - [Block Rewards](./rationale/rewards.md) - [Distributing Rewards and Penalties](./rationale/distributing_rewards.md) + - [Fork Choice Rule with Data Availability Sampling](./rationale/fork_choice_das.md) - [Message Layout](./rationale/message_block_layout.md) diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 30e3838..c80c548 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -13,4 +13,5 @@ - [Rationale](./rationale/README.md) - [Block Rewards](./rationale/rewards.md) - [Distributing Rewards and Penalties](./rationale/distributing_rewards.md) + - [Fork Choice Rule with Data Availability Sampling](./rationale/fork_choice_das.md) - [Message Layout](./rationale/message_block_layout.md) diff --git a/src/rationale/README.md b/src/rationale/README.md index dcf6314..a8af13b 100644 --- a/src/rationale/README.md +++ b/src/rationale/README.md @@ -2,4 +2,5 @@ - [Block Rewards](./rewards.md) - [Distributing Rewards and Penalties](./distributing_rewards.md) +- [Fork Choice Rule with Data Availability Sampling](./fork_choice_das.md) - [Message Layout](./message_block_layout.md) diff --git a/src/rationale/fork_choice_das.md b/src/rationale/fork_choice_das.md new file mode 100644 index 0000000..1e89ae8 --- /dev/null +++ b/src/rationale/fork_choice_das.md @@ -0,0 +1,39 @@ +# Fork Choice Rule with Data Availability Sampling + +- [Preamble](#preamble) +- [Invalid vs Unavailable](#invalid-vs-unavailable) +- [Scenarios](#scenarios) + +## Preamble + +Tendermint provides finality under an honest 2/3 of stake assumption. It is one of several ["BFT" consensus protocols](https://arxiv.org/abs/1807.04938) (also known as "classical" consensus protocols). Under that assumptions, new _valid_ blocks are immediately and forever final as soon as 2/3 of stake commits to the block. Therefore, under that assumption, Tendermint is fork-free. + +Contemporary blockchains support full nodes (which are secure under no assumption on stake honesty) and light nodes (which are secure under an honest majority of stake assumption). LazyLedger is unique in [supporting light nodes with stronger security guarantees](./../specs/node_types.md#node-type-definitions): + +1. full nodes are secure under no assumptions on stake honesty +1. light nodes (and partial nodes) are secure under [an honest minority of nodes and synchronous communication](https://arxiv.org/abs/1809.09044), and no assumptions on stake honesty +1. superlight nodes are secure under an honest majority of stake assumption + +The introduction of light nodes that do not depend on an honest majority assumption also introduces additional cases that must be analyzed. + +## Invalid vs Unavailable + +Tendermint (and other consensus protocols) requires blocks to be _valid_, i.e. pass a [validity predicate](https://arxiv.org/abs/1807.04938) before they are accepted by an honest node. Note that both validity and invalidity are deterministic and monotonic, i.e. that once a block is valid or invalid, it will be valid or invalid for all future time. + +With [Data Availability Sampling](https://arxiv.org/abs/1809.09044) (DAS), there is a notion of _available_ and _unavailable_ blocks. Both are probabilistic rather than deterministic. Availability is assumed monotonic (i.e. once a block is available, it will remain available since The Internet Never Forgets), but unavailability is not. A block proposer may hide a block to make currently-online nodes see the block as unavailable, then reveal the entire (valid) block at a later time. + +## Scenarios + +We consider two scenarios. + +**A dishonest majority hide a committed block, commit to a second block at the same height within the weak subjectivity window to fork the chain, then reveal the first block**. This is trivially equivocation and requires social consensus to resolve which fork to accept. The unavailability of the first block is orthogonal. Nodes that detect equivocation by a majority of stake within the weak subjectivity window must halt regardless. + +**A dishonest majority hide a committed block, commit additional blocks on top of it, then reveal the first block within the weak subjectivity window**. There is no equivocation. Note that a node cannot distinguish a dishonest majority in this scenario from a transient network failure on their end and an honest majority. + +A requirement is that full nodes and light nodes agree on the same head of the chain automatically in this case, i.e. without human intervention. + +Light nodes follow consensus (i.e. validator set changes and commits) and perform DAS. If a block is seen as unavailable but has a commit, DAS is performed on the block continuously until either DAS passes, or the weak subjectivity window is exceeded at which point the node halts. + +Full nodes fully download and execute blocks. If a block is seen as unavailable but has a commit, full downloading is re-attempted continuously until either it succeeds, or the weak subjectivity window is exceeded at which point the node halts. + +Under [an honest minority of nodes and synchronous communication](https://arxiv.org/abs/1809.09044) assumptions, passing DAS probabilistically guarantees the block can be fully downloaded. Therefore, the above protocol guarantees light nodes and full nodes will agree on the same head. From e68aff90b6e295377af0c2c8c3d52b875d79c08d Mon Sep 17 00:00:00 2001 From: John Adler Date: Mon, 14 Jun 2021 09:55:05 -0400 Subject: [PATCH 2/7] Clean up. --- src/rationale/fork_choice_das.md | 2 +- src/specs/node_types.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/src/rationale/fork_choice_das.md b/src/rationale/fork_choice_das.md index 1e89ae8..ebf4af8 100644 --- a/src/rationale/fork_choice_das.md +++ b/src/rationale/fork_choice_das.md @@ -8,7 +8,7 @@ Tendermint provides finality under an honest 2/3 of stake assumption. It is one of several ["BFT" consensus protocols](https://arxiv.org/abs/1807.04938) (also known as "classical" consensus protocols). Under that assumptions, new _valid_ blocks are immediately and forever final as soon as 2/3 of stake commits to the block. Therefore, under that assumption, Tendermint is fork-free. -Contemporary blockchains support full nodes (which are secure under no assumption on stake honesty) and light nodes (which are secure under an honest majority of stake assumption). LazyLedger is unique in [supporting light nodes with stronger security guarantees](./../specs/node_types.md#node-type-definitions): +Contemporary blockchains support full nodes (which are secure under no assumption on stake honesty) and light nodes (which are secure under an honest majority of stake assumption). LazyLedger is unique in [supporting light nodes with stronger security guarantees](../specs/node_types.md#node-type-definitions): 1. full nodes are secure under no assumptions on stake honesty 1. light nodes (and partial nodes) are secure under [an honest minority of nodes and synchronous communication](https://arxiv.org/abs/1809.09044), and no assumptions on stake honesty diff --git a/src/specs/node_types.md b/src/specs/node_types.md index 924bb29..0191319 100644 --- a/src/specs/node_types.md +++ b/src/specs/node_types.md @@ -71,7 +71,7 @@ At most secure under an honest minority of nodes and a weak subjectivity assumpt #### Full Transactions -Nodes that wish to produce new blocks must know the [chain state](./data_structures.md#state). Processing all block bodies is actually not needed to know the LazyLedger state, as [transactions that pay for message inclusion commit to messages](./../rationale/message_block_layout.md). These nodes process all requests [with a reserved namespace ID](./data_structures.md#arranging-available-data-into-shares) from block bodies and perform Data Availability Sampling for the remaining (message) data. +Nodes that wish to produce new blocks must know the [chain state](./data_structures.md#state). Processing all block bodies is actually not needed to know the LazyLedger state, as [transactions that pay for message inclusion commit to messages](../rationale/message_block_layout.md). These nodes process all requests [with a reserved namespace ID](./data_structures.md#arranging-available-data-into-shares) from block bodies and perform Data Availability Sampling for the remaining (message) data. At most secure under a weak subjectivity assumption. From df507db77d8d912b7dff55375052c398f8d4a59a Mon Sep 17 00:00:00 2001 From: John Adler Date: Mon, 14 Jun 2021 09:57:16 -0400 Subject: [PATCH 3/7] Add fork choice short. --- src/specs/consensus.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/src/specs/consensus.md b/src/specs/consensus.md index 0a9c503..ec76252 100644 --- a/src/specs/consensus.md +++ b/src/specs/consensus.md @@ -96,7 +96,9 @@ TODO ## Fork Choice -TODO +The Tendermint consensus protocol is fork-free by construction under an honest majority of stake assumption. + +If a block has a [valid commit](#blocklastcommit), it is part of the canonical chain. If equivocation [evidence](./data_structures.md#evidence) is detected for more than 1/3 of voting power, the node must halt. See [rationale doc](../rationale/fork_choice_das.md) for more information. ## Block Validity From 8d4a58a6a769126f18eec6539c944c41ad09cdd9 Mon Sep 17 00:00:00 2001 From: John Adler Date: Mon, 14 Jun 2021 09:58:16 -0400 Subject: [PATCH 4/7] stake -> voting power --- src/rationale/fork_choice_das.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/src/rationale/fork_choice_das.md b/src/rationale/fork_choice_das.md index ebf4af8..833f949 100644 --- a/src/rationale/fork_choice_das.md +++ b/src/rationale/fork_choice_das.md @@ -6,13 +6,13 @@ ## Preamble -Tendermint provides finality under an honest 2/3 of stake assumption. It is one of several ["BFT" consensus protocols](https://arxiv.org/abs/1807.04938) (also known as "classical" consensus protocols). Under that assumptions, new _valid_ blocks are immediately and forever final as soon as 2/3 of stake commits to the block. Therefore, under that assumption, Tendermint is fork-free. +Tendermint provides finality under an honest 2/3 of voting power assumption. It is one of several ["BFT" consensus protocols](https://arxiv.org/abs/1807.04938) (also known as "classical" consensus protocols). Under that assumptions, new _valid_ blocks are immediately and forever final as soon as 2/3 of voting power commits to the block. Therefore, under that assumption, Tendermint is fork-free. -Contemporary blockchains support full nodes (which are secure under no assumption on stake honesty) and light nodes (which are secure under an honest majority of stake assumption). LazyLedger is unique in [supporting light nodes with stronger security guarantees](../specs/node_types.md#node-type-definitions): +Contemporary blockchains support full nodes (which are secure under no assumption on voting power honesty) and light nodes (which are secure under an honest majority of voting power assumption). LazyLedger is unique in [supporting light nodes with stronger security guarantees](../specs/node_types.md#node-type-definitions): -1. full nodes are secure under no assumptions on stake honesty -1. light nodes (and partial nodes) are secure under [an honest minority of nodes and synchronous communication](https://arxiv.org/abs/1809.09044), and no assumptions on stake honesty -1. superlight nodes are secure under an honest majority of stake assumption +1. full nodes are secure under no assumptions on voting power honesty +1. light nodes (and partial nodes) are secure under [an honest minority of nodes and synchronous communication](https://arxiv.org/abs/1809.09044), and no assumptions on voting power honesty +1. superlight nodes are secure under an honest majority of voting assumption The introduction of light nodes that do not depend on an honest majority assumption also introduces additional cases that must be analyzed. @@ -26,7 +26,7 @@ With [Data Availability Sampling](https://arxiv.org/abs/1809.09044) (DAS), there We consider two scenarios. -**A dishonest majority hide a committed block, commit to a second block at the same height within the weak subjectivity window to fork the chain, then reveal the first block**. This is trivially equivocation and requires social consensus to resolve which fork to accept. The unavailability of the first block is orthogonal. Nodes that detect equivocation by a majority of stake within the weak subjectivity window must halt regardless. +**A dishonest majority hide a committed block, commit to a second block at the same height within the weak subjectivity window to fork the chain, then reveal the first block**. This is trivially equivocation and requires social consensus to resolve which fork to accept. The unavailability of the first block is orthogonal. Nodes that detect equivocation by a majority of voting power within the weak subjectivity window must halt regardless. **A dishonest majority hide a committed block, commit additional blocks on top of it, then reveal the first block within the weak subjectivity window**. There is no equivocation. Note that a node cannot distinguish a dishonest majority in this scenario from a transient network failure on their end and an honest majority. From d76e65b9e04f50b14a03c20c847f60ce83f40993 Mon Sep 17 00:00:00 2001 From: John Adler Date: Mon, 14 Jun 2021 09:59:46 -0400 Subject: [PATCH 5/7] Clarify secure. --- src/rationale/fork_choice_das.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/rationale/fork_choice_das.md b/src/rationale/fork_choice_das.md index 833f949..1619008 100644 --- a/src/rationale/fork_choice_das.md +++ b/src/rationale/fork_choice_das.md @@ -14,7 +14,7 @@ Contemporary blockchains support full nodes (which are secure under no assumptio 1. light nodes (and partial nodes) are secure under [an honest minority of nodes and synchronous communication](https://arxiv.org/abs/1809.09044), and no assumptions on voting power honesty 1. superlight nodes are secure under an honest majority of voting assumption -The introduction of light nodes that do not depend on an honest majority assumption also introduces additional cases that must be analyzed. +Note that _secure_ in this context is defined as "not accepting invalid blocks," as is orthogonal to _consensus safety_. The introduction of light nodes that do not depend on an honest majority assumption also introduces additional cases that must be analyzed. ## Invalid vs Unavailable From b7b3488244b5391a9a913767e57700b93fe51090 Mon Sep 17 00:00:00 2001 From: John Adler Date: Fri, 25 Jun 2021 10:00:36 -0400 Subject: [PATCH 6/7] Update src/rationale/fork_choice_das.md Co-authored-by: Ismail Khoffi --- src/rationale/fork_choice_das.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/rationale/fork_choice_das.md b/src/rationale/fork_choice_das.md index 1619008..d748b7e 100644 --- a/src/rationale/fork_choice_das.md +++ b/src/rationale/fork_choice_das.md @@ -32,7 +32,7 @@ We consider two scenarios. A requirement is that full nodes and light nodes agree on the same head of the chain automatically in this case, i.e. without human intervention. -Light nodes follow consensus (i.e. validator set changes and commits) and perform DAS. If a block is seen as unavailable but has a commit, DAS is performed on the block continuously until either DAS passes, or the weak subjectivity window is exceeded at which point the node halts. +Light nodes follow consensus state (i.e. validator set changes and commits) and perform DAS. If a block is seen as unavailable but has a commit, DAS is performed on the block continuously until either DAS passes, or the weak subjectivity window is exceeded at which point the node halts. Full nodes fully download and execute blocks. If a block is seen as unavailable but has a commit, full downloading is re-attempted continuously until either it succeeds, or the weak subjectivity window is exceeded at which point the node halts. From 4e0092ae630eee7f51dcb0980326b863e0d4445d Mon Sep 17 00:00:00 2001 From: John Adler Date: Sun, 27 Jun 2021 11:25:11 -0400 Subject: [PATCH 7/7] Clarify timeouts and impl details. --- src/rationale/fork_choice_das.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/src/rationale/fork_choice_das.md b/src/rationale/fork_choice_das.md index d748b7e..d1aaaab 100644 --- a/src/rationale/fork_choice_das.md +++ b/src/rationale/fork_choice_das.md @@ -36,4 +36,8 @@ Light nodes follow consensus state (i.e. validator set changes and commits) and Full nodes fully download and execute blocks. If a block is seen as unavailable but has a commit, full downloading is re-attempted continuously until either it succeeds, or the weak subjectivity window is exceeded at which point the node halts. -Under [an honest minority of nodes and synchronous communication](https://arxiv.org/abs/1809.09044) assumptions, passing DAS probabilistically guarantees the block can be fully downloaded. Therefore, the above protocol guarantees light nodes and full nodes will agree on the same head. +Under [an honest minority of nodes and synchronous communication](https://arxiv.org/abs/1809.09044) assumptions, passing DAS probabilistically guarantees the block can be fully downloaded. Therefore, the above protocol guarantees light nodes and full nodes will agree on the same head automatically without manual intervention, under a synchrony assumption equal to the unbonding window (formal proof pending). + +> Note: whether re-downloading/re-sampling is attempted at a fixed interval or on receipt of a new block header with a Tendermint commit is an implementation detail. + +> Note: users may wish to set a much shorter timeout than the unbonding window before they are notified of their node being unable to verify the availability of a committed block. This does not affect the properties analyzed above, but may result in more manual attention in the event of intermittent network disruptions.