Feat: Optimistic fast finality #316
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
minh-bq
requested changes
Jul 20, 2023
minh-bq
reviewed
Jul 21, 2023
phuctd95
commented
Jul 24, 2023
phuctd95
commented
Jul 25, 2023
minh-bq
force-pushed
the
feat/optimistic_fast_finality
branch
from
4ababdc
to
4a794d8
Compare
* feat: implement bls accounts and keymanager * chore: Implement `CreateAccountsKeystore` function * chore: remove prysm import from keymanager
We randomly observe this failure when running unit test
go test -test.v -run=^TestSignatureBatch_AggregateBatch/common_and_uncommon_messages_in_batch_with_multiple_messages
=== RUN TestSignatureBatch_AggregateBatch
=== RUN TestSignatureBatch_AggregateBatch/common_and_uncommon_messages_in_batch_with_multiple_messages
signature_batch_test.go:643: AggregateBatch() Descriptions got = [test signature bls aggregated signature test signature bls aggregated signature test signature bls aggregated signature], want [bls aggregated signature test signature bls aggregated signature test signature bls aggregated signature test signature]
--- FAIL: TestSignatureBatch_AggregateBatch (0.02s)
--- FAIL: TestSignatureBatch_AggregateBatch/common_and_uncommon_messages_in_batch_with_multiple_messages (0.02s)
The problem is that the signature sort forgets to swap the description when a
swap occurs. This commit adds the description swap when swap occurs.
* feat: implement vote manager logics * chore: remove SourceNumber and SourceHash from vote * chore: Add `Debug` function which allow debugging within `VoteManager` * nit: Remove unused code * chore: rename `VoteAddress` to `PublicKey` * core/vote: validator only votes for the higher block height This commit implements new rule that allow validator to only vote for higher block height than its previous vote. With this vote journal is not necessary anymore so we remove it. * cmd, eth: init the vote manager and vote pool * vote_signer: clean up code --------- Co-authored-by: Bui Quang Minh <[email protected]>
RawVoteEnvelop is the same as VoteEnvelop without cached hash
This commit limits the number of votes in future queue per peer as the vote in future queue is not fully verified. The check happens before the costly basicVerify which has to verify the BLS signature.
We randomly get error in CI Caught SIGILL in blst_cgo_init, consult <blst>/bindinds/go/README.md. As in blst documentation, we need to provide CGO_CFLAGS="-O -D__BLST_PORTABLE__" to make the compiled blst portable across different CPU. This commit adds this CGO_CFLAGS to the build process.
* param: introduce Shillin hardfork * consortium/v2: add finality vote to consensus layer This commit extends the snapshot to contains additional BLS public key of the validators. The new information is also added to header's extra data like aggregated BLS signature and voted validator information. A new system transaction is created to reward the finality voted validators. * consortium/v2: test header encoding and finality signature verification
#303) * core/blockchain: choose the chain with highest justified block in fork choice This commit introduces the new logic to fork choice to choose the fork with higher justified block before comparing total diffculty. In case the justified blocks in forks have the same height, we fallback to compare total difficulty same as before. * rpc, backend: add finalized block the block query
…ol (#307) * eth/protocols: implement finality vote communication via ronin protocol ronin protocol is an extension protocol of eth, a peer is required to support eth protocol to run ronin protocol. ronin protocol currently only supports NewVotePacket which contains a list of finality votes. * eth/protocols: don't broadcast known finality vote to peer If a vote is broadcasted from a peer or we have broadcasted to that peer already, we mark this vote is known by that peer and does not broadcast this vote to that peer again. * eth/protocols: test finality vote broadcast via ronin protocol
The FetchVoteByBlockHash is called by the consensus engine and we don't want this function to block the consensus engine's operations. Currently, FetchVoteByBlockHash uses RLock to wait to acquire the read lock but as the writers are uncontrolled, it can be blocked when there are a lot of spamming writers. This commit makes FetchVoteByBlockHash to use TryRLock to keep polling for the lock n times and returns nil in case it cannot acquire the lock. This helps to bound the max duration a caller must wait on this function.
…roof (#314) This precompiled contract is called by system contract to verify signature from the provided proof of finality vote violation. It checkes if there is a validator has contributed to 2 aggregated signatures on the 2 distinct block with the same height.
* consortium: add GetActiveValidatorAt to get validators can vote for a block * monitor/finality_vote: add finality vote monitor This monitor gets the information from block header and check if a voter votes for 2 distinct block at the same height.
The blsCommon.PublicKey cannot be un/marshalled to json, this commit converts that field to hex string before marshalling to json.
Currently, we only add CGO flags when running 'make ronin', so sometimes we still get the error related to blst library built without the CGO_CFLAGS. This commit adds the CGO flags to all places when we use 'go run' to build the binary.
This commit adds the contract interaction to get BLS public when preparing checkpoint block. This also creates a getCheckpointValidatorsFromContract function to be used in both Prepare and Finalize the checkpoint block. Besides, a new interface ContractInteraction is created for mock test contract interaction.
When the vote pool is enabled, make the consortium consensus engine use it.
The commit fixes the bug when collects and aggregates signatures from vote pool.
…ock number (#321) In fork choice, we only need to compare the difficulty of forks when the justified block numbers are the same in those forks. Otherwise, we choose the fork with higher justified block number. This commit fixes a bug in current fork choice to make it follow the correct behavior.
Due to the change in REP-0003, a block is finalized when there are at least floor(2/3*numValidators) + 1, this commit changes the calculation to follow the rule.
This commit adds 2 APIs in consortiumv2 namespace: consortiumv2_getValidatorAtHash, consortiumv2_getFinalityVoteAtHash. consortiumv2_getValidatorAtHash returns the authorized validators that can seal block hash with BLS public key. consortiumv2_getFinalityVoteAtHash returns the finality vote information in the block header at block hash.
This commit adds account commands: - listbls: show the BLS accounts information - importbls: import BLS secret key - checkbls: check if the BLS account with the provided secret key already exists - generatebls: generate BLS secret key This commit also adds BLS management code to entrypoint.sh for running node using docker.
* consortium: add VerifyVote to the outer consortium As required by the FastFinalityPoSA interface, VerifyVote is needed in the outer consortium. We add VerifyVote to outer consortium which simply forwards to the consortium v2 function. * consortium/v2: access blsPublicKey only after Shillin The BLS public key is only available in checkpoint header after Shillin, so don't access this field before Shillin.
…#330) Currently, to check if the justified block number - 1 is justfied, we look at the snapshot at justfied block number - 1. This is incorrect because at the snapshot N, the maximum justified block number is N - 1. So, we need to look at the snapshot at justified block number to check if the justified block number - 1 is justified.
…ge (#326) * entrypoint: fix bash script syntax * Dockerfile: add required libstdc++ package
* protocols/ronin: remove redudant finality vote channel put * vote/vote_pool: make PutVote non-blocking, drop vote in case of failure
* vote/vote_pool: reject older than justified block number vote The vote which targets the block number that is older or equal to justified block number is not useful anymore. We can reject and prune those votes from pool. * vote/vote_pool: prune the future vote map and queue
As we delay assembling finality vote until Seal function, it means the seal hash can be updated after the FinalizeAndAssemble. This breaks the worker which uses seal hash to store and look up seal tasks between FinalizeAndAssemble and Seal function. Worker uses SealHash function to get the header hash and this function is used by worker only not by any other parts in the consensus. So in this commit, we make this function returns a hash of header without the finality vote. This is not the real hash used for sealing but it remains unchanged after FinalizeAndAssemble which meet the requirement of the worker.
The contract field in consortium v2 consensus engine is written under the call to Authorize. The Authorize function is called only once in startup and can run concurrently with other funtions that read the contract field which leads to data race. We move this write under the mutex lock and the read of this field into readSignerAndContract function to avoid data race.
Currently, a vote with wrong target number is not assembled into block as there is a check in assembleFinalityVote. In this commit, we move that check to VerifyVote to reject the vote earlier in vote pool, avoid the vote pool to be DOSed.
blst v0.3.11 fixes this vulnerability. This vulnerability does not affect us because the wrapper we use does not call SigValidate with sigInfcheck argument set to true. However, it is not bad to bump to the latest version.
…ock (#337) * consortium/v2: distribute finality reward for the voters in parent block Because the finality votes are assembled after we FinalizeAndAssemble the block and the finality reward distribution transaction is in FinalizeAndAssemble, we cannot reward the finality voters in the current block right away. Instead, the reward distribution is delayed until next block. So in the FinalizeAndAssemble, we look at the finality voters in the parent header and distribute reward to them. * consortium/v2: integrate with the finality tracking contract This commit integrates with the finality tracking contract (axieinfinity/ronin-dpos-contracts#279) to record the finality reward for voters.
This commit adds a Shillin hardfork on testnet to enable optimistic fast finality. The hardfork is scheduled around Sept 12 2023, 7:00 GMT+7 (https://saigon-app.roninchain.com/block/20268000)
minh-bq
force-pushed
the
feat/optimistic_fast_finality
branch
from
66bb8c6
to
716f8b5
Compare
minh-bq
approved these changes
Sep 7, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.