Feat: Optimistic fast finality #316

* feat: implement bls accounts and keymanager * chore: Implement `CreateAccountsKeystore` function * chore: remove prysm import from keymanager

We randomly observe this failure when running unit test go test -test.v -run=^TestSignatureBatch_AggregateBatch/common_and_uncommon_messages_in_batch_with_multiple_messages === RUN TestSignatureBatch_AggregateBatch === RUN TestSignatureBatch_AggregateBatch/common_and_uncommon_messages_in_batch_with_multiple_messages signature_batch_test.go:643: AggregateBatch() Descriptions got = [test signature bls aggregated signature test signature bls aggregated signature test signature bls aggregated signature], want [bls aggregated signature test signature bls aggregated signature test signature bls aggregated signature test signature] --- FAIL: TestSignatureBatch_AggregateBatch (0.02s) --- FAIL: TestSignatureBatch_AggregateBatch/common_and_uncommon_messages_in_batch_with_multiple_messages (0.02s) The problem is that the signature sort forgets to swap the description when a swap occurs. This commit adds the description swap when swap occurs.

* feat: implement vote manager logics * chore: remove SourceNumber and SourceHash from vote * chore: Add `Debug` function which allow debugging within `VoteManager` * nit: Remove unused code * chore: rename `VoteAddress` to `PublicKey` * core/vote: validator only votes for the higher block height This commit implements new rule that allow validator to only vote for higher block height than its previous vote. With this vote journal is not necessary anymore so we remove it. * cmd, eth: init the vote manager and vote pool * vote_signer: clean up code --------- Co-authored-by: Bui Quang Minh <[email protected]>

RawVoteEnvelop is the same as VoteEnvelop without cached hash

This commit limits the number of votes in future queue per peer as the vote in future queue is not fully verified. The check happens before the costly basicVerify which has to verify the BLS signature.

We randomly get error in CI Caught SIGILL in blst_cgo_init, consult <blst>/bindinds/go/README.md. As in blst documentation, we need to provide CGO_CFLAGS="-O -D__BLST_PORTABLE__" to make the compiled blst portable across different CPU. This commit adds this CGO_CFLAGS to the build process.

* param: introduce Shillin hardfork * consortium/v2: add finality vote to consensus layer This commit extends the snapshot to contains additional BLS public key of the validators. The new information is also added to header's extra data like aggregated BLS signature and voted validator information. A new system transaction is created to reward the finality voted validators. * consortium/v2: test header encoding and finality signature verification

#303) * core/blockchain: choose the chain with highest justified block in fork choice This commit introduces the new logic to fork choice to choose the fork with higher justified block before comparing total diffculty. In case the justified blocks in forks have the same height, we fallback to compare total difficulty same as before. * rpc, backend: add finalized block the block query

…ol (#307) * eth/protocols: implement finality vote communication via ronin protocol ronin protocol is an extension protocol of eth, a peer is required to support eth protocol to run ronin protocol. ronin protocol currently only supports NewVotePacket which contains a list of finality votes. * eth/protocols: don't broadcast known finality vote to peer If a vote is broadcasted from a peer or we have broadcasted to that peer already, we mark this vote is known by that peer and does not broadcast this vote to that peer again. * eth/protocols: test finality vote broadcast via ronin protocol

The FetchVoteByBlockHash is called by the consensus engine and we don't want this function to block the consensus engine's operations. Currently, FetchVoteByBlockHash uses RLock to wait to acquire the read lock but as the writers are uncontrolled, it can be blocked when there are a lot of spamming writers. This commit makes FetchVoteByBlockHash to use TryRLock to keep polling for the lock n times and returns nil in case it cannot acquire the lock. This helps to bound the max duration a caller must wait on this function.

…roof (#314) This precompiled contract is called by system contract to verify signature from the provided proof of finality vote violation. It checkes if there is a validator has contributed to 2 aggregated signatures on the 2 distinct block with the same height.

* consortium: add GetActiveValidatorAt to get validators can vote for a block * monitor/finality_vote: add finality vote monitor This monitor gets the information from block header and check if a voter votes for 2 distinct block at the same height.

The blsCommon.PublicKey cannot be un/marshalled to json, this commit converts that field to hex string before marshalling to json.

Currently, we only add CGO flags when running 'make ronin', so sometimes we still get the error related to blst library built without the CGO_CFLAGS. This commit adds the CGO flags to all places when we use 'go run' to build the binary.

This commit adds the contract interaction to get BLS public when preparing checkpoint block. This also creates a getCheckpointValidatorsFromContract function to be used in both Prepare and Finalize the checkpoint block. Besides, a new interface ContractInteraction is created for mock test contract interaction.

When the vote pool is enabled, make the consortium consensus engine use it.

The commit fixes the bug when collects and aggregates signatures from vote pool.

…ock number (#321) In fork choice, we only need to compare the difficulty of forks when the justified block numbers are the same in those forks. Otherwise, we choose the fork with higher justified block number. This commit fixes a bug in current fork choice to make it follow the correct behavior.

Due to the change in REP-0003, a block is finalized when there are at least floor(2/3*numValidators) + 1, this commit changes the calculation to follow the rule.

This commit adds 2 APIs in consortiumv2 namespace: consortiumv2_getValidatorAtHash, consortiumv2_getFinalityVoteAtHash. consortiumv2_getValidatorAtHash returns the authorized validators that can seal block hash with BLS public key. consortiumv2_getFinalityVoteAtHash returns the finality vote information in the block header at block hash.

This commit adds account commands: - listbls: show the BLS accounts information - importbls: import BLS secret key - checkbls: check if the BLS account with the provided secret key already exists - generatebls: generate BLS secret key This commit also adds BLS management code to entrypoint.sh for running node using docker.

* consortium: add VerifyVote to the outer consortium As required by the FastFinalityPoSA interface, VerifyVote is needed in the outer consortium. We add VerifyVote to outer consortium which simply forwards to the consortium v2 function. * consortium/v2: access blsPublicKey only after Shillin The BLS public key is only available in checkpoint header after Shillin, so don't access this field before Shillin.

…lds (#331)

… copy (#329)

…#330) Currently, to check if the justified block number - 1 is justfied, we look at the snapshot at justfied block number - 1. This is incorrect because at the snapshot N, the maximum justified block number is N - 1. So, we need to look at the snapshot at justified block number to check if the justified block number - 1 is justified.

…ge (#326) * entrypoint: fix bash script syntax * Dockerfile: add required libstdc++ package

* protocols/ronin: remove redudant finality vote channel put * vote/vote_pool: make PutVote non-blocking, drop vote in case of failure

…ion (#335)

* vote/vote_pool: reject older than justified block number vote The vote which targets the block number that is older or equal to justified block number is not useful anymore. We can reject and prune those votes from pool. * vote/vote_pool: prune the future vote map and queue

As we delay assembling finality vote until Seal function, it means the seal hash can be updated after the FinalizeAndAssemble. This breaks the worker which uses seal hash to store and look up seal tasks between FinalizeAndAssemble and Seal function. Worker uses SealHash function to get the header hash and this function is used by worker only not by any other parts in the consensus. So in this commit, we make this function returns a hash of header without the finality vote. This is not the real hash used for sealing but it remains unchanged after FinalizeAndAssemble which meet the requirement of the worker.

The contract field in consortium v2 consensus engine is written under the call to Authorize. The Authorize function is called only once in startup and can run concurrently with other funtions that read the contract field which leads to data race. We move this write under the mutex lock and the read of this field into readSignerAndContract function to avoid data race.

Currently, a vote with wrong target number is not assembled into block as there is a check in assembleFinalityVote. In this commit, we move that check to VerifyVote to reject the vote earlier in vote pool, avoid the vote pool to be DOSed.

…ion (#347)

blst v0.3.11 fixes this vulnerability. This vulnerability does not affect us because the wrapper we use does not call SigValidate with sigInfcheck argument set to true. However, it is not bad to bump to the latest version.

…ock (#337) * consortium/v2: distribute finality reward for the voters in parent block Because the finality votes are assembled after we FinalizeAndAssemble the block and the finality reward distribution transaction is in FinalizeAndAssemble, we cannot reward the finality voters in the current block right away. Instead, the reward distribution is delayed until next block. So in the FinalizeAndAssemble, we look at the finality voters in the parent header and distribute reward to them. * consortium/v2: integrate with the finality tracking contract This commit integrates with the finality tracking contract (axieinfinity/ronin-dpos-contracts#279) to record the finality reward for voters.

This commit adds a Shillin hardfork on testnet to enable optimistic fast finality. The hardfork is scheduled around Sept 12 2023, 7:00 GMT+7 (https://saigon-app.roninchain.com/block/20268000)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat: Optimistic fast finality #316

Feat: Optimistic fast finality #316

Commits on Sep 7, 2023