Documentation #496

MatheusFranco99 · 2024-08-30T11:12:00Z

Overview

This PR adds and updates the repository's documentation.

It updates the repository's main README.md and the p2p documentation files. It also adds documentation for the qbft, ssv, and types modules.

* Filter shares for slot `CommitteeRunner` based on validators that have duty for that slot. * Filter duty and create share map according to owned validators * Add test: start duty with no shares for duty's validators * Add test: happy flow for committee with fraction of duty's validators * Generate JSON tests * Apply suggestions --------- Co-authored-by: MatheusFranco99 <[email protected]>

* Update go1.20 to go1.22 * Update go.sum with mod tidy

* Update dependencies * Fix lint issue * Generate JSON tests to trigger actions * Update fastssz * Generate JSON tests and align ssz error * Revert go-eth2-client version change * Revert fastssz upgrade * Generate SSZ and JSON tests

* Solve potential file inclusion via variable * Fix file permission (0644 to 0600) * Add nosec comment for PRNG (pseudo-random number generator) used for testing * Fix lint issue on nil check in []byte type * Update permission from 0444 to 0600 * Update 0444 to 0400

* Remove nolint comment and export timeout variables * Drop unnecessary nolint * Add comment * Fix lint issue

* Add share length validation in runner construction * Align to error handling in runners constructions * Add validation to committee runner * Add runner construction tests * Refactor runner construction in testingutil to deal with creation errors * Generate JSON tests * Fix lint issue * Fix comments

* Remove redundant validation * Align error string

* Sort signers in decided message * Add test for sorted signers in decided msg * Generate JSON tests * Fix lint issue

* Stop processing consensus messages after instance is decided * Align error in qbft tests * Align errors in ssv tests * Generate JSON tests * Fix lint issue

* Remove leftover err check * Align argument variable name to type

iurii-ssv

Great summary! I've left some suggestions to clarify some things (hope you don't mind that list is quite long).

iurii-ssv · 2024-10-03T10:26:53Z

ssv/README.md

+- there's a connection to a **Beacon node** that provides duties' related data (e.g. an attestation vote) and allows the node to submit signed data to the blockchain.
+- there's a connection to the **Beacon network** that provides information about the blockchain (e.g. the current slot and fork version).


Seems Beacon network isn't even mentioned in this document ... and it kind of begs the question of how Beacon network is different from Beacon node. Plus (from what I've gathered in the code) Beacon network is just a bunch of data pulled together (and a bunch of functions that compute something on that data) - which is quite a low level implementation detail that isn't desirable to have in high-level overview docs anyway.

So, I would just probably omit the whole section:

there's a connection to the Beacon network that provides information about the blockchain (e.g. the current slot and fork version).

iurii-ssv · 2024-10-03T10:35:24Z

ssv/README.md

+Within the duty's slot, thhe time at which the operators should start the duty depends on the duty type.
+
+- For the Proposer duty, the protocol should be executed as soon as the slot starts (so no delay).
+- For the Attestation and Sync Committee duties, the protocol should be executed after one third (1/3) of the slot duration, i.e. 4 seconds. This helps maximizing the chance of voting for the most recent proposed block.


for the most recent proposed block

Just to verify my understanding,

is the most recent proposed block the same as most likely block to win in Ethereum consensus of this slot in this context ? Perhaps worth clarifying this. Plus, I assume it's 1/3 and not 1/2 or 3/4 or something because there is a time bound (which is probabilistic) on how much we can delay the execution of this duty for the Validator in question to actually perform it (as opposed to miss it) ? Also would appreciate clarification.

iurii-ssv · 2024-10-03T10:37:23Z

ssv/README.md

+- `ValidatorRegistration`: to perform the registration of the validator to the blockchain.
+- `VoluntaryExit`: to exit the validator from the blockchain.


the blockchain -> Ethereum blockchain ?

iurii-ssv · 2024-10-03T10:38:45Z

ssv/README.md

+- `ValidatorRegistration`: to perform the registration of the validator to the blockchain.
+- `VoluntaryExit`: to exit the validator from the blockchain.
+
+Both duties are only composed by the Pre-Consensus phase since only a signature construction is needed over an a prior known data.


perhaps signature construction is needed over an a prior known data. -> signature construction is needed over a priori known data.

iurii-ssv · 2024-10-03T10:44:31Z

ssv/README.md

+Regarding the duties' start time, both should start at the beginning of the slot.
+
+## The Committee duty
+Since the committee may manage several validators, many Attestation duties may need to be performed in the same slot. This represents a workload concern. Luckly, we have that all consensus executions for each attestation duty agree on the same data (the Casper FFG and LMD GHOST votes). Therefore, we can perform a single consensus execution for all validators assigned in the same slot (though a unique Post-Consensus phase is required for each).


for each attestation duty -> for each Attestation duty

all validators assigned in the same slot -> all validators assigned Attestation duty in the same slot

though a unique Post-Consensus phase is required for each -> though a unique Post-Consensus phase is required for each of those duties to be carried out fully

iurii-ssv · 2024-10-03T12:08:33Z

ssv/docs/README.md

@@ -0,0 +1,54 @@
+# Documentation


Maybe also Documentation -> Class diagram

iurii-ssv · 2024-10-03T12:13:29Z

ssv/docs/README.md

+
+`BeaconNode` is an interface that allows the node to fetch duty data (e.g. an Attestation vote and a block to be proposed) and to submit a signed data (e.g. a signed Attestation vote and a signed block).
+
+It also provides a method to get the blockchain domain (a sequence of bytes used in signatures that identifies the blockchain) and it holds a `BeaconNetwork` that provides information about the blockchain such as the current slot and fork version.


identifies the blockchain -> identifies the blockchain, eg. mainnet / holesky / ...

iurii-ssv · 2024-10-03T12:23:10Z

types/docs/README.md

+
+## BeaconNetwork
+
+`BeaconNetwork` is a type that allows accessing information about the blockchain, e.g. such as the current slot, the number of seconds per slot, and the fork version.


Based on what I see in the code,

perhaps worth noting that BeaconNetwork does not need to send requests to ETH2 node (or any requests at all) and is rather just a struct of data bundled together + some compute on this data (executions over some constants and values provided to it when asked to do this computation).

iurii-ssv · 2024-10-03T12:35:31Z

ssv/docs/README.md

+- `ValidatorRegistrationRunner`
+- `VoluntaryExitRunner`
+
+Each implementation of `Runner` performs its associated duty type, as suggested by their names. A notable difference is that a `CommitteeRunner` object will only execute one `CommitteeDuty` throughout its life. Other `CommitteeRunner` objects will be created to execute upcoming `CommitteeDuties`. For the other runners, they will execute all duties of their associated types during its life.


Perhaps clarify why the Runners differ in this behavior of

some execute all the duties

others are just 1-time performers

iurii-ssv · 2024-10-03T12:52:32Z

ssv/README.md

+Our **system model** is composed of a set of replicas $N$ (a committee of operators), out of which at most $f = \lfloor \frac{N-1}{3} \rfloor$ may be faulty. I.e., we are dealing with the optimal resiliency for the **Byzantine Fault Tolerance (BFT)** model.
+
+This **committee** is assigned to a set of **validators**, from whom the duties must be executed. For each validator, the private key is shared in a way that each operator holds a **private share key** and its associated public key. Moreover, each operator holds a **network private and public key** for authentication in the network communication.


Haven't seen this mentioned in this PR, but seems pretty important for general understanding of the whole system:

how does the process of grouping Operators into Committees look like ? And then Validators are distributed across those Committees somehow ?

or is it that Validators are grouped into Committees and then Operators are assigned to serve specific committee(s) ?

The way this section currently describes it is incomplete and kind of confuses me even more:

This committee is assigned to a set of validators, from whom the duties must be executed.

My current understanding is:

User chooses Operators he desires (to operate his shares), this bunch of Operators essentially constitutes a Commitee

this Committee entity ^ could have already existed in the network (originally created by another User in the past) - in which case we don't need to create a 2nd Committee (which will be processed by it's own set of QBFT instances) and can just bundle 2 or more Validators from independent users together; otherwise (Commitee doesn't exist yet) it's created to serve this 1 User for now

and the whole point of having this Commitee thing is that QBFT iterations are run once per Commitee (as opposed to once per Validator - which seems to be the case for current mainnet setup)

there is also a concept of Topic(s)/Subnet(s) Operators use to actually send messages around (aka pub-sub in libp2p); to reduce the amount of messages sent across SSV network I assume there needs to be some sort of mapping to distribute Commitees into Topics (which this SIP seems to be describing)

is that about right, or is there more to it ? Regardless, would be nice to have it written down somewhere (at least as short high-level summary)

y0sher and others added 14 commits July 22, 2024 16:23

Spec - check att and sync duties exist before submitting (ssvlabs#468)

6921f58

Meta - Update to go1.22 (ssvlabs#474)

96429fb

* Update go1.20 to go1.22 * Update go.sum with mod tidy

Meta - Update dependencies (ssvlabs#483)

8edc50a

* Update dependencies * Fix lint issue * Generate JSON tests to trigger actions * Update fastssz * Generate JSON tests and align ssz error * Revert go-eth2-client version change * Revert fastssz upgrade * Generate SSZ and JSON tests

Meta - Drop unnecessary nolint comments (ssvlabs#477)

642d9d2

* Remove nolint comment and export timeout variables * Drop unnecessary nolint * Add comment * Fix lint issue

Meta - Drop redundant error (ssvlabs#475)

5daa242

Spec - Drop redundant validation for decided messages (ssvlabs#476)

b0f2b4d

* Remove redundant validation * Align error string

Spec - Sort signers in decided message (ssvlabs#484)

c61218a

* Sort signers in decided message * Add test for sorted signers in decided msg * Generate JSON tests * Fix lint issue

Spec - Stop processing after decided (ssvlabs#487)

9abeeec

* Stop processing consensus messages after instance is decided * Align error in qbft tests * Align errors in ssv tests * Generate JSON tests * Fix lint issue

Spec - Drop leftover error check (ssvlabs#469)

496d839

* Remove leftover err check * Align argument variable name to type

Add documentation for the SSV module

96fac27

Add documentation for the QBFT module

8a62e82

MatheusFranco99 self-assigned this Aug 30, 2024

MatheusFranco99 added 8 commits August 30, 2024 15:13

Add README for types

a42db96

Rename documentation file in QBFT

3316705

Rename documentation file in SSV

a636f0c

Put types' module documentation in separate file (as in other modules)

0b9e4f7

Increase the size of images

ec914af

Add more details to the SSVMessage.MessageID

b1ae3e6

Update main README

8fcf27d

Update p2p documentation

7060cac

MatheusFranco99 requested review from GalRogozinski and alan-ssvlabs September 3, 2024 18:02

MatheusFranco99 marked this pull request as ready for review September 3, 2024 18:02

Fix all committees with the same ID

770e204

MatheusFranco99 added the documentation Improvements or additions to documentation label Sep 25, 2024

iurii-ssv reviewed Oct 3, 2024

View reviewed changes

GalRogozinski force-pushed the dev branch from ec462ef to cf804b6 Compare November 12, 2024 13:44

GalRogozinski force-pushed the dev branch 2 times, most recently from b5bd1d3 to bf0ff16 Compare December 19, 2024 22:36

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Documentation #496

Documentation #496

MatheusFranco99 commented Aug 30, 2024 •

edited

Loading

iurii-ssv left a comment

iurii-ssv Oct 3, 2024

iurii-ssv Oct 3, 2024

iurii-ssv Oct 3, 2024

iurii-ssv Oct 3, 2024

iurii-ssv Oct 3, 2024

iurii-ssv Oct 3, 2024

iurii-ssv Oct 3, 2024

iurii-ssv Oct 3, 2024 •

edited

Loading

iurii-ssv Oct 3, 2024

iurii-ssv Oct 3, 2024

iurii-ssv Oct 5, 2024 •

edited

Loading

		- there's a connection to a Beacon node that provides duties' related data (e.g. an attestation vote) and allows the node to submit signed data to the blockchain.
		- there's a connection to the Beacon network that provides information about the blockchain (e.g. the current slot and fork version).

		- `ValidatorRegistration`: to perform the registration of the validator to the blockchain.
		- `VoluntaryExit`: to exit the validator from the blockchain.


		`BeaconNode` is an interface that allows the node to fetch duty data (e.g. an Attestation vote and a block to be proposed) and to submit a signed data (e.g. a signed Attestation vote and a signed block).

		It also provides a method to get the blockchain domain (a sequence of bytes used in signatures that identifies the blockchain) and it holds a `BeaconNetwork` that provides information about the blockchain such as the current slot and fork version.


		## BeaconNetwork

		`BeaconNetwork` is a type that allows accessing information about the blockchain, e.g. such as the current slot, the number of seconds per slot, and the fork version.

		Our system model is composed of a set of replicas $N$ (a committee of operators), out of which at most $f = \lfloor \frac{N-1}{3} \rfloor$ may be faulty. I.e., we are dealing with the optimal resiliency for the Byzantine Fault Tolerance (BFT) model.

		This committee is assigned to a set of validators, from whom the duties must be executed. For each validator, the private key is shared in a way that each operator holds a private share key and its associated public key. Moreover, each operator holds a network private and public key for authentication in the network communication.

Documentation #496

Are you sure you want to change the base?

Documentation #496

Conversation

MatheusFranco99 commented Aug 30, 2024 • edited Loading

Overview

iurii-ssv left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iurii-ssv Oct 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

iurii-ssv Oct 5, 2024 • edited Loading

Choose a reason for hiding this comment

MatheusFranco99 commented Aug 30, 2024 •

edited

Loading

iurii-ssv Oct 3, 2024 •

edited

Loading

iurii-ssv Oct 5, 2024 •

edited

Loading