Moved Grandine book to this repo

grandinetech · Apr 2, 2024 · 5ae9306 · 5ae9306
1 parent 3cfefca
commit 5ae9306
Show file tree

Hide file tree

Showing 15 changed files with 348 additions and 0 deletions.
diff --git a/.gitignore b/.gitignore
@@ -3,3 +3,5 @@
 cobertura.xml
 
 /target/
+book/book
+book/deploy
diff --git a/book/book.toml b/book/book.toml
@@ -0,0 +1,9 @@
+[book]
+authors = ["Saulius Grigaitis"]
+language = "en"
+multilingual = false
+src = "src"
+title = "Grandine"
+
+[output.html]
+default-theme = "ayu"
diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md
@@ -0,0 +1,14 @@
+# Summary
+
+- [Beacon Node API](./beacon_node_api.md)
+- [Builder API and MEV](./builder_api_and_mev.md)
+- [Checkpoint Sync](./checkpoint_sync.md)
+- [CLI Options](./cli_options.md)
+- [Downloads](./downloads.md)
+- [Metrics](./metrics.md)
+- [Slashing Protection](./slashing_protection.md)
+- [Storage](./storage.md)
+- [Subcommands](./subcommands.md)
+- [Validator Client](./validator_client.md)
+- [Web3Signer](./web3signer.md)
+- [Support](./support.md)
diff --git a/book/src/beacon_node_api.md b/book/src/beacon_node_api.md
@@ -0,0 +1,21 @@
+## Beacon Node API
+
+Grandine supports standard [Beacon Node API](https://ethereum.github.io/beacon-APIs/). This API is extensively tested against other CL validator clients and other Beacon Node API consumers such as [Vouch](https://github.com/attestantio/vouch). API is enabled by default. An example of running Grandine Beacon Node with API enabled by default:
+
+```
+docker run                                                          \
+  -p 9000:9000/tcp                                                  \
+  -p 9000:9000/udp                                                  \
+  -v $HOME/.grandine:/root/.grandine                                \
+  -v $HOME/.grandine/jwtsecret:/root/.grandine/jwtsecret            \
+  sifrai/grandine:unstable grandine                                 \
+  --checkpoint-sync-url CHECKPOINT-SYNC-URL                         \
+  --eth1-rpc-urls ETH1-RPC-URL                                      \
+  --jwt-secret JWT-SECRET-PATH
+```
+
+### Relevant command line options:
+
+* `--http-port` - sets API listen port (default: `5052`);
+* `--http-address` - sets API listen address (default: `127.0.0.1`);
+* `--timeout` - sets API timeout in miliseconds (default: `10000`).
diff --git a/book/src/builder_api_and_mev.md b/book/src/builder_api_and_mev.md
@@ -0,0 +1,10 @@
+## Builder API and MEV
+
+Grandine supports [Builder API](https://github.com/ethereum/builder-specs) for stakers that use MEV. Only a single builder URL can be passed to Grandine. Multiple builders can be used via relay such as [mev-boost](https://github.com/flashbots/mev-boost) or [mev-rs](https://github.com/ralexstokes/mev-rs). Grandine provides a configurable circuit breaker that disables external block building in certain conditions.
+
+### Relevant command line options:
+
+* `--builder-api-url` - external block builder URL (default: does not use any external builder);
+* `--builder-disable-checks` - always specified external block builder without checking for circuit breaker conditions (default: disabled);
+* `--builder-max-skipped-slots` - number of consecutive missing blocks to trigger circuit breaker condition and switch to a local execution engine for payload construction (default: `3`);
+* `--builder-max-skipped-slots-per-epoch` - number of missing blocks in the last rolling epoch to trigger circuit breaker condition and switch to a local execution engine for payload construction (default: `5`).
diff --git a/book/src/checkpoint_sync.md b/book/src/checkpoint_sync.md
@@ -0,0 +1,8 @@
+## Checkpoint Sync
+
+Grandine supports checkpoint sync. Currently, it's the preferred way to sync the chain. By default, back-syncing is disabled, so no historical blocks are fetched and no historical states are reconstructed. This default behavior is sufficient for staking, however, for other use cases (such as historical data access via Beacon Node API) back-syncing must be enabled.
+
+### Relevant command line options
+
+* `--checkpoint-sync-url` - Beacon Node API URL to load a recent finalized checkpoint and sync from it (default: disabled)
+* `--back-sync` - enables back-syncing blocks and reconstructing states (default: disabled)
diff --git a/book/src/cli_options.md b/book/src/cli_options.md
@@ -0,0 +1,160 @@
+## CLI options
+
+The list of command line options:
+
+```
+      --network <NETWORK>
+          Name of the Eth2 network to connect to [default: mainnet] [possible values: mainnet, goerli, custom]
+      --configuration-file <YAML_FILE>
+          Load configuration from YAML_FILE
+      --configuration-directory <DIRECTORY>
+          Load configuration from directory
+      --verify-phase0-preset-file <YAML_FILE>
+          Verify that Phase 0 variables in preset match YAML_FILE
+      --verify-altair-preset-file <YAML_FILE>
+          Verify that Altair variables in preset match YAML_FILE
+      --verify-bellatrix-preset-file <YAML_FILE>
+          Verify that Bellatrix variables in preset match YAML_FILE
+      --verify-capella-preset-file <YAML_FILE>
+          Verify that Capella variables in preset match YAML_FILE
+      --verify-configuration-file <YAML_FILE>
+          Verify that configuration matches YAML_FILE
+      --terminal-total-difficulty-override <DIFFICULTY>
+          Override TERMINAL_TOTAL_DIFFICULTY
+      --terminal-block-hash-override <BLOCK_HASH>
+          Override TERMINAL_BLOCK_HASH
+      --terminal-block-hash-activation-epoch-override <EPOCH>
+          Override TERMINAL_BLOCK_HASH_ACTIVATION_EPOCH
+      --deposit-contract-starting-block <BLOCK_NUMBER>
+          Start tracking deposit contract from BLOCK_NUMBER
+      --genesis-state-file <SSZ_FILE>
+          Load genesis state from SSZ_FILE
+      --max-empty-slots <MAX_EMPTY_SLOTS>
+          [default: 32]
+      --checkpoint-sync-url <CHECKPOINT_SYNC_URL>
+          Beacon node API URL to load recent finalized checkpoint and sync from it [default: None]
+      --force-checkpoint-sync
+          Force checkpoint sync. Requires --checkpoint-sync-url [default: disabled]
+      --eth1-rpc-urls <ETH1_RPC_URLS>...
+          List of Eth1 RPC URLs
+      --data-dir <DATA_DIR>
+          Parent directory for application data files [default: $HOME/.grandine/{network}]
+      --store-directory <STORE_DIRECTORY>
+          Directory to store application data files [default: {data_dir}/beacon]
+      --network-dir <NETWORK_DIR>
+          Directory to store application network files [default: {data_dir}/network]
+      --archival-epoch-interval <ARCHIVAL_EPOCH_INTERVAL>
+          [default: 32]
+      --prune-storage
+          Enable prune mode where only single checkpoint state & block are stored in the DB [default: disabled]
+      --unfinalized-states-in-memory <UNFINALIZED_STATES_IN_MEMORY>
+          Number of unfinalized states to keep in memory. Specifying this number enables unfinalized state pruning. By default all unfinalized states are kept in memory. [default: None]
+      --database-size <DATABASE_SIZE>
+          Max size of the Eth2 database [default: "274.9 GB"]
+      --eth1-database-size <ETH1_DATABASE_SIZE>
+          Max size of the Eth1 database [default: "17.2 GB"]
+      --request-timeout <REQUEST_TIMEOUT>
+          Default global request timeout for various services in milliseconds [default: 30000]
+      --http-address <HTTP_ADDRESS>
+          HTTP API address [default: 127.0.0.1]
+      --http-port <HTTP_PORT>
+          HTTP API port [default: 5052]
+      --state-slot <STATE_SLOT>
+          State slot [default: None]
+      --disable-block-verification-pool
+          Disable block signature verification pool [default: enabled]
+      --subscribe-all-subnets
+          Subscribe to all subnets
+      --suggested-fee-recipient <EXECUTION_ADDRESS>
+          Suggested value for the feeRecipient field of the new payload
+      --jwt-id <JWT_ID>
+          Optional CL unique identifier to send to EL in the JWT token claim [default: None]
+      --jwt-secret <JWT_SECRET>
+          Path to a file containing the hex-encoded 256 bit secret key to be used for verifying/generating JWT tokens
+      --jwt-version <JWT_VERSION>
+          Optional CL node type/version to send to EL in the JWT token claim [default: None]
+      --back-sync
+          Enable syncing historical data [default: disabled]
+      --metrics
+          Collect Prometheus metrics
+      --metrics-address <METRICS_ADDRESS>
+          Metrics address for metrics endpoint [default: 127.0.0.1]
+      --metrics-port <METRICS_PORT>
+          Listen port for metrics endpoint [default: 5054]
+      --remote-metrics-url <REMOTE_METRICS_URL>
+          Optional remote metrics URL that Grandine will periodically send metrics to
+      --track-liveness
+          Enable validator liveness tracking [default: disabled]
+      --max-events <MAX_EVENTS>
+          Max number of events stored in a single channel for HTTP API /events api call [default: 100]
+      --timeout <TIMEOUT>
+          HTTP API timeout in milliseconds [default: 10000]
+      --listen-address <LISTEN_ADDRESS>
+          Listen IPv4 address [default: 0.0.0.0]
+      --listen-address-ipv6 <LISTEN_ADDRESS_IPV6>
+          Listen IPv6 address [default: None]
+      --libp2p-port <LIBP2P_PORT>
+          libp2p IPv4 port [default: 9000]
+      --libp2p-port-ipv6 <LIBP2P_PORT_IPV6>
+          libp2p IPv6 port [default: 9050]
+      --disable-peer-scoring
+          Disable peer scoring
+      --disable-upnp
+          Disable NAT traversal via UPnP [default: enabled]
+      --discovery-port <DISCOVERY_PORT>
+          discv5 IPv4 port [default: 9000]
+      --discovery-port-ipv6 <DISCOVERY_PORT_IPV6>
+          discv5 IPv6 port [default: 9050]
+      --enr-address <ENR_ADDRESS>
+          ENR IPv4 address
+      --enr-address-ipv6 <ENR_ADDRESS_IPV6>
+          ENR IPv6 address
+      --enr-tcp-port <ENR_TCP_PORT>
+          ENR TCP IPv4 port
+      --enr-tcp-port-ipv6 <ENR_TCP_PORT_IPV6>
+          ENR TCP IPv6 port
+      --enr-udp-port <ENR_UDP_PORT>
+          ENR UDP IPv4 port
+      --enr-udp-port-ipv6 <ENR_UDP_PORT_IPV6>
+          ENR UDP IPv6 port
+      --boot-nodes <BOOT_NODES>
+          List of ENR boot node addresses
+      --libp2p-nodes <LIBP2P_NODES>
+          List of Multiaddr node addresses
+      --target-peers <TARGET_PEERS>
+          Target number of network peers [default: 80]
+      --trusted-peers <TRUSTED_PEERS>
+          List of trusted peers
+      --slashing-enabled
+          Enable slasher [default: disabled]
+      --slashing-history-limit <SLASHING_HISTORY_LIMIT>
+          Number of epochs for slasher to search for violations [default: 54000]
+      --keystore-dir <KEYSTORE_DIR>
+          Path to a directory containing EIP-2335 keystore files
+      --keystore-password-dir <KEYSTORE_PASSWORD_DIR>
+          Path to a directory containing passwords for keystore files
+      --keystore-password-file <KEYSTORE_PASSWORD_FILE>
+          Path to a file containing password for keystore files
+      --pack-extra-attestations
+          Pack extra singular attestations to proposed block
+      --builder-api-url <BUILDER_API_URL>
+          External block builder API URL
+      --builder-disable-checks
+          Always use specified external block builder without checking for circuit breaker conditions
+      --builder-max-skipped-slots <BUILDER_MAX_SKIPPED_SLOTS>
+          Max allowed consecutive missing blocks to trigger circuit breaker condition and switch to local execution engine for payload construction [default: 3]
+      --builder-max-skipped-slots-per-epoch <BUILDER_MAX_SKIPPED_SLOTS_PER_EPOCH>
+          Max allowed missing blocks in the last rolling epoch to trigger circuit breaker condition and switch to local execution engine for payload construction [default: 5]
+      --web3signer-api-urls <WEB3SIGNER_API_URLS>...
+          List of Web3Signer API URLs
+      --use-validator-key-cache
+          Use validator key cache for faster startup
+      --graffiti <GRAFFITI>
+          
+      --features <FEATURES>
+          List of optional runtime features to enable
+  -h, --help
+          Print help
+  -V, --version
+          Print version
+```
diff --git a/book/src/downloads.md b/book/src/downloads.md
@@ -0,0 +1,16 @@
+## Downloads 
+
+### Docker
+
+Docker [images](https://hub.docker.com/r/sifrai/grandine/tags) is the best way to get the latest versions of Grandine. We provide multi-platform images supporting `linux/amd64` and `linux/arm64`. Available tags:
+
+* `latest` - latest stable version for `linux/amd64` and `linux/arm64`;
+* `latest-amd64` - latest stable version for `linux/amd64`;
+* `latest-arm64` - latest stable version for `linux/arm64`;
+* `unstable` - latest unstable version `linux/amd64` and `linux/arm64`;
+* `unstable-amd64` - latest unstable version for `linux/amd64`;
+* `unstable-arm64` - latest unstable version for `linux/arm64`.
+
+### Binary
+
+The latest stable binaries are provided on [Github](https://github.com/sifraitech/grandine/tags). Unstable binary builds are not released to Github.
diff --git a/book/src/metrics.md b/book/src/metrics.md
@@ -0,0 +1,16 @@
+## Metrics
+
+### Beacon Node metrics
+
+Grandine provides a set of Beacon Node metrics that are suitable to be consumed by [Prometheus](https://github.com/prometheus/prometheus) and visualized via [Grafana](https://github.com/grafana/grafana) dashboards. 
+
+### Remote Metrics 
+
+Grandine can to push metrics to a remote endpoint every 60 seconds. This option is useful for services such as [beaconcha.in](https://kb.beaconcha.in/beaconcha.in-explorer/mobile-app-less-than-greater-than-beacon-node).
+
+### Relevant command line options:
+
+* `--metrics` - enables metrics (default: disabled);
+* `--metrics-address` - address for metrics endpoint (default: `127.0.0.1`);
+* `--metrics-port` - port for metrics endpoint (default: `5054`);
+* `--remote-metrics-url` - remote metrics URL that Grandine will periodically send metrics to (default: disabled).
diff --git a/book/src/slashing_protection.md b/book/src/slashing_protection.md
@@ -0,0 +1,17 @@
+## Slashing Protection
+
+Grandine supports the [Slashing Protection Interchange Format](https://eips.ethereum.org/EIPS/eip-3076). It's a must to migrate slashing protection data if you are switching validators between clients or servers.
+
+Import Slashing Protection history to Grandine:
+
+```
+./grandine --network goerli interchange export slashing_protection.json
+```
+
+Export Slashing Protection history from Grandine:
+
+```
+./grandine --network goerli interchange export slashing_protection.json
+```
+
+We highly recommend waiting for a few epochs before starting validators after migration.
diff --git a/book/src/storage.md b/book/src/storage.md
@@ -0,0 +1,22 @@
+## Storage
+
+### Memory
+
+By default, Grandine keeps the non-finalized part of the chain in the memory using structural sharing. This approach contributes to the high performance of Grandine because full state copies are avoided. This is a perfect approach for healthy chains (such as Ethereum Mainnet) that don't experience very long non-finalization periods. In such conditions, Grandine uses only ~1GB of memory on the Mainnet. However, during long non-finalization, this approach increases memory usage. In such cases, Grandine allows limiting the number of the latest memory stored states by settings the maximum number of the latest slots that should keep states in the memory.
+
+### Disk
+
+Grandine stores finalized part of the chain in the disk using an embedded key-value database `libmdbx`. Disk storage is passive and mainly used for storing/loading checkpoints, and serving historical data via API. Historical blocks and corresponding intermediate states are stored on the disk. It's possible to set the length of the intermediate states period. A higher value for this interval means lower disk usage and slower API responses for historical data.
+
+Grandine allows starting the Beacon Node from an earlier stored checkpoint by using `--state-slot` option. In this case, Grandine will try to find and load from the disk the closest stored checkpoint before the specified `--state-slot`.
+
+### Prune Mode
+
+Grandine provides `--prune-storage` option for prune mode that only stores a single checkpoint state with the corresponding block. This mode also stores unfinalized blocks on Grandine shutdown. This mode is sufficient for staking.
+
+### Relevant command line options
+
+* `--archival-epoch-interval` - sets the number of epochs between stored states (default: `32`);
+* `--prune-storage` - enables pruning mode that doesn't store historical states and blocks (default: disabled);
+* `--state-slot` - sets the slot at which Grandine Beacon Node should start (default: latest finalized slot);
+* `--unfinalized-states-in-memory` - the number of the latest slots that will store states in the memory (default: all unfinalized states stored in the memory).
diff --git a/book/src/subcommands.md b/book/src/subcommands.md
@@ -0,0 +1,8 @@
+## Subcommands
+
+Grandine has a few helpful subcommands:
+
+*  `export` - Export blocks and state to ssz files within slot range for debugging (example: `grandine export --from 0 --to 5`);
+*  `replay` - Replay blocks within slot range (example: `grandine replay --from 0 --to 5`);
+*  `interchange` - Import/export slashing protection interchange file (example: `grandine interchange import file.json`);
+*  `help` - Print this message or the help of the given subcommand(s).
diff --git a/book/src/support.md b/book/src/support.md
@@ -0,0 +1,9 @@
+## Support
+
+### Discord
+
+It's best to reach us via [Grandine Discord](https://discord.gg/H9XCdUSyZd). Feel free to join if you have any question!
+
+### Commercial support
+
+Please [contacts us](mainto:[email protected]) if you are interested in commercial support for Grandine and related services.
diff --git a/book/src/validator_client.md b/book/src/validator_client.md
@@ -0,0 +1,29 @@
+## Validator client
+
+The Validator Client is a built-in component that is activated if validator keys are passed to Grandine. Let's try running Grandine with a new validator enabled on the Goerli network (assuming you already have keys, secrets etc.):
+
+```
+docker run                                                          \ 
+  -p 9000:9000/tcp                                                  \
+  -p 9000:9000/udp                                                  \
+  -v $HOME/.grandine:/root/.grandine                                \
+  -v $HOME/.grandine/jwtsecret:/root/.grandine/jwtsecret            \
+  -v $HOME/.grandine/validator_keys:/root/.grandine/validator_keys  \
+  -v $HOME/.grandine/secrets:/root/.grandine/secrets                \
+  sifrai/grandine:unstable grandine                                 \
+  --eth1-rpc-urls ETH1-RPC-URL                                      \
+  --network goerli                                                  \
+  --keystore-dir /root/.grandine/validator_keys                     \
+  --keystore-password-file /root/.grandine/secrets
+  --jwtsecret /root/.grandine/jwtsecret
+```
+
+In this example, the same secret is used to secure all the keystores, this secret should be placed in `$HOME/.grandine/secrets` file. Otherwise, for every keystore file in `$HOME/.grandine/validator_keys` there should be a corresponding file in the `$HOME/.grandine/secrets` directory with the secret file named the same as the corresponding keystore file except the extension should be `.txt` instead of `.json`.
+
+For any sensitive keys it's a must to use a remote signer.
+
+### Relevant command line options:
+
+* `--keystore-dir` - a directory containing validator keystore files; 
+* `--keystore-password-file` - a file containing a single secret for all the validator keystore files (this option usable if all the keystores are secured with the same secret);
+* `--keystore-password-dir` - a directory containing secrets for all the validator keystore files (this option usable if all the keystores are secured with not the same secret), for every keystore file in `--keystore-dir` there should be a corresponding file in the `--keystore-password-dir` directory with the secret file named the same as the corresponding keystore file except the extension should be `.txt` instead of `.json`.
diff --git a/book/src/web3signer.md b/book/src/web3signer.md
@@ -0,0 +1,7 @@
+## Web3Signer
+
+Grandine has a built-in validator that supports [Web3Signer](https://github.com/ConsenSys/web3signer). This is a recommended way to use Grandine's built-in validator with sensitive keys. We also recommend using Web3signer's built-in slashing protection even though the built-in validator has slashing protection too. Grandine automatically refreshes the validators list from the given Web3Signer instances.
+
+### Relevant command line options
+
+* `--web3signer-api-urls` - comma separated list of Web3Signer urls. Web3Signer is not used if this option is not set.