From 75b474c687050a97f57ac8fd30a58319aaeea22c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jos=C3=A9=20Duarte?= Date: Fri, 22 Nov 2024 12:00:52 +0000 Subject: [PATCH] update book --- src/SUMMARY.md | 2 + src/architecture/pallets/faucet.md | 68 ++++++++ src/architecture/pallets/randomness.md | 2 +- src/getting-started/building/source.md | 2 +- .../local-testnet/getting-funds.md | 22 ++- src/introduction.md | 14 +- src/storage-provider-cli/client/proofs.md | 161 +++++++++++++++++- src/storagext-cli/faucet.md | 22 +++ src/storagext-cli/index.md | 1 + 9 files changed, 277 insertions(+), 17 deletions(-) create mode 100644 src/architecture/pallets/faucet.md create mode 100644 src/storagext-cli/faucet.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index df2e5bd..999cad5 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -11,6 +11,7 @@ - [Storage Provider](./architecture/pallets/storage-provider.md) - [Proofs](./architecture/pallets/proofs.md) - [Randomness](./architecture/pallets/randomness.md) + - [Faucet](./architecture/pallets/faucet.md) - [Getting Started](./getting-started/index.md) - [Building](./getting-started/building/index.md) - [From Source](./getting-started/building/source.md) @@ -34,6 +35,7 @@ - [`proofs`](./storagext-cli/proofs.md) - [`randomness`](./storagext-cli/randomness.md) - [`system`](./storagext-cli/system.md) + - [`faucet`](./storagext-cli/faucet.md) - [Mater CLI](./mater-cli/index.md) - [Zombienet Configuration](./zombienet-config.md) diff --git a/src/architecture/pallets/faucet.md b/src/architecture/pallets/faucet.md new file mode 100644 index 0000000..49e93c6 --- /dev/null +++ b/src/architecture/pallets/faucet.md @@ -0,0 +1,68 @@ +# Faucet Pallet + +## Table of Contents + +- [Faucet Pallet](#faucet-pallet) + - [Table of Contents](#table-of-contents) + - [Overview](#overview) + - [Usage](#usage) + - [Extrinsics](#extrinsics) + - [`drip`](#drip) + - [Events](#events) + - [Errors](#errors) + - [Constants](#constants) + +## Overview + +The Faucet Pallet enables users to drip funds into their wallet to start using the polka-storage chain. + +
+ +The faucet pallet only exists on the testnet. Any of the funds dripped do not have any real-world value. + +
+ +## Usage + +The Faucet Pallet is used to get funds on testnet into an externally generated account. + +> Only 1 drip per 24 hours per account is allowed. When trying to drip more often than once per 24 hours the transaction will be rejected. + +## Extrinsics + +### `drip` + +The `drip` extrinsic is an [unsigned extrinsic (or inherit)](https://docs.substrate.io/learn/transaction-types/#unsigned-transactions) with no gas fees. This means that any account can get funds, even if their current balance is 0. + +| Name | Description | Type | +| --------- | --------------------------------------- | ------------------------------------------------------------------------ | +| `account` | The target account to transfer funds to | [SS58 address](https://docs.substrate.io/learn/accounts-addresses-keys/) | + +#### Example + +```bash +storagext-cli faucet drip 5GpRRVXgPSoKVmUzyinpJPiCjfn98DsuuHgMV2f9s5NCzG19 +``` + +## Events + +The Faucet Pallet only emits a single event: + +- `Dripped` - Emits what account was dripped to and at what block number. + - `who` - [SS58 address](https://docs.substrate.io/learn/accounts-addresses-keys/) of the dripped account. + - `when` - Block at which the drip occurred. + +## Errors + +The Faucet Pallet actions can fail with the following errors: + +- `FaucetUsedRecently` - the provided account had funds dripped within the last 24 hours. + +## Constants + +The Faucet Pallet has the following constants: + +| Name | Description | Value | +| ------------------ | --------------------------------------------------------------------- | ------------------ | +| `FaucetDripAmount` | The amount that is dispensed in [planck](../../glossary.md#planck)'s. | 10_000_000_000_000 | +| `FaucetDripDelay` | How often an account can be topped up. | 1 Day | diff --git a/src/architecture/pallets/randomness.md b/src/architecture/pallets/randomness.md index 9dd5d28..68c4615 100644 --- a/src/architecture/pallets/randomness.md +++ b/src/architecture/pallets/randomness.md @@ -39,7 +39,7 @@ The Randomness Pallet actions can fail with the following errors: ## Constants -The Storage Provider Pallet has the following constants: +The Randomness Pallet has the following constants: | Name | Description | Value | | ----------------- | ----------------------------------------------------------------- | ------- | diff --git a/src/getting-started/building/source.md b/src/getting-started/building/source.md index 59126ed..b53eb56 100644 --- a/src/getting-started/building/source.md +++ b/src/getting-started/building/source.md @@ -23,7 +23,7 @@ cd polka-storage To build the binaries directly on your system you will need the following tools: -* Rust 1.77 — you can install it using [`rustup`](https://rustup.rs/) and its [guide](https://rust-lang.github.io/rustup/installation/other.html) for help. +* Rust 1.81.0 — you can install it using [`rustup`](https://rustup.rs/) and its [guide](https://rust-lang.github.io/rustup/installation/other.html) for help. * Other dependencies — keep reading, we'll get to it after the end of this list! * `just` (optional) — (after installing Rust) you can use `cargo install just` or check the [official list of packages](https://just.systems/man/en/packages.html). diff --git a/src/getting-started/local-testnet/getting-funds.md b/src/getting-started/local-testnet/getting-funds.md index c33a7b5..3c5a98f 100644 --- a/src/getting-started/local-testnet/getting-funds.md +++ b/src/getting-started/local-testnet/getting-funds.md @@ -20,7 +20,27 @@ Or you can watch the following video: allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen> -## Transferring funds +## Dripping funds using the storagext CLI + +Make sure to run the local testnet, you can find how to do so in the [local testnet guide](index.md). + +Once the local testnet is up and running we can drip funds into the account. + +Funds can be dripped using a single command: + +```bash +storagext-cli faucet drip +``` + +Where the `` is the [SS58 address](https://docs.substrate.io/learn/accounts-addresses-keys/) generated in the previous steps. + +> For more information about the faucet pallet and the storagext faucet subcommand check out their respective documenation. +> +> [Faucet pallet docs link](../../architecture/pallets/faucet.md) +> +> [storagext faucet subcommand docs link](../../storagext-cli/faucet.md) + +## Getting funds through the Sudo pallet Make sure to run the local testnet, you can find how to do so in the [local testnet guide](index.md). Once the local testnet is up and running navigate to the polkadot-js web app interface by going to the [default polkadot.js web interface URL](https://polkadot.js.org/apps/?rpc=ws://127.0.0.1:42069). diff --git a/src/introduction.md b/src/introduction.md index b44c0b0..2532ce5 100644 --- a/src/introduction.md +++ b/src/introduction.md @@ -1,12 +1,12 @@ # Introduction -Welcome to the Polka Storage project! +Welcome to the Polka Storage project book. This document is a work in progress and will be constantly updated. -This project aims to deliver a Polkadot-native system parachain for data storage. +This project aims to build a native storage network for Polkadot. -Since the Referendum approval, we've been busy developing the deliverables for Phase 2. +We've now completed Phase 2 and have started work on Phase 3. -**For [**Phase 2**](https://polkadot.polkassembly.io/referenda/1150), we have implemented:** +**During [**Phase 2**](https://polkadot.polkassembly.io/referenda/1150), we have implemented:** - Storage Provider Pallet - [`terminate_sectors`](./architecture/pallets/storage-provider.md#terminate_sectors) @@ -37,7 +37,7 @@ Dedicated CLIs - [`mater-cli`](./mater-cli/index.md) to convert or extract CARv2 files. - [`storagext-cli`](./storagext-cli/index.md) to interact **directly** with the parachain — watch out, this is a low-level tool! -Filecoin actor ports: +Pallets: - [Storage Provider](./architecture/pallets/storage-provider.md) - [Market](./architecture/pallets/market.md) @@ -50,7 +50,7 @@ Filecoin actor ports: alt="Polka Storage Client Upload">

-**As a refresher, for [Phase 1](https://polkadot.polkassembly.io/referenda/494), we implemented the following:** +**During [Phase 1](https://polkadot.polkassembly.io/referenda/494), we implemented the following:** - Keeping track of [Storage Providers](./glossary.md#storage-provider), - [Publishing](./architecture/pallets/market.md#publish_storage_deals) Market Deals on-chain, @@ -69,7 +69,7 @@ We present a demo on how to [store a file](./getting-started/demo-file-store.md) - OpenGov Referendum - Part 1 — - OpenGov Referendum - Part 2 — - Research Report — -- Polkadot Forum News Post — +- Weekly dev updates --- diff --git a/src/storage-provider-cli/client/proofs.md b/src/storage-provider-cli/client/proofs.md index 4edb913..6eeeee5 100644 --- a/src/storage-provider-cli/client/proofs.md +++ b/src/storage-provider-cli/client/proofs.md @@ -5,11 +5,158 @@ The following subcommands are contained under `proofs`. > These are advanced commands and only useful for demo purposes. > This functionality is covered in the server by the [pipeline](../../architecture/polka-storage-provider-server.md#sealing-pipeline). -| Name | Description | -| ---------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | -| `calculate-piece-commitment` | Calculate a piece commitment for the provided data stored at the a given path | -| `porep-params` | Generates PoRep verifying key and proving parameters for zk-SNARK workflows (prove commit) | -| `post-params` | Generates PoSt verifying key and proving parameters for zk-SNARK workflows (submit windowed PoSt) | -| `porep` | Generates PoRep for a piece file. Takes a piece file (in a CARv2 archive, unpadded), puts it into a sector (temp file), seals and proves it | -| `post` | Creates a PoSt for a single sector | +| Name | Description | +| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------- | +| `commp` | Calculate a piece commitment (CommP) for the provided data stored at the a given path. | +| `porep-params` | Generates PoRep verifying key and proving parameters for zk-SNARK workflows (prove commit) | +| `post-params` | Generates PoSt verifying key and proving parameters for zk-SNARK workflows (submit windowed PoSt) | +| `porep` | Generates PoRep for a piece file. Takes a piece file (in a CARv2 archive, unpadded), puts it into a sector (temp file), seals and proves it | +| `post` | Creates a PoSt for a single sector | +## `commp` + +Produces a CommP out of the CARv2 archive and calculates [piece_size](https://spec.filecoin.io/#section-systems.filecoin_files.piece.data-representation) that will be accepted by the network in a [deal](./index.md#propose-deal). +If the file at the path is not a CARv2 archive, it fails. +To create a CARv2 archive, you can use [`mater-cli convert`](../../mater-cli/index.md#convert) command. + +### Example + +```bash +$ mater-cli convert polkadot.svg +Converted polkadot.svg and saved the CARv2 file at polkadot.car with a CID of bafkreihoxd7eg2domoh2fxqae35t7ihbonyzcdzh5baevxzrzkaakevuvy +$ polka-storage-provider-client proofs commp polkadot.car +{ + "cid": "baga6ea4seaqabpfwrqjcwrb4pxmo2d3dyrgj24kt4vqqqcbjoph4flpj2e5lyoq", + "size": 2048 +} +``` + +## `porep-params` + +Generates a [PoRep](../../glossary.md#proofs) parameters which consist of Proving Params (`*.porep.params` file) and Verifying Key (`*.porep.vk`, `*.porep.vk.scale`). +Proving Parameters are used by the Storage Provider to generate a PoRep and the corresponding Verifying Key is used to [verify proofs on chain](../../architecture/pallets/proofs.md#set_porep_verifying_key) by pallet-proofs and [pallet-storage-provider](../../architecture/pallets/storage-provider.md#prove_commit_sectors). + +### Example + +```bash +$ polka-storage-provider-client proofs porep-params +Generating params for 2KiB sectors... It can take a couple of minutes ⌛ +Generated parameters: +[...]/polka-storage/2KiB.porep.params +[...]/polka-storage/2KiB.porep.vk +[...]/polka-storage/2KiB.porep.vk.scale +``` + +## `post-params` + +Generates a [PoSt](../../glossary.md#proofs) parameters which consist of Proving Params (`*.post.params` file) and Verifying Key (`*.post.vk`, `*.post.vk.scale`). +Proving Parameters are used by the Storage Provider to generate a PoSt and the corresponding Verifying Key is used to [verify proofs on chain](../../architecture/pallets/proofs.md#set_post_verifying_key) by pallet-proofs and [pallet-storage-provider](../../architecture/pallets/storage-provider.md#submit_windowed_post). + +### Example + +```bash +$ polka-storage-provider-client proofs post-params +Generating PoSt params for 2KiB sectors... It can take a few secs ⌛ +Generated parameters: +[...]/polka-storage/2KiB.post.params +[...]/polka-storage/2KiB.post.vk +[...]/polka-storage/2KiB.post.vk.scale +``` + +## `porep` + +Generates a 2KiB sector-size PoRep proof for an input file and its piece commitment. +Creates the sector containing only 1 piece, [seals it](https://spec.filecoin.io/#section-algorithms.pos.porep) by creating a replica and then creates a proof for it. + +> This is a *demo command*, showcasing the ability to generate a PoRep +> given the proving parameters so it can later be used to verify proof on-chain. +> It uses hardcoded values, which normally would be sourced from the chain i.e: +> +> ```rust +> let sector_id = 77; +> let ticket = [12u8; 32]; +> let seed = [13u8; 32]; +> ``` + +```bash +polka-storage-provider-client proofs porep \ + --sr25519-key|--ecdsa-key|--ed25519-key \ + --cache-directory \ + --proofs-parameters-path \ + +``` + +### Example + +```bash +$ mater-cli convert polkadot.svg +Converted polkadot.svg and saved the CARv2 file at polkadot.car with a CID of bafkreihoxd7eg2domoh2fxqae35t7ihbonyzcdzh5baevxzrzkaakevuvy +$ polka-storage-provider-client proofs commp polkadot.car +{ + "cid": "baga6ea4seaqabpfwrqjcwrb4pxmo2d3dyrgj24kt4vqqqcbjoph4flpj2e5lyoq", + "size": 2048 +} +$ polka-storage-provider-client proofs porep-params +Generating params for 2KiB sectors... It can take a couple of minutes ⌛ +Generated parameters: +[...]/polka-storage/2KiB.porep.params +[...]/polka-storage/2KiB.porep.vk +[...]/polka-storage/2KiB.porep.vk.scale +$ mkdir -p /tmp/psp-cache +$ polka-storage-provider-client proofs porep --sr25519-key "//Alice" --cache-directory /tmp/psp-cache --proof-parameters-path 2KiB.porep.params polkadot.car baga6ea4seaqabpfwrqjcwrb4pxmo2d3dyrgj24kt4vqqqcbjoph4flpj2e5lyoq +Creating sector... +Precommitting... +2024-11-18T10:48:29.858550Z INFO filecoin_proofs::api::seal: seal_pre_commit_phase1:start: SectorId(77) +2024-11-18T10:48:29.863782Z INFO storage_proofs_porep::stacked::vanilla::proof: replicate_phase1 +2024-11-18T10:48:29.864120Z INFO storage_proofs_porep::stacked::vanilla::graph: using parent_cache[64 / 64] +[...] +CommD: Cid(baga6ea4seaqabpfwrqjcwrb4pxmo2d3dyrgj24kt4vqqqcbjoph4flpj2e5lyoq) +CommR: Cid(bagboea4b5abcb7rgo7kuqigb2wjybggbvlmmatmki52by3wov5uwjrjwefxwzxi5) +Wrote proof to [...]/polka-storage/77.sector.proof.porep.scale +``` + +## `post` + +Generates a 2KiB sector-sized PoSt proof. +To be able to create a PoSt proof, first you need to generate a PoRep proof and a replica via `porep` command. + +> This is a *demo command*, showcasing the ability to generate a PoSt, +> given the proving parameters so it can later be used to verify proof on-chain. +> It uses hardcoded values, which normally would be sourced from the chain i.e: +> +> ```rust +> let sector_id = 77; +> let randomness = [1u8; 32]; +> ``` + +```bash +polka-storage-provider-client proofs post + --sr25519-key|--ecdsa-key|--ed25519-key \ + --proof-parameters-path \ + --cache-directory \ + + +``` + +### Example + +```bash +$ polka-storage-provider-client proofs post-params +Generating PoSt params for 2KiB sectors... It can take a few secs ⌛ +Generated parameters: +[...]/polka-storage/2KiB.post.params +[...]/polka-storage/2KiB.post.vk +[...]/polka-storage/2KiB.post.vk.scale +$ polka-storage-provider-client proofs post --sr25519-key "//Alice" --cache-directory /tmp/psp-cache --proof-parameters-path 2KiB.post.params 77.sector.sealed bagboea4b5abcb7rgo7kuqigb2wjybggbvlmmatmki52by3wov5uwjrjwefxwzxi5 +Loading parameters... +2024-11-18T11:20:26.718119Z INFO storage_proofs_core::compound_proof: vanilla_proofs:start +2024-11-18T11:20:26.750347Z INFO storage_proofs_core::compound_proof: vanilla_proofs:finish +2024-11-18T11:20:26.750712Z INFO storage_proofs_core::compound_proof: snark_proof:start +2024-11-18T11:20:26.750797Z INFO bellperson::groth16::prover::native: Bellperson 0.26.0 is being used! +2024-11-18T11:20:26.771368Z INFO bellperson::groth16::prover::native: synthesis time: 20.550334ms +2024-11-18T11:20:26.771385Z INFO bellperson::groth16::prover::native: starting proof timer +2024-11-18T11:20:26.772676Z INFO bellperson::gpu::locks: GPU is available for FFT! +2024-11-18T11:20:26.772687Z INFO bellperson::gpu::locks: BELLPERSON_GPUS_PER_LOCK fallback to single lock mode +Proving... +Wrote proof to [...]/polka-storage/77.sector.proof.post.scale +``` \ No newline at end of file diff --git a/src/storagext-cli/faucet.md b/src/storagext-cli/faucet.md new file mode 100644 index 0000000..7ede0e2 --- /dev/null +++ b/src/storagext-cli/faucet.md @@ -0,0 +1,22 @@ +# The `faucet` command + +Under the `faucet` subcommand [faucet](../architecture/pallets/faucet.md) related extrinsics are available. +This chapter covers the provided commands and how to use them. + +## `drip` + +The `drip` command tops up the provided account. + +### Parameters + +| Name | Description | Type | +| --------- | --------------------- | ------- | +| `ACCOUNT` | Account ID to drip to | Account | + +### Example + +Topping up `5GpRRVXgPSoKVmUzyinpJPiCjfn98DsuuHgMV2f9s5NCzG19` + +```bash +storagext-cli faucet drip 5GpRRVXgPSoKVmUzyinpJPiCjfn98DsuuHgMV2f9s5NCzG19 +``` diff --git a/src/storagext-cli/index.md b/src/storagext-cli/index.md index dcf415d..aebcfa4 100644 --- a/src/storagext-cli/index.md +++ b/src/storagext-cli/index.md @@ -74,3 +74,4 @@ and will report it's result — whether the call was successful or not. - [Subcommand `proofs`](proofs.md) - [Subcommand `randomness`](randomness.md) - [Subcommand `system`](system.md) +- [Subcommand `faucet`](faucet.md)