diff --git a/data_platform_with_deep_indexed_data_and_staking_reports.md b/data_platform_with_deep_indexed_data_and_staking_reports.md index b0932a48f94..a41d78079c0 100644 --- a/data_platform_with_deep_indexed_data_and_staking_reports.md +++ b/data_platform_with_deep_indexed_data_and_staking_reports.md @@ -209,36 +209,20 @@ We suggest proposal with 1 milestone to: ### Milestone 1 — Forked Superset with indexed data - **Estimated duration:** 1.5 month -- **FTE:** 200 FTE hours [100 USD/hr] +- **FTE:** 3.5 - **Costs:** 20,000 USD | Number | Deliverable | Specification | | --- | --- | --- | | 0a. | License | Apache 2.0 | -| 0b. | Documentation | We will provide a documentation page about how to -- self-host indexer -- how to setup airflow -- how to connect it with DWH -- how to setup DWH -- how to setup UI (BI tool) | +| 0b. | Documentation | We will provide a documentation page about how to launch docker’s files for ETL (indexer + orchestration), DWH connection, launching UI (BI tool). | | 0c. | Testing and Testing Guide | Core functions will be fully covered by comprehensive unit tests to ensure functionality and robustness. In the guide, we will describe how to run these tests. | | 0d. | Docker | We will provide a Dockerfile(s) that can be used to run indexer locally, set up data base (DWH), and run BI. | -| 1. | ETL component development | We already develop Mbelt, so this step is for: -- deploy indexer for public data platform -- develop the airflow DAG to run data pipeline -- start to write data to DWH | -| 2. | Public data in DWH | We has public project in bigQ already, so we need: -- make Polkadot/Kusama/Moonbeam/Moonriver data public -- provide test for pipeline -- publish SLI/SLO dashboard of refreshing the data -- Staking data marts development and publishing | -| 3. | Forking Superset | - Diving into superset’s arhcitecture of creating new functions etc. -- Modifying Sign In page, creating additional actions in controllers -- Main page modifications -- Dashboard tab modification -- Charts tab modification -- SQL lab tab modification -- User’s settings modification | +| 1. | Develop of ETL component | The result of developing ETL component contains the docker with Mbelt (indexer) and Airflow with DAGs to write indexer’s output into DWH (data base). | +| 2. | Publish DWH data | Our goal for publishing data is: providing raw (indexer’s data) data into public bigQ project with the ability to query and use this data for public users. The chains are Polkadot, Kusama, Moonbeam, Moonriver. +We would also provide reports about staking on top of raw data in chains above. +The additional result is SLI/SLO dashboard of ETL process to provide more transparency for users. | +| 3. | Forking Superset | Forking Superset will allow us to modify sign-in page to provide new users to join. We will make some changes on UI side to improve user’s experience as well. | ## **Future plans** @@ -259,3 +243,7 @@ Not applicable**** **How did you hear about the Grants Program?** Web3 Foundation Website and previous grants: [Multiblockchain ETL](https://github.com/w3f/General-Grants-Program/blob/master/grants/speculative/Multiblockchain%20ETL.md) + +**How did you hear about the Grants Program?** + +Web3 Foundation Website and previous grants: [Multiblockchain ETL](https://github.com/w3f/General-Grants-Program/blob/master/grants/speculative/Multiblockchain%20ETL.md)