XDC ETL lets you convert blockchain data into convenient formats like CSVs and relational databases.
Do you just want to query XDC data right away? Use the public dataset in BigQuery.
Full documentation available here.
Install XDC ETL:
pip3 install xdcetlExport blocks and transactions (Schema, Reference):
> xdcetl export_blocks_and_transactions --start-block 0 --end-block 100000 \
--blocks-output blocks.csv --transactions-output transactions.csv \
--provider-uri Https://rpc.XDC.orgExport ERC20 and ERC721 transfers (Schema, Reference):
> xdcetl export_token_transfers --start-block 0 --end-block 500000 \
--provider-uri file://$HOME/Library/Ethereum/geth.ipc --output token_transfers.csvExport traces (Schema, Reference):
> xdcetl export_traces --start-block 0 --end-block 500000 \
--provider-uri file://$HOME/Library/Ethereum/parity.ipc --output traces.csvStream blocks, transactions, logs, token_transfers continually to console (Reference):
> pip3 install xdcetl[streaming]
> xdcetl stream --start-block 500000 -e block,transaction,log,token_transfer --log-file log.txt \
--provider-uri Https://rpc.XDC.orgFind other commands here.
For the latest version, check out the repo and call
> pip3 install -e .
> python3 xdcetl.py> pip3 install -e .[dev,streaming]
> export ETHEREUM_ETL_RUN_SLOW_TESTS=True
> export PROVIDER_URL=<your_porvider_uri>
> pytest -vv> pip3 install tox
> tox-
Install Docker: https://docs.docker.com/get-docker/
-
Build a docker image
> docker build -t xdc-etl:latest . > docker image ls -
Run a container out of the image
> docker run -v $HOME/output:/xdc-etl/output xdc-etl:latest export_all -s 0 -e 5499999 -b 100000 -p Https://rpc.XDC.org > docker run -v $HOME/output:/xdc-etl/output xdc-etl:latest export_all -s 2018-01-01 -e 2018-01-01 -p Https://rpc.XDC.org -
Run streaming to console or Pub/Sub
> docker build -t xdc-etl:latest . > echo "Stream to console" > docker run xdc-etl:latest stream --start-block 500000 --log-file log.txt > echo "Stream to Pub/Sub" > docker run -v /path_to_credentials_file/:/xdc-etl/ --env GOOGLE_APPLICATION_CREDENTIALS=/xdc-etl/credentials_file.json xdc-etl:latest stream --start-block 500000 --output projects/<your-project>/topics/crypto_ethereum
If running on Apple M1 chip add the --platform linux/x86_64 option to the build and run commands e.g.:
docker build --platform linux/x86_64 -t xdc-etl:latest .
docker run --platform linux/x86_64 xdc-etl:latest stream --start-block 500000