{"repo":"blockchain-etl/ethereum-etl","free":true,"listed":false,"github":"https://github.com/blockchain-etl/ethereum-etl","clone":"git clone https://github.com/blockchain-etl/ethereum-etl.git","description":"Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ","language":"Python","stars":3128,"topics":["ethereum","csv","export","erc20","erc20-tokens","etl","transaction","sql","aws","gcp"],"license":"MIT","category":"blockchain-web3","readme_excerpt":"Ethereum ETL Ethereum ETL lets you convert blockchain data into convenient formats like CSVs and relational databases. Do you just want to query Ethereum data right away? Use the public dataset in BigQuery. Full documentation available here. Quickstart Install Ethereum ETL: Export blocks and transactions (Schema, Reference): Export ERC20 and ERC721 transfers (Schema, Reference): Export traces (Schema, Reference): --- Stream blocks, transactions, logs, token transfers continually to console (Reference): Find other commands here. For the latest version, check out the repo and call Rust Implementation For improved performance, check out ethereum-etl.rs - a Rust rewrite that is 1.4x faster than this Python version while maintaining full CLI compatibility. Useful Links - Schema - Command Reference - Documentation - Public Datasets in BigQuery - Exporting the Blockchain - Querying in Amazon Athena - Querying in Google BigQuery - Querying in Kaggle - Airflow DAGs - Postgres ETL - Ethereum 2.0 ETL Running Tests Running Tox Tests Running in Docker 1. Install Docker: https://docs.docker.com/get-docker/ 2. Build a docker image docker build -t ethereum-etl:latest . docker image ls 3. Run a container out of the image docker run -v $HOME/output:/ethereum-etl/output ethereum-etl:latest export all -s 0 -e 5499999 -b 100000 -p https://mainnet.infura.io docker run -v $HOME/output:/ethereum-etl/output ethereum-etl:latest export all -s 2018-01-01 -e 2018-01-01 -p https://mainnet.infura.io 4. Run","default_branch":null,"files":null,"tree":[],"storefront":"/r/blockchain-etl","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/blockchain-etl/ethereum-etl/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}