{"repo":"Broscorp-net/web3-flight-rpc-server","free":true,"listed":false,"github":"https://github.com/Broscorp-net/web3-flight-rpc-server","clone":"git clone https://github.com/Broscorp-net/web3-flight-rpc-server.git","description":"Blockchain data streaming in Apache Arrow format. Web3 data for data engineers, ready for large scale analysis","language":"Java","stars":17,"topics":["apache-arrow","apache-arrow-flight","ethereum-json-rpc","java","python","web3"],"license":"Apache-2.0","category":"blockchain-web3","readme_excerpt":"web3-flight-rpc-server Blockchain data streaming in Apache Arrow format. Web3 data for data engineers, ready for large-scale analysis Project Overview This is an implementation of Apache Arrow Flight RPC server for blockchain data. Apache Arrow format allows zero-copy streaming data processing on top of standard eth-web3 RPC. There are a few key points why to consider flight rpc: - Plenty of clients implementations including JavaScript, Go, Python, R and Rust (see: https://arrow.apache.org/docs/implementations.html) - Rich libraries support. Process eth json-rpc data with your favorite data framework: - pandas - polars - apache datafusion - and many more natively supporting apache arrow format, all with zero copy and without time loss on serialization and deserialization between the formats - ADBC Support Ready-made connectors to most popular databases and formats allow you to write data effortlessly Supported Datasets All datasets requests supports following params: - start block block number to start streaming from (default LATEST) - end block block number to stop streaming (default NULL) - batch size Blocks See schema for stream return type Schema Logs Additional parameters: - topics - list of topics to pass to web3 rpc filter - addresses - list of addresses to pass to web3 rpc filter Schema Requirements - Java 21 - Access to an Ethereum node (for the server). Infura was used for development. Quick Start with Docker The easiest way to try the project is using Docker Compos","default_branch":null,"files":null,"tree":[],"storefront":"/r/Broscorp-net","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Broscorp-net/web3-flight-rpc-server/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}