{"repo":"ArroyoSystems/arroyo","free":true,"listed":false,"github":"https://github.com/ArroyoSystems/arroyo","clone":"git clone https://github.com/ArroyoSystems/arroyo.git","description":"Distributed stream processing engine in Rust","language":"Rust","stars":4998,"topics":["data","dev-tools","infrastructure","kafka","rust","sql","stream-processing","data-stream-processing","stream-processing-engine"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"Getting started Docs Discord Website Arroyo is a distributed stream processing engine written in Rust, designed to efficiently perform stateful computations on streams of data. Unlike traditional batch processing, streaming engines can operate on both bounded and unbounded sources, emitting results as soon as they are available. In short: Arroyo lets you ask complex questions of high-volume real-time data with subsecond results. Features 🦀 SQL streaming pipelines 🚀 Scales up to millions of events per second 🪟 Stateful operations including windows and joins 🔥State checkpointing for fault-tolerance and recovery of pipelines 🕒 Time-oriented stream processing via the Dataflow model 🔌 A wide variety of connectors, including Kafka and Iceberg Use cases Some example use cases include: Detecting fraud and security incidents Real-time product and business analytics Real-time ingestion into your data warehouse or data lake Real-time ML feature generation Why Arroyo There are already a number of existing streaming engines out there, including Apache Flink, Spark Streaming, and Kafka Streams. Why create a new one? Serverless operations : Arroyo pipelines are designed to run in modern cloud environments, supporting seamless scaling, recovery, and rescheduling High performance SQL : SQL is a first-class concern, with consistently excellent performance Designed for non-experts : Arroyo cleanly separates the pipeline APIs from its internal implementation. You don't need to be a streami","default_branch":null,"files":null,"tree":[],"storefront":"/r/ArroyoSystems","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/ArroyoSystems/arroyo/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}