{"repo":"valeriouberti/rigatoni","free":true,"listed":false,"github":"https://github.com/valeriouberti/rigatoni","clone":"git clone https://github.com/valeriouberti/rigatoni.git","description":"A high-performance, type-safe CDC/Data replication framework for Rust, focused on real-time data pipelines.","language":"Rust","stars":11,"topics":["cdc","data-engineering","etl","pipeline","streaming","replication"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"Rigatoni A high-performance, type-safe CDC/Data Replication framework for Rust, focused on real-time data pipelines. 🎯 Overview Rigatoni is a modern CDC (Change Data Capture) and data replication framework built for speed, reliability, and developer experience. Built with Rust's type system and async/await, it provides production-ready data pipelines for real-time streaming workloads from databases to data lakes and other destinations. Currently supporting: - MongoDB Change Streams - Real-time CDC (Change Data Capture) from MongoDB - S3 Destination - Export to AWS S3 with multiple formats (JSON, CSV, Parquet, Avro) - Redis State Store - Distributed state management for multi-instance deployments - Pipeline Orchestration - Multi-worker architecture with retry logic and state management - Metrics & Observability - Prometheus metrics with Grafana dashboards - Async-first design - Powered by Tokio for high throughput - Type-safe transformations - Compile-time guarantees with Rust's type system - Modular architecture - Extensible with feature flags ⚡ Performance Rigatoni delivers exceptional performance for high-throughput CDC workloads: - 780ns per event - Core processing with linear scaling - 1.2μs per event - JSON serialization - 7.65ms for 1000 events - S3 writes with ZSTD compression - 10K-100K events/sec - Production-ready throughput - Sub-millisecond - State store operations See our detailed benchmarks for comprehensive performance analysis. ✨ Features - 🚀 High Performanc","default_branch":null,"files":null,"tree":[],"storefront":"/r/valeriouberti","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/valeriouberti/rigatoni/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}