{"repo":"datazip-inc/olake","free":true,"listed":false,"github":"https://github.com/datazip-inc/olake","clone":"git clone https://github.com/datazip-inc/olake.git","description":"OLake - Fastest Databases, Kafka & S3 Replication to Apache Iceberg with Table optimization (Called OLake Fusion). ⚡ Efficient, quick and scalable data ingestion for real-time analytics. Supported sources : Postgres, MongoDB, MySQL, Oracle, MSSql, DB2, Kafka, S3.","language":"Go","stars":1422,"topics":["cdc","change-data-capture","data-pipeline","database","elt","lakehouse","replication","apache-iceberg","parquet","s3"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"OLake Go OLake Go is a high-performance, open-source data ingestion engine for replicating databases, S3, and Kafka into Apache Iceberg (or plain Parquet). Built for scalable, real-time pipelines, OLake Go provides a simple web UI and CLI - used to ingest into vendor-lock-in free Iceberg tables supporting all the query-engines/warehouses. Read the docs and benchmarks at olake.io/docs . Join our active community on Slack . [!NOTE] 🎉 OLake Fusion is now live! — Automate your Apache Iceberg Table Maintenance. Check it out here → github.com/datazip-inc/olake-fusion 🎉 OLake Go — Super-fast Sync to Apache Iceberg OLake Go supports replication from transactional databases such as PostgreSQL, MySQL, MongoDB, Oracle, DB2, and MSSQL , event-streaming systems like Apache Kafka and Object-store like S3 , into open data lakehouse formats such as Apache Iceberg or Plain Parquet — delivering blazing-fast performance with minimal infrastructure cost. --- 🚀 Why OLake Go? - 🧠 Smart sync : Full + CDC replication with automatic schema discovery & schema evolution - ⚡ High throughput : 580K RPS (Postgres) & 338K RPS (MySQL) - ➡️ Exactly once delivery & Arrow writes : Accuracy with speed. - 💾 Iceberg-native : Supports Glue, Hive, JDBC, REST catalogs - 🖥️ Self-serve UI : Deploy via Docker Compose and sync in minutes - 💸 Infra-light : No Spark, no Flink, no Kafka, no Debezium --- 📊 Benchmarks Full Load Source → Destination Full Load Relative Performance (Full Load) Full Report --------------","default_branch":null,"files":null,"tree":[],"storefront":"/r/datazip-inc","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/datazip-inc/olake/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}