{"repo":"apache/hudi","free":true,"listed":false,"github":"https://github.com/apache/hudi","clone":"git clone https://github.com/apache/hudi.git","description":"Upserts, Deletes And Incremental Processing on Big Data.","language":"Java","stars":6214,"topics":["hudi","apachehudi","datalake","bigdata","apachespark","incremental-processing","stream-processing","data-integration","apacheflink"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"Apache Hudi Apache Hudi is an open data lakehouse platform, built on a high-performance open table format to ingest, index, store, serve, transform and manage your data across multiple cloud data environments. Features Hudi stores all data and metadata on cloud storage in open formats, providing the following features across different aspects. Ingestion Built-in ingestion tools for Apache Spark/Apache Flink users. Supports half-dozen file formats, database change logs and streaming data systems. Connect sink for Apache Kafka, to bring external data sources. Storage Optimized storage format, supporting row & columnar data. Timeline metadata to track history of changes Automatically manages file sizes, layout using statistics Savepoints for data versioning and recovery Schema tracking and evolution. Indexing Scalable indexing subsystem to speed up snapshot queries, maintained automatically by writes. Tracks file listings, column-level and partition-level statistics to help plan queries efficiently. Record-level indexing mechanisms built on row-oriented file formats and bloom filters. Logical partitioning on tables, using expression indexes to decouple from physical partitioning on storage. Writing Atomically commit data with rollback/restore support. Fast upsert/delete support leveraging record-level indexes. Snapshot isolation between writer & queries. Optimistic concurrency control to implement relational data model, with Read-Modify-Write style consistent writes. Non-blockin","default_branch":null,"files":null,"tree":[],"storefront":"/r/apache","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/apache/hudi/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}