{"repo":"apache/flink-cdc","free":true,"listed":false,"github":"https://github.com/apache/flink-cdc","clone":"git clone https://github.com/apache/flink-cdc.git","description":"Flink CDC is a streaming data integration tool","language":"Java","stars":6460,"topics":["change-data-capture","cdc","batch","data-integration","data-pipeline","distributed","elt","etl","flink","kafka"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"Flink CDC is a distributed data integration tool for real-time data and batch data, built on top of Apache Flink. It prioritizes efficient end-to-end data integration and offers enhanced functionalities such as full database synchronization, sharding table synchronization, schema evolution and data transformation. API Layers Flink CDC provides three API layers for different usage scenarios: 1. YAML API (Pipeline API) The YAML API provides a declarative, zero-code approach to define data pipelines. Users describe the source, sink, routing, transformation, and schema evolution rules in a YAML file and submit it via the flink-cdc.sh CLI. Please refer to the Quickstart Guide for detailed setup instructions. Pipeline connectors: Doris Elasticsearch Fluss Hudi Iceberg Kafka MaxCompute MySQL OceanBase Oracle Paimon PostgreSQL StarRocks See the connector overview for a full list and configurations. 2. SQL API (Table/SQL API) The SQL API integrates with Flink SQL, allowing users to define CDC sources using SQL DDL statements. Deploy the SQL connector JAR to FLINK HOME/lib/ and use it directly in Flink SQL Client: Available SQL connectors (dependencies bundled): MySQL PostgreSQL Oracle SQL Server MongoDB OceanBase TiDB Db2 Vitess See the source connector overview for a full list and configurations. 3. DataStream API The DataStream API provides programmatic access for building custom Flink streaming applications. Add the corresponding connector as a Maven dependency: Available source co","default_branch":null,"files":null,"tree":[],"storefront":"/r/apache","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/apache/flink-cdc/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}