{"repo":"redpanda-data/connect","free":true,"listed":false,"github":"https://github.com/redpanda-data/connect","clone":"git clone https://github.com/redpanda-data/connect.git","description":"Fancy stream processing made operationally mundane","language":"Go","stars":8732,"topics":["message-queue","stream-processing","streaming-data","message-bus","logs","stream-processor","cqrs","event-sourcing","go","golang"],"license":null,"category":"data-pipelines","readme_excerpt":"Redpanda Connect [![Build Status][actions-badge]][actions-url] [![Apache V2 API][godoc-badge]][godoc-url-apache] [![Enterprise API][godoc-badge]][godoc-url-enterprise] Redpanda Connect is a stream processor that moves data between a wide range of [sources][inputs] and [sinks][outputs], with support for [hydration, enrichment, transformation, and filtering][processors] along the way. That includes a rich set of change-data-capture (CDC) connectors — for Postgres, MySQL, MongoDB, Oracle, MSSQL, and more — so database changes can flow through your pipelines as first-class events. It uses [Bloblang][bloblang-about] for mapping, runs as a single static binary or container image, and is easy to operate and monitor. Highlights - Declarative pipelines — a stream topology fits in a single YAML file. - At-least-once delivery by default — in-process transactions, no disk state required. - A large connector catalog — cloud services, message brokers, databases, HTTP, and more. - First-class CDC — change-data-capture connectors for Postgres, MySQL, MongoDB, Oracle, and MSSQL. - Bloblang — a mapping language designed for stream data. - Cloud-friendly — stateless and horizontally scalable, with metrics and tracing built in. Example Stream Postgres changes into Apache Iceberg tables on S3, one Iceberg table per source table: Quickstart Install Linux: macOS (Homebrew): Docker: See the [getting started guide][getting-started] for more options. Run With Docker: Connectors The catalog includes AW","default_branch":null,"files":null,"tree":[],"storefront":"/r/redpanda-data","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/redpanda-data/connect/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}