{"repo":"rayokota/kwack","free":true,"listed":false,"github":"https://github.com/rayokota/kwack","clone":"git clone https://github.com/rayokota/kwack.git","description":"In-Memory Analytics for Kafka using DuckDB","language":"Java","stars":155,"topics":["analytics","duckdb","kafka"],"license":"Apache-2.0","category":"analytics","readme_excerpt":"kwack - In-Memory Analytics for Kafka using DuckDB [![Build Status][github-actions-shield]][github-actions-link] [github-actions-shield]: https://github.com/rayokota/kwack/actions/workflows/build.yml/badge.svg?branch=master [github-actions-link]: https://github.com/rayokota/kwack/actions kwack supports in-memory analytics for Kafka data using DuckDB. Getting Started Note that kwack requires Java 17 or higher. To run kwack, download a release, unpack it. Then change to the kwack-${version} directory and run the following to see the command-line options: kwack shares many command-line options with kcat (formerly kafkacat). In addition, a file containing configuration properties can be used. The available configuration properties are listed here. Simply modify config/kwack.properties to point to an existing Kafka broker and Schema Registry. Then run the following: Starting kwack is as easy as specifying a Kafka broker, topic, and Schema Registry URL: When kwack starts, it will enter interactive mode, where you can enter SQL queries to analyze Kafka data. For non-interactive mode, specify a query on the command line: The output of the above command will be in JSON, and so can be piped to other commands like jq. One can load multiple topics, and then perform a query that joins the resulting tables on a common column: One can convert Kafka data into Parquet format by using the COPY commmand in DuckDB: If not using Confluent Schema Registry, one can pass an external schema: For a gi","default_branch":null,"files":null,"tree":[],"storefront":"/r/rayokota","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/rayokota/kwack/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}