{"repo":"zendesk/racecar","free":true,"listed":false,"github":"https://github.com/zendesk/racecar","clone":"git clone https://github.com/zendesk/racecar.git","description":"Racecar: a simple framework for Kafka consumers in Ruby","language":"Ruby","stars":509,"topics":["ruby","rails","kafka","kafka-consumer"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"Racecar Racecar is a friendly and easy-to-approach Kafka consumer framework. It allows you to write small applications that process messages stored in Kafka topics while optionally integrating with your Rails models. The framework is based on rdkafka-ruby, which, when used directly, can be a challenge: it's a flexible library with lots of knobs and options. Most users don't need that level of flexibility, though. Racecar provides a simple and intuitive way to build and configure Kafka consumers. NOTE: Racecar requires Kafka 0.10 or higher. Table of content 1. Installation 2. Usage 1. Creating consumers 2. Running consumers 3. Producing messages 4. Configuration 5. Testing consumers 6. Deploying consumers 7. Handling errors 8. Logging 9. Operations 10. Upgrading from v1 to v2 11. Compression 3. Development 4. Contributing 5. Support and Discussion 6. Copyright and license Installation Add this line to your application's Gemfile: And then execute: $ bundle Or install it yourself as: $ gem install racecar Then execute (if you're in a Rails application): $ bundle exec rails generate racecar:install This will add a config file in config/racecar.yml . Usage Racecar is built for simplicity of development and operation. First, a short introduction to the Kafka consumer concept as well as some basic background on Kafka. Kafka stores messages in so-called partitions which are grouped into topics . Within a partition, each message gets a unique offset. In Kafka, consumer groups are sets","default_branch":null,"files":null,"tree":[],"storefront":"/r/zendesk","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/zendesk/racecar/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}