{"repo":"Trendyol/kafka-cronsumer","free":true,"listed":false,"github":"https://github.com/Trendyol/kafka-cronsumer","clone":"git clone https://github.com/Trendyol/kafka-cronsumer.git","description":"Cron based Kafka exception consumer with the power of auto retry & concurrency","language":"Go","stars":89,"topics":["cron","exception-handling","go","golang","kafka"],"license":"MIT","category":"data-pipelines","readme_excerpt":"Kafka C[r]onsumer Description Kafka Cronsumer is mainly used for retry/exception strategy management. It works based on cron expression and consumes messages in a timely manner with the power of auto pause and concurrency. For details check our blog post If you need a whole consumer lifecycle with exception management, check Kafka Konsumer How Kafka Cronsumer Works When to use it? - Iteration-based back-off strategies are applicable - Messages could be processed in an eventually consistent state - Max retry exceeded messages could be ignored and send to dead letter topic - To increase consumer resiliency - To increase consumer performance with concurrency When to avoid? - Messages should be processed in order - Messages should be certainly processed (we discard messages if max retry is exceeded) - Messages should be committed (we use auto-commit interval for increasing performance) - Messages with TTL (Time to Live) Guide Installation Examples You can find a number of ready-to-run examples at this directory. After running docker-compose up command, you can run any application you want. Don't forget its cron based :) Single Consumer Single Consumer With Dead Letter Multiple Consumers Single Consumer With Metric collector Configurations config description default example ---------------------------------- -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------","default_branch":null,"files":null,"tree":[],"storefront":"/r/Trendyol","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Trendyol/kafka-cronsumer/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}