{"repo":"Monika-Sonnadmath/kafka-resilience-toolkit","free":true,"listed":false,"github":"https://github.com/Monika-Sonnadmath/kafka-resilience-toolkit","clone":"git clone https://github.com/Monika-Sonnadmath/kafka-resilience-toolkit.git","description":"kafka python microservices retry dead-letter-queue lag-monitoring distributed-systems resilience","language":"Python","stars":12,"topics":["confluent","dead-letter-queue","distributed-systems","kafka","lag-monitor","microservice","python","resilience","retry"],"license":null,"category":"data-pipelines","readme_excerpt":"⚡ kafka-resilience-toolkit Retry, dead letter queue, and lag monitoring utilities for resilient Kafka consumers. Built from real-world experience running Kafka pipelines handling hundreds of millions of telecom events. Handles the failure patterns that production systems actually hit — transient downstream errors, poison pill messages, and consumer groups quietly falling behind. --- The Problem Production Kafka consumers fail in predictable ways: - Transient errors — a downstream service is briefly unavailable; the message should be retried, not dropped - Poison pills — a malformed message will never succeed; it should be routed to a DLQ, not block the partition - Silent lag — a consumer group falls thousands of messages behind; you find out when users complain This toolkit gives you composable utilities to handle all three. --- Features - 🔁 RetryHandler — exponential backoff retry with configurable retryable vs non-retryable exceptions - 💀 DLQProducer — routes failed messages to a dead letter queue topic with full failure metadata - 📊 LagMonitor — per-partition lag tracking with configurable warn/critical thresholds - 🧪 Well tested — retry and DLQ logic fully covered without a real broker --- Installation Requires confluent-kafka: pip install confluent-kafka --- Quick Start Retry with exponential backoff Dead Letter Queue DLQ message payload: Lag Monitoring Decorator usage --- Configuration Reference RetryConfig Parameter Default Description --- --- --- max retries 3 Max","default_branch":null,"files":null,"tree":[],"storefront":"/r/Monika-Sonnadmath","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Monika-Sonnadmath/kafka-resilience-toolkit/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}