{"repo":"RegioHelden/django-kafka","free":true,"listed":false,"github":"https://github.com/RegioHelden/django-kafka","clone":"git clone https://github.com/RegioHelden/django-kafka.git","description":"Confluent's Kafka Python Client combined with Django","language":"Python","stars":19,"topics":["django","kafka"],"license":null,"category":"data-pipelines","readme_excerpt":"django-kafka This library is using confluent-kafka-python which is a wrapper around the librdkafka (Apache Kafka C/C++ client library). It helps to integrate kafka with Django. Quick start Configure: Considering you have locally setup kafka instance with no authentication. All you need is to define the bootstrap servers. Define a Topic: Topics define how to handle incoming messages and how to produce an outgoing message. Topic inherits from the TopicProducer and TopicConsumer classes. If you only need to consume or produce messages, inherit from one of these classes instead to avoid defining unnecessary abstract methods. Define a Consumer: Consumers define which topics they take care of. Usually you want one consumer per project. If 2 consumers are defined, then they will be started in parallel. Consumers are auto-discovered and are expected to be located under the some django app/kafka/consumers.py or some django app/consumers.py . Start the Consumers: You can use django management command to start defined consumers. Or you can use DjangoKafka class API. Check Confluent Python Consumer for API documentation. Produce: Message are produced using a Topic instance. Check Confluent Python Producer for API documentation. Define schema registry: The library is using Confluent's SchemaRegistryClient. In order to use it define a SCHEMA REGISTRY setting. Find available configs in the SchemaRegistryClient docs. Note: take django kafka.topic.AvroTopic as an example if you want to implem","default_branch":null,"files":null,"tree":[],"storefront":"/r/RegioHelden","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/RegioHelden/django-kafka/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}