{"repo":"evolution-gaming/skafka","free":true,"listed":false,"github":"https://github.com/evolution-gaming/skafka","clone":"git clone https://github.com/evolution-gaming/skafka.git","description":"Scala wrapper for kafka consumer and producer","language":"Scala","stars":46,"topics":["scala","kafka","cats","cats-effect"],"license":"MIT","category":"data-pipelines","readme_excerpt":"Skafka Scala wrapper for kafka-clients v4 Motivation Kafka provides an official Java client out of the box, which could be used from Scala code without any additional modifications. The main disadvantage of using an official client directly is that it implies a very specific threading model to the application. I.e. the consumer is not thread safe and also expects a rebalance listener to do the operations in the same thread. This makes wrapping a client with Cats Effect classes a bit more complicated than just calling IO { consumer.poll() } unless this is the only call, which is expected to be used. Skafka does exactly that: a very thin wrapper over official Kafka client to provide a ready-made Cats Effect API and handle some corner cases concerning ConsumerRebalanceListener calls. Comparing to more full-featured libraries such as FS2 Kafka, it might be a little bit more reliable, because there is little code/logic to hide the accidenital bugs in. To summarize: 1. If it suits your goals (i.e. you only ever need to do consumer.poll() without acting on rebalance etc.) then using an official Kafka client directly, optionally, wrapping all the calls with cats.effect.IO , is a totally fine idea. 2. If more complicated integration to Cats Effect is required, i.e. ConsumerRebalanceListener is going to be used then consider using Skafka . 3. If streaming with FS2 is required or any other features the library provides then FS2 Kafka could be a good choice. Note, that it is less trivial","default_branch":null,"files":null,"tree":[],"storefront":"/r/evolution-gaming","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/evolution-gaming/skafka/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}