{"repo":"segmentio/kafka-go","free":true,"listed":false,"github":"https://github.com/segmentio/kafka-go","clone":"git clone https://github.com/segmentio/kafka-go.git","description":"Kafka library in Go","language":"Go","stars":8605,"topics":["kafka","golang","go","segment","kafka-client","consumer","producer","golang-library","golang-native"],"license":"MIT","category":"data-pipelines","readme_excerpt":"kafka-go Motivations We rely on both Go and Kafka a lot at Segment. Unfortunately, the state of the Go client libraries for Kafka at the time of this writing was not ideal. The available options were: - sarama, which is by far the most popular but is quite difficult to work with. It is poorly documented, the API exposes low level concepts of the Kafka protocol, and it doesn't support recent Go features like contexts. It also passes all values as pointers which causes large numbers of dynamic memory allocations, more frequent garbage collections, and higher memory usage. - confluent-kafka-go is a cgo based wrapper around librdkafka, which means it introduces a dependency to a C library on all Go code that uses the package. It has much better documentation than sarama but still lacks support for Go contexts. - goka is a more recent Kafka client for Go which focuses on a specific usage pattern. It provides abstractions for using Kafka as a message passing bus between services rather than an ordered log of events, but this is not the typical use case of Kafka for us at Segment. The package also depends on sarama for all interactions with Kafka. This is where kafka-go comes into play. It provides both low and high level APIs for interacting with Kafka, mirroring concepts and implementing interfaces of the Go standard library to make it easy to use and integrate with existing software. Note: In order to better align with our newly adopted Code of Conduct, the kafka-go project has r","default_branch":null,"files":null,"tree":[],"storefront":"/r/segmentio","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/segmentio/kafka-go/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}