{"repo":"fluent/fluent-plugin-kafka","free":true,"listed":false,"github":"https://github.com/fluent/fluent-plugin-kafka","clone":"git clone https://github.com/fluent/fluent-plugin-kafka.git","description":"Kafka input and output plugin for Fluentd","language":"Ruby","stars":307,"topics":["fluentd","kafka","fluentd-plugin"],"license":null,"category":"data-pipelines","readme_excerpt":"fluent-plugin-kafka, a plugin for Fluentd A fluentd plugin to both consume and produce data for Apache Kafka. Installation Add this line to your application's Gemfile: gem 'fluent-plugin-kafka' And then execute: $ bundle Or install it yourself as: $ gem install fluent-plugin-kafka --no-document If you want to use zookeeper related parameters, you also need to install zookeeper gem. zookeeper gem includes native extension, so development tools are needed, e.g. ruby-devel, gcc, make and etc. Requirements - Ruby 2.1 or later - Input plugins work with kafka v0.9 or later - Output plugins work with kafka v0.8 or later Usage Common parameters SSL authentication - ssl ca cert - ssl client cert - ssl client cert key - ssl client cert key password - ssl ca certs from system Set path to SSL related files. See Encryption and Authentication using SSL for more detail. SASL authentication with GSSAPI - principal - keytab Set principal and path to keytab for SASL/GSSAPI authentication. See Authentication using SASL for more details. with Plain/SCRAM - username - password - scram mechanism - sasl over ssl Set username, password, scram mechanism and sasl over ssl for SASL/Plain or Scram authentication. See Authentication using SASL for more details. Input plugin (@type 'kafka') Consume events by single consumer. @type kafka brokers : , : ,.. topics format :default = json message key add prefix add suffix # Optionally, you can manage topic offset by using zookeeper offset zookeeper : , : ,..) ","default_branch":null,"files":null,"tree":[],"storefront":"/r/fluent","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/fluent/fluent-plugin-kafka/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}