{"repo":"etf1/kafka-mongo-watcher","free":true,"listed":false,"github":"https://github.com/etf1/kafka-mongo-watcher","clone":"git clone https://github.com/etf1/kafka-mongo-watcher.git","description":"A MongoDB collection watcher that pushes oplog events into Kafka","language":"Go","stars":16,"topics":["kafka","mongodb","event-driven","go","golang","database"],"license":null,"category":"data-pipelines","readme_excerpt":"Kafka MongoDB Watcher /badge.svg) This project listens for a MongoDB collection events (insert, update, delete, ...) also called \"oplogs\" for operation logs and distribute them into a Kafka topic of your choice. There is also a replay mode that allows you to initialize all items of a collection into a Kafka topic for the first time. Prerequisites In addition of the binary, you will also need the following the Kafka library: librdkafka (report to the Installation part) Installation Download binary You can download the latest version of the binary built for your architecture here: Architecture i386 [Darwin / Linux / Windows ] Architecture amd64 [Darwin / Linux / Windows ] Using Docker The watcher is also available as a Docker image. You can run it using the following example and pass configuration environment variables: From sources Optionally, you can also download and build it from the sources. You have to retrieve the project sources by using one of the following way: Then, build the binary: Usage In order to run the watcher, type the following command with the desired arguments. You can use flags (as in this example) or environment variables: Available configuration variables In dev environment you can copy .env.dist in .env and edit his content in order to customize easily the env variables. You can set/override configuration variables from .env file and from variables environment and or from cli arguments (If a variables was configured in multiple sources the last will ov","default_branch":null,"files":null,"tree":[],"storefront":"/r/etf1","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/etf1/kafka-mongo-watcher/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}