{"repo":"linkedin/kafka-monitor","free":true,"listed":false,"github":"https://github.com/linkedin/kafka-monitor","clone":"git clone https://github.com/linkedin/kafka-monitor.git","description":"Xinfra Monitor monitors the availability of Kafka clusters by producing synthetic workloads using end-to-end pipelines to obtain derived vital statistics - E2E latency, service produce/consume availability, offsets commit availability & latency, message loss rate and more.","language":"Java","stars":2063,"topics":["kafka-monitor","kafka-cluster","monitor-topic","partition","broker","partition-count","leader","latency","cluster","metrics"],"license":"Apache-2.0","category":"analytics","readme_excerpt":"Xinfra Monitor Xinfra Monitor (formerly Kafka Monitor) is a framework to implement and execute long-running kafka system tests in a real cluster. It complements Kafka’s existing system tests by capturing potential bugs or regressions that are only likely to occur after prolonged period of time or with low probability. Moreover, it allows you to monitor Kafka cluster using end-to-end pipelines to obtain a number of derived vital stats such as End-to-end latency Service availability Produce and Consume availability Consumer offset commit availability Consumer offset commit latency Kafka message loss rate And many, many more. You can easily deploy Xinfra Monitor to test and monitor your Kafka cluster without requiring any change to your application. Xinfra Monitor can automatically create the monitor topic with the specified config and increase partition count of the monitor topic to ensure partition# = broker#. It can also reassign partition and trigger preferred leader election to ensure that each broker acts as leader of at least one partition of the monitor topic. This allows Xinfra Monitor to detect performance issue on every broker without requiring users to manually manage the partition assignment of the monitor topic. Xinfra Monitor is used in conjunction with different middle-layer services such as li-apache-kafka-clients in order to monitor single clusters, pipeline desination clusters, and other types of clusters as done in Linkedin engineering for real-time cluster h","default_branch":null,"files":null,"tree":[],"storefront":"/r/linkedin","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/linkedin/kafka-monitor/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}