{"repo":"linkedin/cruise-control","free":true,"listed":false,"github":"https://github.com/linkedin/cruise-control","clone":"git clone https://github.com/linkedin/cruise-control.git","description":"Cruise-control is the first of its kind to fully automate the dynamic workload rebalance and self-healing of a Kafka cluster. It provides great value to Kafka users by simplifying the operation of Kafka clusters.","language":"Java","stars":3039,"topics":["kafka","cluster-management","self-healing"],"license":"BSD-2-Clause","category":"data-pipelines","readme_excerpt":"Cruise Control for Apache Kafka =================== Introduction ### Cruise Control is a product that helps run Apache Kafka clusters at large scale. Due to the popularity of Apache Kafka, many companies have increasingly large Kafka clusters with hundreds of brokers. At LinkedIn, we have 10K+ Kafka brokers, which means broker deaths are an almost daily occurrence and balancing the workload of Kafka also becomes a big overhead. Kafka Cruise Control is designed to address this operational scalability issue. Features ### Kafka Cruise Control provides the following features out of the box: Resource utilization tracking for brokers, topics, and partitions. Query the current Kafka cluster state to see the online and offline partitions, in-sync and out-of-sync replicas, replicas under min.insync.replicas , online and offline logDirs, and distribution of replicas in the cluster. Multi-goal rebalance proposal generation for: Rack-awareness Resource capacity violation checks (CPU, DISK, Network I/O) Per-broker replica count violation check Resource utilization balance (CPU, DISK, Network I/O) Leader traffic distribution Replica distribution for topics Global replica distribution Global leader replica distribution Custom goals that you wrote and plugged in Anomaly detection, alerting, and self-healing for the Kafka cluster, including: Goal violation Broker failure detection Metric anomaly detection Disk failure detection (not available in kafka 0 11 and 1 0 branch) Slow broker detectio","default_branch":null,"files":null,"tree":[],"storefront":"/r/linkedin","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/linkedin/cruise-control/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}