{"repo":"cerndb/spark-dashboard","free":true,"listed":false,"github":"https://github.com/cerndb/spark-dashboard","clone":"git clone https://github.com/cerndb/spark-dashboard.git","description":"Spark-Dashboard is an open-source monitoring solution for Apache Spark that provides real-time performance dashboards using containers and Kubernetes.","language":"Shell","stars":138,"topics":["grafana","spark","docker","helm","grafana-dashboard","kubernetes","monitoring"],"license":"Apache-2.0","category":"deployment-docker-iac","readme_excerpt":"Spark Dashboard Real-time monitoring and performance troubleshooting for Apache Spark --- Key Features - Real-time monitoring Visualize Spark and system metrics in Grafana, including CPU, memory, active tasks, and I/O, to quickly spot trends and anomalies. - Easy deployment Run locally with a container or deploy on Kubernetes with Helm. - Broad compatibility Supports Apache Spark 3.x and 4.x across Hadoop, Kubernetes, and Spark Standalone environments. --- Contents - Architecture - Deploying Spark Dashboard v2 - Run Spark Dashboard v2 in a container - Run Spark Dashboard v2 on Kubernetes with Helm - Extended Spark Dashboard - Notes on Spark Connect - Examples and getting started - Start small with Spark local mode - Run TPC-DS on a Spark cluster - Legacy implementation (v1) --- Resources - Watch the Spark Dashboard demo and tutorial - Notes on Spark Dashboard - Building an Apache Spark Performance Lab - Blog post on Spark Dashboard - Talk at Data + AI Summit 2021, slides - sparkMeasure, a tool for troubleshooting Apache Spark workloads - TPCDS PySpark, a TPC-DS workload generator for Apache Spark Main author and contact: Luca.Canali@cern.ch --- Architecture Spark Dashboard provides an end-to-end monitoring pipeline for Apache Spark using open-source components. It is designed to deliver real-time visibility into Spark cluster health and performance, from metric generation to visualization. - Apache Spark metrics Apache Spark generates detailed performance metrics through its ","default_branch":null,"files":null,"tree":[],"storefront":"/r/cerndb","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/cerndb/spark-dashboard/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}