{"repo":"rayalex/spark-databricks-observability","free":true,"listed":false,"github":"https://github.com/rayalex/spark-databricks-observability","clone":"git clone https://github.com/rayalex/spark-databricks-observability.git","description":"Monitoring Databricks using Prometheus, Grafana and Pyroscope","language":"HCL","stars":28,"topics":["databricks","databricks-automation","grafana","observability","prometheus","pyroscope","spark"],"license":"MIT","category":"analytics","readme_excerpt":"[![Contributors][contributors-shield]][contributors-url] [![Forks][forks-shield]][forks-url] [![Stargazers][stars-shield]][stars-url] [![Issues][issues-shield]][issues-url] [![MIT License][license-shield]][license-url] [![LinkedIn][linkedin-shield]][linkedin-url] Databricks Spark Observability Demo Monitoring and profiling Spark applications in Databricks with Prometheus, Grafana and Pyroscope Table of Contents About The Project Built With Getting Started Prerequisites Installation Usage Roadmap Contributing License Contact Acknowledgments About The Project Dive deeply into performance details and uncover what Spark Execution Plan doesn't typically show. ![Product Name Screen Shot][product-screenshot] ( back to top ) Built With [![Databricks][Databricks.com]][Databricks-url] [![Prometheus][Prometheus.com]][Prometheus-url] [![Grafana][Grafana.com]][Grafana-url] [![Pyroscope][Pyroscope.com]][Pyroscope-url] [![Spark][Spark]][Spark-url] ( back to top ) Getting Started This project demonstrates how to monitor and profile Spark applications in Databricks using Prometheus, Grafana and Pyroscope. This is applicable to any Spark application running on Databricks, including batch, streaming, and interactive workloads (including ephemeral Jobs). Besides Prometheus, Pyroscope and Grafana, this project will create a small single-node Spark Cluster and a set of init scripts to configure it to push metrics to Prometheus Pushgateway and Pyroscope. High-level architecture Prerequisites This d","default_branch":null,"files":null,"tree":[],"storefront":"/r/rayalex","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/rayalex/spark-databricks-observability/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}