{"repo":"EmbrasureAI/spark-observability-skills","free":true,"listed":false,"github":"https://github.com/EmbrasureAI/spark-observability-skills","clone":"git clone https://github.com/EmbrasureAI/spark-observability-skills.git","description":"Open-source agent skills for debugging and optimizing Apache Spark workloads","language":"Python","stars":47,"topics":["agent-skills","apache-spark","celeborn","data-observability"],"license":"Apache-2.0","category":"analytics","readme_excerpt":"Spark observability skills Open-source agent skills from Embrasure for diagnosing and optimizing Apache Spark workloads. Each skill is a single SKILL.md with an ordered list of the highest-impact causes to check, plus a read-only Spark History Server REST client under scripts/ that collects the runtime evidence in one bounded snapshot. Install Point the symlinks at whichever skills directory your harness reads ( /.codex/skills , /.claude/skills , ...), creating it first if needed, then restart or reload the harness. Or paste this into your agent: Clone https://github.com/EmbrasureAI/spark-observability-skills and symlink each directory under skills/ into your skills directory, then tell me to reload. Setup The skills need HTTP access to a Spark History Server, or to the live UI of a running application (the driver UI on port 4040 serves the same REST API): - History data exists only for applications that ran with spark.eventLog.enabled=true . - If the server is cluster-internal, open a tunnel first, for example kubectl port-forward svc/spark-history-server 18080:18080 . - Behind an SSO proxy, reuse your browser session with SPARK HISTORY COOKIE or SPARK HISTORY HEADERS JSON ; pass --ca-file for a private CA. Skills - Debug Spark failures: a run failed. Trace driver and executor crashes, out-of-memory kills, fetch failures, task exceptions, and aborted stages back to the earliest supported cause instead of the last retry error. - Debug slow Spark jobs: a run is slower or more ","default_branch":null,"files":null,"tree":[],"storefront":"/r/EmbrasureAI","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/EmbrasureAI/spark-observability-skills/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}