{"repo":"exacaster/lighter","free":true,"listed":false,"github":"https://github.com/exacaster/lighter","clone":"git clone https://github.com/exacaster/lighter.git","description":"REST API for Apache Spark on K8S or YARN","language":"Java","stars":112,"topics":["apache-spark","jupyter","k8s","yarn","livy","spark","sparkmagic"],"license":"MIT","category":"deployment-docker-iac","readme_excerpt":"Lighter Lighter is an opensource application for interacting with Apache Spark on Kubernetes or Apache Hadoop YARN. It is heavily inspired by Apache Livy and has some overlapping features. Lighter support: - Interactive Python Sessions through Sparkmagic kernel - Batch application submissions through the REST API You can read a description on how Lighter works here. Using Lighter - Quickstart with Minikube - Installation on Kubernetes - Installation on Docker - Configuration Properties - Securing a deployment - Configuring Sparkmagic - Using REST API [!IMPORTANT] Lighter has no built-in authentication: anyone who can reach its REST API can run code on your Spark cluster. Read Securing a deployment before exposing it. Developing Lighter Building To build Docker image, containing application, you need to run Spark versions Lighter will always try to support the latest patch version for all officially recommended Apache Spark releases i.e.: 3.5.8 and 4.1.1. Contributing See Contribution guide License Lighter is MIT licensed.","default_branch":null,"files":null,"tree":[],"storefront":"/r/exacaster","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/exacaster/lighter/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}