{"repo":"kubeflow/spark-operator","free":true,"listed":false,"github":"https://github.com/kubeflow/spark-operator","clone":"git clone https://github.com/kubeflow/spark-operator.git","description":"Kubernetes operator for managing the lifecycle of Apache Spark applications on Kubernetes.","language":"Python","stars":3149,"topics":["kubernetes","kubernetes-operator","apache-spark","kubernetes-crd","kubernetes-controller","spark","google-cloud-dataproc"],"license":"Apache-2.0","category":"deployment-docker-iac","readme_excerpt":"Kubeflow Spark Operator What is Spark Operator? The Kubernetes Operator for Apache Spark aims to make specifying and running Spark applications as easy and idiomatic as running other workloads on Kubernetes. It uses Kubernetes custom resources for specifying, running, and surfacing status of Spark applications. Quick Start For a more detailed guide, please refer to the Getting Started guide. Installation with Helm Installation with Kustomize An alternative install method is using kustomize manifests. For more information go to the kustomize installation docs. Overview For a complete reference of the custom resource definitions, please refer to the API Definition. For details on its design, please refer to the Architecture. It requires Spark 2.3 and above that supports Kubernetes as a native scheduler backend. The Kubernetes Operator for Apache Spark currently supports the following list of features: - Supports Spark 2.3 and up. - Enables declarative application specification and management of applications through custom resources. - Automatically runs spark-submit on behalf of users for each SparkApplication eligible for submission. - Provides native cron support for running scheduled applications. - Supports customization of Spark pods beyond what Spark natively is able to do through the mutating admission webhook, e.g., mounting ConfigMaps and volumes, and setting pod affinity/anti-affinity. - Supports automatic application re-submission for updated SparkApplication objects","default_branch":null,"files":null,"tree":[],"storefront":"/r/kubeflow","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/kubeflow/spark-operator/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}