{"repo":"awslabs/amazon-emr-cli","free":true,"listed":false,"github":"https://github.com/awslabs/amazon-emr-cli","clone":"git clone https://github.com/awslabs/amazon-emr-cli.git","description":"A command-line interface for packaging, deploying, and running your EMR Serverless Spark jobs","language":"Python","stars":47,"topics":["aws","amazon-emr","apache-spark","emr-serverless"],"license":"Apache-2.0","category":"cli-tools","readme_excerpt":"EMR CLI So we're all working on data pipelines every day, but wouldn't be nice to just hit a button and have our code automatically deployed to staging or test accounts? I thought so, too, thats why I created the EMR CLI ( emr ) that can help you package and deploy your EMR jobs so you don't have to. The EMR CLI supports a wide variety of configuration options to adapt to your data pipeline, not the other way around. 1. Packaging - Ensure a consistent approach to packaging your production Spark jobs. 2. Deployment - Easily deploy your Spark jobs across multiple EMR environments or deployment frameworks like EC2, EKS, and Serverless. 3. CI/CD - Easily test each iteration of your code without resorting to messy shell scripts. :) The initial use cases are: 1. Consistent packaging for PySpark projects. 2. Use in CI/CD pipelines for packaging, deployment of artifacts, and integration testing. Warning : This tool is still under active development, so commands may change until a stable 1.0 release is made. Quick Start You can use the EMR CLI to take a project from nothing to running in EMR Serverless is 2 steps. First, let's install the emr command. Note This tutorial assumes you have already setup EMR Serverless and have an EMR Serverless application, job role, and S3 bucket you can use. If not, you can use the emr bootstrap command. 1. Create a sample project 📔 Tip: Use --project-type poetry to create a Poetry project! You should now have a sample PySpark project in your scratch ","default_branch":null,"files":null,"tree":[],"storefront":"/r/awslabs","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/awslabs/amazon-emr-cli/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}