{"repo":"cloudposse/terraform-aws-glue","free":true,"listed":false,"github":"https://github.com/cloudposse/terraform-aws-glue","clone":"git clone https://github.com/cloudposse/terraform-aws-glue.git","description":"Terraform modules for provisioning and managing AWS Glue resources","language":"HCL","stars":34,"topics":["aws","aws-glue","etl","etl-job","glue","workflow"],"license":"Apache-2.0","category":"workflow-automation","readme_excerpt":"Terraform modules for provisioning and managing AWS Glue resources. The following Glue resources are supported: - Catalog database - Catalog table - Connection - Crawler - Job - Registry - Schema - Trigger - Workflow Refer to modules for more details. [!TIP] #### 👽 Use Atmos with Terraform Cloud Posse uses atmos to easily orchestrate multiple environments using Terraform. Works with Github Actions, Atlantis, or Spacelift. Watch demo of using Atmos with Terraform Example of running atmos to manage infrastructure from our Quick Start tutorial. Usage For a complete example, see examples/complete. The example provisions a Glue catalog database and a Glue crawler that crawls a public dataset in an S3 bucket and writes the metadata into the Glue catalog database. It also provisions an S3 bucket with a Glue Job Python script, and a destination S3 bucket for Glue job results. And finally, it provisions a Glue job pointing to the Python script in the S3 bucket, and a Glue trigger that triggers the Glue job on a schedule. The Glue job processes the dataset, cleans up the data, and writes the result into the destination S3 bucket. For an example on how to provision source and destination S3 buckets, Glue Catalog database and table, and a Glue crawler that processes data in the source S3 bucket and writes the result into the destination S3 bucket, see examples/crawler. For automated tests of the examples using bats and Terratest (which tests and deploys the examples on AWS), see test. [","default_branch":null,"files":null,"tree":[],"storefront":"/r/cloudposse","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/cloudposse/terraform-aws-glue/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}