{"repo":"astronomer/ebook-etl-elt","free":true,"listed":false,"github":"https://github.com/astronomer/ebook-etl-elt","clone":"git clone https://github.com/astronomer/ebook-etl-elt.git","description":"Companion repository to the ETL & ELT Pipelines with Apache Airflow® eBook","language":"Python","stars":45,"topics":["apache-airflow","data-engineering","elt","etl","open-source","orchestration","postgres","python","sql"],"license":null,"category":"data-pipelines","readme_excerpt":"Best practices for writing ETL and ELT pipelines This repository contains the code for the webinar demo shown in: Best practices for writing ETL and ELT pipelines. Watch the webinar here for free! This repository is configured to spin up 7 Docker containers when you run astro dev start (See Install the Astro CLI). if you cannot use the Astro CLI see Running the demo in GH codespaces. The containers are: - Postgres, port 5432: Airflow's Metadata Database - API server, port 8080: The Airflow component responsible for rendering the Airflow UI and serving several APIs - Scheduler: The Airflow component responsible for monitoring and triggering tasks - DAG Processor: The Airflow component responsible for parsing DAGs - Triggerer: The Airflow component responsible for triggering deferred tasks - Postgres, port 5433: A Postgres database for the demo data - MinIO, port 9000: An S3-compatible object storage service for the demo data To connect Airflow to both the Postgres database and MinIO, create a .env file in the root directory of the project with the exact contents of the .env.example file. Note that you need to restart the Airflow instance with astro dev restart after creating the .env file for the changes to take effect. [!TIP] You need to be on at least version 1.34.0 of the Astro CLI in oder to run this repo. You can check your version with astro version and upgrade with brew upgrade astro All the DAGs run without any further setup or tools needed! Content This repository con","default_branch":null,"files":null,"tree":[],"storefront":"/r/astronomer","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/astronomer/ebook-etl-elt/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}