{"repo":"cnstlungu/portable-data-stack-airflow","free":true,"listed":false,"github":"https://github.com/cnstlungu/portable-data-stack-airflow","clone":"git clone https://github.com/cnstlungu/portable-data-stack-airflow.git","description":"A portable Datamart and Business Intelligence suite built with Docker, Airflow, dbt, duckdb and Superset","language":"Python","stars":49,"topics":["airflow","superset","docker","etl","python3","dbt","postgresql","duckdb"],"license":"MIT","category":"data-pipelines","readme_excerpt":"Portable Airflow Data Stack This application is an Analytics suite for an imaginary company selling postcards. The company sells both directly but also through resellers in the majority of European countries. Stack - Airflow - Docker (Docker Compose) - DuckDB - dbt core - Superset Interested in the data model? Generation of example data and the underlying dbt-core model is available in the postcard-company-datamart project For implementations using other tools, using the same data model, check the below: - portable-data-stack-mage - portable-data-stack-dagster - portable-data-stack-sqlmesh Implementations not using the same dbt-core model: - portable-data-stack-bruin - postcard-company-dataform The Airflow / docker-compose setup is based on the official Airflow docker-compose example Setup 1. Rename .env.example file to .env and set your desired credentials. Remember to never commit files containing passwords or any other sensitive information. 2. Rename shared/db/datamart.duckdb.example to shared/db/datamart.duckdb or init an empty database file there with that name. 3. With Docker Engine installed, change directory to the root folder of the project (also the one that contains docker-compose.yml) and run docker compose up --build Note that this may take several minutes to complete. Check out the console to see when the Airflow interface is ready. 4. Once the Docker suite has finished loading, open up Airflow and execute one of the DAGs: - run dbt individual models - this wil","default_branch":null,"files":null,"tree":[],"storefront":"/r/cnstlungu","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/cnstlungu/portable-data-stack-airflow/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}