{"repo":"k3XD16/contoso-retail-analytics","free":true,"listed":false,"github":"https://github.com/k3XD16/contoso-retail-analytics","clone":"git clone https://github.com/k3XD16/contoso-retail-analytics.git","description":"End-to-end ELT batch pipeline | Medallion Architecture (Bronze → Silver → Gold) | dbt + Snowflake + AWS S3 + Apache Airflow + Docker | 3.5M+ records","language":"Python","stars":11,"topics":["airflow","aws-s3","data-engineering","dbt","docker","elt-pipeline","kimball","medallion-architecture","python","snowflake"],"license":"MIT","category":"data-pipelines","readme_excerpt":"End-to-end batch pipeline dbt + Snowflake + AWS S3 + Apache Airflow Medallion Architecture 3.5M+ records --- Overview An ELT pipeline implementing Medallion Architecture (Bronze → Silver → Gold) on the Microsoft Contoso dataset — a realistic multi-country retail dataset with intentional data quality issues, making it well-suited for building and testing production-style transformation patterns. The pipeline ingests raw CSVs from S3 into Snowflake, applies layered dbt transformations into a Kimball star schema, and is fully orchestrated with Apache Airflow running in Docker. Metric Value -------- ------- Records Processed 3,537,806 dbt Models 15 (4 dims + 1 fact + 1 OBT + 7 staging + 2 intermediate) Seeds 2 ( seed currency lookup , seed calendar ) SCD Type 2 Snapshots 3 ( snap customers , snap products , snap stores ) Data Quality Tests 46 (41 generic + 5 singular) Airflow Tasks 8 --- Architecture --- Tech Stack Layer Technology ------- ----------- Storage AWS S3 Warehouse Snowflake Transformation dbt Core 1.11.7 Orchestration Apache Airflow 3.1.7 Containerization Docker + Docker Compose IAM AWS IAM Roles Language SQL, Jinja, YAML, Python --- Data Model (Gold Layer) Star Schema — Kimball methodology - fact sales — 2,349,091 rows grain: order line item incremental load - dim customers — 104,990 rows demographics, CLV, segmentation (A/B/C/D) - dim products — 2,517 rows category hierarchy, cost & retail pricing - dim stores — 74 rows 8 countries, open/closed/restructured status -","default_branch":null,"files":null,"tree":[],"storefront":"/r/k3XD16","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/k3XD16/contoso-retail-analytics/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}