{"repo":"himewel/covid19retail","free":true,"listed":false,"github":"https://github.com/himewel/covid19retail","clone":"git clone https://github.com/himewel/covid19retail.git","description":"Lambda data architecture reference project — dbt medallion models, Airflow DAGs, Superset dashboards, and Marquez lineage for Iowa retail and Covid-19 analytics on GCP","language":"Jupyter Notebook","stars":14,"topics":["dbt","openlineage","apache-airflow","apache-superset","bigquery","covid-19","data-analytics","data-engineering","data-lineage","dimensional-modeling"],"license":null,"category":"data-pipelines","readme_excerpt":"Covid-19 / Retail analysis with a modern data stack This project builds a lambda data architecture using services and technologies from a modern data stack. Serverless or PaaS services are preferred over IaaS where possible, and UI-based data wrangling tools are favored over hand-written glue code when practical. The pipeline ingests Iowa liquor retail sales and Covid-19 public datasets from BigQuery, transforms them with dbt into a dimensional model, orchestrates runs with Airflow, tracks lineage with Marquez, and visualizes results in Superset. Table of contents - Architecture - Data model - Prerequisites - Quick start - Running services individually - Project structure - Dashboards and notebooks Architecture The following diagram shows the local stack running in Docker: Locally, the stack is available at: Service URL Default credentials ---------- ------------------------- --------------------- Airflow http://localhost:8080 airflow / airflow Marquez http://localhost:3000 — Superset http://localhost:8088 admin / admin For a cloud deployment, the project targets GCP services together with Airbyte Cloud, the Astronomer platform, Datakin, and dbt Cloud: Data model Data is organized in a medallion-style layout inside the covid19retail BigQuery dataset: Layer dbt folder / schema Materialization Description --------- --------------------- ----------------- ------------- Silver trusted Incremental Cleaned extracts from BigQuery public datasets Gold app Table Dimensional model (sta","default_branch":null,"files":null,"tree":[],"storefront":"/r/himewel","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/himewel/covid19retail/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}