{"repo":"ManuelGuerra1987/data-engineering-zoomcamp-notes","free":true,"listed":false,"github":"https://github.com/ManuelGuerra1987/data-engineering-zoomcamp-notes","clone":"git clone https://github.com/ManuelGuerra1987/data-engineering-zoomcamp-notes.git","description":"Detailed notes and homeworks from 2025 Data Engineering Zoomcamp by Datatalks.Club","language":"Python","stars":59,"topics":["airflow","bigquery","data-engineering","docker","kubernetes","data","data-warehouse","dbt","pyspark","spark"],"license":null,"category":"data-pipelines","readme_excerpt":"Data Engineering Zoomcamp 2025 This repo contains notes and homeworks for the Data Engineering Zoomcamp by Datatalks.Club Syllabus Module 1: Containerization and Infrastructure as Code Docker and docker-compose Running Postgres in a container Ingesting data to Postgres with Python Running Postgres and pgAdmin with Docker-compose Google Cloud Platform (GCP) Terraform Setting up infrastructure on GCP with Terraform Module 2: Workflow Orchestration with Kestra/) Introduction to Workflow Orchestration Introduction to Kestra Launch Kestra using Docker Compose ETL Pipelines: Load Data to Local Postgres ETL Pipelines: Load Data to Google Cloud Platform (GCP) Module 3: Data Warehouse with BigQuery OLAP vs OLTP Data Warehouse BigQuery Creating an external table Partitioning and clustering BigQuery best practices Module 4: Analytics Engineering with dbt Introduction to analytics engineering Introduction to dbt Setting up dbt with bigquery Development of dbt Models Building the model Testing and documenting Deployment Visualizing the data Module 5: Batch Processing with spark Introduction to Batch Processing Introduction to Spark Spark SQL and DataFrames Spark Internals Running Spark in the Cloud Extra: Module 2: Workflow Orchestration with Airflow Data Lake vs Data Warehouse ETL vs ELT Introduction to Workflow Orchestration Airflow architecture Setting up Airflow with Docker Ingesting data to local Postgres with Airflow Ingesting data to GCP with Airflow Airflow with kubernetes Environ","default_branch":null,"files":null,"tree":[],"storefront":"/r/ManuelGuerra1987","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/ManuelGuerra1987/data-engineering-zoomcamp-notes/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}