{"repo":"gbazad93/AirFlow-ML-Data-Integration","free":true,"listed":false,"github":"https://github.com/gbazad93/AirFlow-ML-Data-Integration","clone":"git clone https://github.com/gbazad93/AirFlow-ML-Data-Integration.git","description":"An Airflow-based pipeline that fetches data from a free API, cleans and transforms it, and saves it to a database—ready for downstream machine learning.","language":"Python","stars":15,"topics":["airflow","api-integration","data-engineering","data-integration","data-pipeline","etl","machine-learning","postgresql","python","python3"],"license":null,"category":"data-pipelines","readme_excerpt":"AirFlow-ML-Data-Integration This project demonstrates how to use Apache Airflow for orchestrating a daily ETL pipeline that fetches weather data from the free OpenWeather API, cleans and transforms the data, and stores it in a PostgreSQL database. It provides a scalable and reproducible example that can be extended for machine learning tasks such as training predictive models on weather-related datasets. Table of Contents - Overview - Screenshots - Features - Architecture - Setup Instructions - Airflow Setup - Database Setup - Folder Setup - IDE Setup - Airflow Database Connection Setup - API Connection Setup - Project Structure - Contributing Overview This repository offers a sample Airflow project integrating a daily weather data fetch from OpenWeather’s free API into a PostgreSQL database. The data is cleaned, validated, and ready for further downstream tasks, such as ML model training or dashboard visualization. To run this project: 1. Follow the steps in the Setup Instructions section in the specified order, as the sequence is crucial to properly configuring your environment, database, and connections. 2. Download or clone the DAG and related codes in the exact same structure as described in the Project Structure section. 3. Use the Airflow UI to trigger and monitor the pipeline execution. Screenshots Below are some screenshots showcasing the Airflow UI and the weather data pipeline DAG in action. These visuals provide an overview of how the pipeline is orchestrated with","default_branch":null,"files":null,"tree":[],"storefront":"/r/gbazad93","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/gbazad93/AirFlow-ML-Data-Integration/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}