{"repo":"simovilab/databus-flow","free":true,"listed":false,"github":"https://github.com/simovilab/databus-flow","clone":"git clone https://github.com/simovilab/databus-flow.git","description":"Data pipeline orchestration for Databús with Airflow.","language":"Python","stars":42,"topics":["airflow","python"],"license":null,"category":"data-pipelines","readme_excerpt":"Flow This project uses Apache Airflow 2.10 to collect and process data from the Databús API. Features - Historical Data Collection : Collects arrival times from Databús API every 5 seconds - Same-day Prediction Models : Updates prediction models every 5 minutes - Long-term Prediction Models : Updates long-term models daily Setup 1. Install dependencies: 2. Set up environment variables: 3. Initialize Airflow database: 4. Create an admin user: 5. Start the web server: 6. Start the scheduler (in another terminal): 7. Access the Airflow UI: - URL: http://localhost:8080 - Username: databus - Password: databus123 DAGs The project includes three separate DAGs: 1. databus data collection : Collects historical arrival times every 5 seconds 2. databus same day models : Updates same-day prediction models every 5 minutes 3. databus long term models : Updates long-term prediction models daily Configuration - Modify airflow.cfg for Airflow settings - Update .env with your Databús API credentials - DAGs are located in the dags/ directory Project Structure Next Steps 1. Implement the actual Databús API integration in the DAG functions 2. Add data storage logic (database connections, file storage) 3. Implement the machine learning model training and prediction logic 4. Configure proper error handling and monitoring 5. Set up production deployment with external databases","default_branch":null,"files":null,"tree":[],"storefront":"/r/simovilab","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/simovilab/databus-flow/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}