{"repo":"DhanushN2005/crypto-price-streaming-kafka-snowflake-ml-pipeline","free":true,"listed":false,"github":"https://github.com/DhanushN2005/crypto-price-streaming-kafka-snowflake-ml-pipeline","clone":"git clone https://github.com/DhanushN2005/crypto-price-streaming-kafka-snowflake-ml-pipeline.git","description":"A real world crypto currencies streaming and analysing using kafka through snowflake.","language":"Python","stars":15,"topics":["cryptocurrency","dockerr","kakfa","live-streaming","ml-model","regression","snowflake","streamlit-webapp"],"license":null,"category":"blockchain-web3","readme_excerpt":"🚀 Real-Time Crypto Streaming & ML Pipeline An end-to-end real-time data engineering and machine learning project that streams live crypto price data from an external API, processes it using Kafka, stores it in Snowflake, trains ML models for prediction and anomaly detection, and visualizes results using a dashboard. --- 🧠 Project Overview This project demonstrates how modern data systems handle real-time streaming data , perform analytics and ML model training , and serve insights through a simple application. 🔄 Data Flow Finnhub API → Kafka (Docker) → Snowflake → ML Models → Dashboard --- 🛠️ Tech Stack - API : Finnhub (Live Crypto Prices) - Streaming : Apache Kafka (Dockerized) - Data Warehouse : Snowflake - Machine Learning : - Regression (Price Prediction) - Isolation Forest (Anomaly Detection) - Visualization : Streamlit - Language : Python --- Sample output; ⚙️ Setup & Installation 1️⃣ Prerequisites - Python 3.9+ - Docker Desktop - Snowflake account - Finnhub API key --- 2️⃣ Install Python Dependencies 3️⃣ Start Kafka (Docker) Verify: 4️⃣ Create Snowflake Table Run in Snowflake worksheet: ▶️ Running the Pipeline 🔹 Start Producer (Finnhub → Kafka) python finnhub producer.py 🔹 Start Consumer (Kafka → Snowflake) python kafka to snowflake.py Data will start streaming into Snowflake in real time. 🤖 Machine Learning Run the ML pipeline: This performs: Feature engineering -Price prediction (regression) -Anomaly detection -Saves outputs to CSV files 📊 Dashboard Launch th","default_branch":null,"files":null,"tree":[],"storefront":"/r/DhanushN2005","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/DhanushN2005/crypto-price-streaming-kafka-snowflake-ml-pipeline/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}