{"repo":"itsmekhoathekid/RecSys-MLops","free":true,"listed":false,"github":"https://github.com/itsmekhoathekid/RecSys-MLops","clone":"git clone https://github.com/itsmekhoathekid/RecSys-MLops.git","description":"Highly Scalable RecSys ML & Data system","language":"Python","stars":70,"topics":["apache-flink","apache-spark","apche-kafka","datahub","dbt","kubeflow-pipelines","kubernetes","mlflow","pytorch","trino"],"license":null,"category":"machine-learning","readme_excerpt":"End-to-End E-commerce Recommendation Platform A production-style, end-to-end recommendation platform for data engineering, machine learning, deployment, serving, governance, and observability workflows on Kubernetes. 🛍️ Business Domain This project is an end-to-end recommendation platform for e-commerce. It turns catalog, user, session, impression, behavior, and order data into batch and real-time features, trains a Behavior Sequence Transformer (BST), and serves personalized Top-K product recommendations through a production-style MLOps workflow. --- 📝 System Overview - Data and analytics platform: Generates configurable historical and real-time e-commerce events in PostgreSQL and MinIO, then streams CDC records through Debezium and Kafka. Spark builds batch features and Iceberg Bronze/Silver/Gold tables, while Flink handles event-time processing, deduplication, watermarking, streaming quality windows, and online feature updates. Airflow orchestrates ingestion, validation, compaction, materialization, drift, and analytics workflows; Feast serves PostgreSQL offline features and Redis online features; Hudi, DataHub, Trino, dbt, Superset, and Evidently provide dataset versioning, static batch dataset lineage, governed analytics, data quality, and drift monitoring. - ML training and retraining platform: Trains a PyTorch Behavior Sequence Transformer with time-aware datasets, negative sampling, ranking metrics, checkpointing, and ONNX/Triton model packaging. Kubeflow Pipelines ","default_branch":null,"files":null,"tree":[],"storefront":"/r/itsmekhoathekid","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/itsmekhoathekid/RecSys-MLops/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}