{"repo":"ubunye-ai-ecosystems/ubunye_engine","free":true,"listed":false,"github":"https://github.com/ubunye-ai-ecosystems/ubunye_engine","clone":"git clone https://github.com/ubunye-ai-ecosystems/ubunye_engine.git","description":"Config-driven Spark framework for data and ML pipelines. Describe a pipeline once as config plus Python, then run that exact folder on a laptop, Docker, Kubernetes, cloud clusters or Databricks.","language":"Python","stars":15,"topics":["apache-spark","data-engineering","data-pipelines","databricks","etl","kubernetes","machine-learning","mlops","pipeline-framework","pyspark"],"license":null,"category":"data-pipelines","readme_excerpt":"Ubunye (oo-BOON-yeh) — isiZulu for \"unity\" One framework. Every pipeline. Any environment. Docs • Quickstart • Why Ubunye • Community --- Hey there 👋 A data pipeline is a program that moves data from one place to another — a database to a file, a REST API to a data warehouse — and usually reshapes the data along the way. Building one from scratch is mostly plumbing: wire up the connection, juggle credentials, learn a framework's quirks, write the same \"read → transform → write\" scaffold for the tenth time this year. It's a lot of glue code standing between you and the three lines that actually matter. Ubunye Engine writes that plumbing for you. You describe the pipeline in a short YAML file and put your transformation in a normal Python class. Ubunye takes care of connections, the compute engine (Apache Spark), and the read/write loop. Same pipeline runs on your laptop today and on a production cluster tomorrow, with no code changes. That sentence is tested, not hoped: a build job runs one pipeline on six environments and fails if the outputs differ by a single byte. --- What the engine gives you today - Proven portability. The same task runs on a laptop, in Docker, on Kubernetes, against object storage, through the cloud submit path, and on Databricks. One output hash across all of them, checked on every change. - Models saved anywhere. The model registry writes to a local folder, a Databricks volume, S3 or GCS, chosen purely by the path. New storage kinds are one class and","default_branch":null,"files":null,"tree":[],"storefront":"/r/ubunye-ai-ecosystems","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/ubunye-ai-ecosystems/ubunye_engine/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}