{"repo":"ClusterlessHQ/clusterless","free":true,"listed":false,"github":"https://github.com/ClusterlessHQ/clusterless","clone":"git clone https://github.com/ClusterlessHQ/clusterless.git","description":"Clusterless is a tool for scheduling decentralized, scalable, and secure data pipelines for continuously arriving data, across clouds.","language":"Java","stars":15,"topics":["aws","cli","cloud","data-engineering","data-science","java","python","mlops"],"license":null,"category":"data-pipelines","readme_excerpt":"Clusterless Status This project is under active development, some apis are experimental and may change between releases. Draft documentation can be found here: https://docs.clusterless.io/ - CLI usage - Project components - Project file format See the ROADMAP for planned capabilities. For example scenarios, see - How To - step-by-step guides - clusterless-aws-example - simple examples to start with - aws-s3-log-pipeline - end-to-end sample pipeline for processing AWS S3 access logs About Clusterless is a tool for deploying decentralized, scalable, and secure data-processing workloads for continuously arriving data, across clouds. By leveraging native pay-as-you-go primitives, no runtimes or dedicated services need to be managed. Zero data arriving means zero costs (other than storage for historical data). By standardizing the metadata as new data arrives, workloads can be implemented behind a consistent interface, and can be easily deployed to listen for upstream availability events and fire new events to downstream listeners. Not only can heterogeneous workloads can be chained together via events, they can be: - back-filled - a new workload (or version) can run against historical data - replayed - a fixed workload can re-run over a range of data to correct for an error in place - checked for gaps - missing data can be accounted for by walking the upstream dependencies - enabled/disabled - trivially pause or restart event listening - versioned - new versions of workloads can ","default_branch":null,"files":null,"tree":[],"storefront":"/r/ClusterlessHQ","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/ClusterlessHQ/clusterless/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}