{"repo":"pathwaycom/pathway","free":true,"listed":false,"github":"https://github.com/pathwaycom/pathway","clone":"git clone https://github.com/pathwaycom/pathway.git","description":"Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.","language":"Python","stars":62439,"topics":["batch-processing","kafka","pathway","python","streaming","machine-learning-algorithms","real-time","data-analytics","data-pipelines","data-processing"],"license":null,"category":"data-pipelines","readme_excerpt":"Getting Started Deployment Documentation and Support Blog License Pathway Live Data Framework Pathway Live Data Framework is a Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG. Pathway Live Data Framework comes with an easy-to-use Python API , allowing you to seamlessly integrate your favorite Python ML libraries. Pathway Live Data Framework code is versatile and robust: you can use it in both development and production environments, handling both batch and streaming data effectively . The same code can be used for local development, CI/CD tests, running batch jobs, handling stream replays, and processing data streams. Pathway Live Data Framework is powered by a scalable Rust engine based on Differential Dataflow and performs incremental computation. Your Pathway Live Data Framework code, despite being written in Python, is run by the Rust engine, enabling multithreading, multiprocessing, and distributed computations. All the pipeline is kept in memory and can be easily deployed with Docker and Kubernetes . You can install Pathway Live Data Framework with pip: For any questions, you will find the community and team behind the project on Discord. Use-cases and templates Ready to see what Pathway Live Data Framework can do? Try one of our easy-to-run examples! Available in both notebook and docker formats, these ready-to-launch examples can be launched in just a few clicks. Pick one and start your hands-on experience with Pathway Live Data","default_branch":null,"files":null,"tree":[],"storefront":"/r/pathwaycom","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/pathwaycom/pathway/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}