{"repo":"tradewelltech/beavers","free":true,"listed":false,"github":"https://github.com/tradewelltech/beavers","clone":"git clone https://github.com/tradewelltech/beavers.git","description":"Python stream processing for analytics","language":"Python","stars":41,"topics":["analytics","apache-arrow","data","kafka","pandas","python","realtime","stream-processing"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"[![PyPI Version][pypi-image]][pypi-url] [![Python Version][versions-image]][versions-url] [![Github Stars][stars-image]][stars-url] [![codecov][codecov-image]][codecov-url] [![Build Status][build-image]][build-url] [![Documentation][doc-image]][doc-url] [![License][license-image]][license-url] [![Downloads][downloads-image]][downloads-url] [![Downloads][downloads-month-image]][downloads-month-url] [![Ruff][ruff-image]][ruff-url] [![prek][prek-image]][prek-url] [![snyk][snyk-image]][snyk-url] ![Beavers Logo][5] Beavers [Documentation][6] / [Installation][7] / [Repository][1] / [PyPI][8] [Beavers][1] is a python library for stream processing, optimized for analytics. It is used at [Tradewell Technologies][2], to calculate analytics and serve model predictions, for both realtime and batch jobs. Key Features - Works in real time (eg: reading from Kafka) and replay mode (eg: reading from Parquet files). - Optimized for analytics, using micro-batches (instead of processing records one by one). - Similar to [incremental][3], it updates nodes in a dag incrementally. - Taking inspiration from [kafka streams][4], there are two types of nodes in the dag: - Stream : ephemeral micro-batches of events (cleared after every cycle). - State : durable state derived from streams. - Clear separation between the business logic and the IO. So the same dag can be used in real time mode, replay mode or can be easily tested. - Functional interface: no inheritance or decorator required. - Support for ","default_branch":null,"files":null,"tree":[],"storefront":"/r/tradewelltech","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/tradewelltech/beavers/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}