{"repo":"reugn/go-streams","free":true,"listed":false,"github":"https://github.com/reugn/go-streams","clone":"git clone https://github.com/reugn/go-streams.git","description":"A lightweight stream processing library for Go","language":"Go","stars":2170,"topics":["stream-processing","pipeline","etl","kafka","data-stream","kafka-streams","streams","redis","pulsar","data-pipeline"],"license":"MIT","category":"data-pipelines","readme_excerpt":"go-streams go-streams provides a lightweight and efficient stream processing framework for Go. Its concise DSL allows for easy definition of declarative data pipelines using composable sources, flows, and sinks. Wiki) In computing, a pipeline, also known as a data pipeline, is a set of data processing elements connected in series, where the output of one element is the input of the next one. The elements of a pipeline are often executed in parallel or in time-sliced fashion. Some amount of buffer storage is often inserted between elements. Overview The core module has no external dependencies and provides three key components for constructing stream processing pipelines: - Source: The entry point of a pipeline, emitting data into the stream. (One open output) - Flow: A processing unit, transforming data as it moves through the pipeline. (One open input, one open output) - Sink: The termination point of a pipeline, consuming processed data and often acting as a subscriber. (One open input) Flows The flow package provides a collection of Flow implementations for common stream processing operations. These building blocks can be used to transform and manipulate data within pipelines. - Map: Transforms each element in the stream. - FlatMap: Transforms each element into a stream of slices of zero or more elements. - Filter: Selects elements from the stream based on a condition. - Fold: Combines elements of the stream with the last folded value and emits the new value. Requires an i","default_branch":null,"files":null,"tree":[],"storefront":"/r/reugn","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/reugn/go-streams/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}