{"repo":"line/decaton","free":true,"listed":false,"github":"https://github.com/line/decaton","clone":"git clone https://github.com/line/decaton.git","description":"High throughput asynchronous task processing on Apache Kafka","language":"Java","stars":371,"topics":["kafka","java","kafka-consumer"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"Decaton ======= Decaton is a streaming task processing framework built on top of Apache Kafka. It is designed to enable \"concurrent processing of records consumed from one partition\" which isn't possible in many Kafka consumer frameworks. Here is the list of property that Decaton enables by default: Concurrent (either multi-threaded or asynchronous) processing of records consumed from one partition Preserve ordering guarantee by record keys Preserve At-least-once delivery semantics no matter in which order does records gets processed Decaton is a library rather than a full-stack execution environment so you can easily integrate it into your existing/new JVM applications just by adding few lines of artifact dependencies. Since it has been designed, optimized and being used for LINE's server system which produces over 1 million, I/O intensive tasks per second for each stream, its internal implementation for enabling concurrent processing of records is highly optimized and can produce ideal throughput with minimal count of servers, maximizing their resource utilization. Getting Started / Tutorial Please see Getting Started When to use (and when to not) It's good idea to use Decaton when you have requirement for high-throughput and/or low-latency processing capability with your processing logic containing I/O access against external systems (e.g, DB access, Web API access) which tends to apply certain processing latency in each tasks. It would be better idea to look for other fra","default_branch":null,"files":null,"tree":[],"storefront":"/r/line","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/line/decaton/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}