{"repo":"evolution-gaming/kafka-journal","free":true,"listed":false,"github":"https://github.com/evolution-gaming/kafka-journal","clone":"git clone https://github.com/evolution-gaming/kafka-journal.git","description":"Event sourcing journal implementation using Kafka as main storage","language":"Scala","stars":113,"topics":["kafka","journal","event-sourcing","akka-persistence","cassandra","scala"],"license":"MIT","category":"data-pipelines","readme_excerpt":"Kafka Journal Stream data from two sources where one is eventually consistent and the other one loses its tail This library provides the ability to use Kafka as storage for events. Kafka is a perfect fit in case you want to have streaming capabilities for your events. However, kafka-journal also uses Cassandra to keep performance of data access at acceptable level and overcome Kafka's retention policy . Cassandra is a default choice, but you may use any other storage which satisfies the following interfaces: Read side, called within client library EventualJournal Write side, called from replicator app ReplicatedJournal High level description Writing flow 1. Journal client publishes events to Kafka 2. Replicator app stores events in Cassandra Reading flow 1. Client publishes special marker Mark to Kafka, so we can make sure there are no more events to expect 2. Client reads events from Cassandra, however, at this point we are not yet sure that all events are replicated from Kafka to Cassandra 3. Client reads events from Kafka using offset of last event found in Cassandra 4. We consider recovery finished when the marker Mark is read from Kafka Notes one Kafka topic may be used for many different entities not all events must be on Kafka, as long as they are stored in Cassandra there is no snapshot support yet Replicator is a separate application running concurrently with Journal clients it is relatively easy to replace Cassandra with some other relational database State recovery","default_branch":null,"files":null,"tree":[],"storefront":"/r/evolution-gaming","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/evolution-gaming/kafka-journal/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}