{"repo":"CleverTap/stormdb","free":true,"listed":false,"github":"https://github.com/CleverTap/stormdb","clone":"git clone https://github.com/CleverTap/stormdb.git","description":"An embeddable, persistent key-value store optimised for sequential reads.","language":"Java","stars":10,"topics":["kv-store","kvstore","key-value","database","db","stormdb"],"license":"MIT","category":"databases-storage","readme_excerpt":"StormDB An embeddable, persistent key-value store optimised for sequential reads. Current Status: Used in production by CleverTap. Goals - High speed sequential read - High speed random writes - Affinity to cluster recently updated keys for higher random IO on such keys - Ability to recover from file corruption Benchmarks We ran a small benchmark for 100 million 4 byte keys with 28 byte values, and the results are shown below: Iteration Random Note: We've only compared this with RocksDB since in all our previous benchmarks, we've observed that RocksDB was the fastest. More details about this benchmark may be found here. Motivation The primary motivation behind StormDB was to achieve fast sequential scans across all keys in the database. When we benchmarked sequential reads of popular key value stores, we realised: 1. None of them are as fast as simply reading the file from an SSD - We used time cat 4GB file /dev/null to understand the raw throughput possible - None of the key value stores out there delivered anything close to this 1. Most are based on LSM, which requires all data to be sorted - This inherently doesn't provide an optimised technique to cluster recently updated keys together 1. Although a few keys are frequently updated and queried, LSM based stores would require all their data to be cached by the operating system file cache - On the contrary, StormDB clusters recently updated keys at the head of its data file - Therefore, the operating system can cache just th","default_branch":null,"files":null,"tree":[],"storefront":"/r/CleverTap","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/CleverTap/stormdb/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}