{"repo":"Gifted-s/velarixdb","free":true,"listed":false,"github":"https://github.com/Gifted-s/velarixdb","clone":"git clone https://github.com/Gifted-s/velarixdb.git","description":"An LSM storage engine designed for high throughput and significant reduction in I/O amplification written in safe rust (Under active development)","language":"Rust","stars":32,"topics":["async","database","database-engine","datastore","key-value-store","kv-store","leveldb","lsm-storage","lsm-tree","rocksdb"],"license":"MIT","category":"databases-storage","readme_excerpt":"Contributors ✨ Thanks to these amazing people for their contributions: @NickNYU, @arrowler VelarixDB is an LSM-based storage engine designed to significantly reduce IO amplification, resulting in better performance and durability for storage devices. Introduction VelarixDB: Designed to reduce I/O amplification VelarixDB is an ongoing project ( not production ready ) designed to optimize data movement during load times and compaction. Inspired by the WiscKey paper, WiscKey: Separating Keys from Values in SSD-conscious Storage, velarixdb aims to significantly enhance performance over traditional key-value stores. Problem During compaction in LevelDB or RocksDB, in the worst case, up to 10 SSTable files need to be read (more accurately, based on the key distribution per level), sorted, and re-written since keys are not allowed to overlap across all the sstables from Level 1 downwards. Suppose after merging SSTables in one level, the next level exceeds its threshold; compaction can cascade from Level 0 all the way to Level 6, meaning the overall write amplification can be up to 50 (ignoring the first compaction level). Reference - [Official LevelDB Compaction Process Docs ]. This repetitive data movement can cause significant wear on SSDs, reducing their lifespan due to the high number of write cycles. The goal is to minimize the amount of data moved during compaction, thereby reducing the amount of data re-written and extending the device's lifetime. Solution To address this, we","default_branch":null,"files":null,"tree":[],"storefront":"/r/Gifted-s","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Gifted-s/velarixdb/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}