{"repo":"nubskr/satoriDB","free":true,"listed":false,"github":"https://github.com/nubskr/satoriDB","clone":"git clone https://github.com/nubskr/satoriDB.git","description":"High performance embedded vector database","language":"Rust","stars":240,"topics":["database","embeddings","rust","vector-database","ai","document-retrieval","fuzzy-search","llm","nearest-neighbor-search","rag"],"license":"MIT","category":"databases-storage","readme_excerpt":"SatoriDB: Billion scale embedded vector database SatoriDB is an embedded vector database built for bigger than memory workloads for ANN search and handles billion scale datasets with 95%+ recall and predictable latencies. Architecture SatoriDB is a two-tier search system: a small \"hot\" index in RAM routes queries to \"cold\" vector data on disk. This lets us handle billion-scale datasets without holding everything in memory. Routing (Hot Tier) Quantized HNSW index over bucket centroids. Centroids are scalar-quantized (f32 → u8) so the whole routing index fits in RAM even at 500k+ buckets. When a query comes in, HNSW finds the top-K most relevant buckets in O(log N). We only search those, not the entire dataset. Scanning (Cold Tier) CPU-pinned Glommio workers scan selected buckets in parallel. Shared-nothing: each worker has its own io uring ring, LRU cache, and pre-allocated heap. No cross-core synchronization on the query path. SIMD everywhere: L2 distance, dot products, quantization, k-means assignments all have AVX2/AVX-512 paths. Cache misses stream from disk without blocking. Clustering & Rebalancing Vectors are grouped into buckets (clusters) via k-means. A background rebalancer automatically splits buckets when they exceed 2000 vectors, keeping bucket sizes predictable. Predictable sizes = predictable query latency. Inspired by SPFresh. Storage Walrus handles bulk vector storage (append-only, io uring, topic-per-bucket). RocksDB indexes handle point lookups (fetch-by-id,","default_branch":null,"files":null,"tree":[],"storefront":"/r/nubskr","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/nubskr/satoriDB/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}