{"repo":"DevsHero/db2vec","free":true,"listed":false,"github":"https://github.com/DevsHero/db2vec","clone":"git clone https://github.com/DevsHero/db2vec.git","description":"db2vec: High-performance Rust CLI to parse database dumps (.sql, .surql), generate vector embeddings via Ollama, TEI, Gemini, and load into vector databases (Pinecone, Redis, Chroma, Milvus, Qdrant, SurrealDB). Optimized for speed on large datasets.","language":"Rust","stars":29,"topics":["chroma","cli","embedding","migration","mssql","mysql","ollama","oracle","pinecone","qdrant"],"license":null,"category":"databases-storage","readme_excerpt":"db2vec: From Database Dumps to Vector Search at Speed Tired of waiting hours for Python scripts to embed large database exports, especially on machines without powerful GPUs? So was I. Processing millions of records demands performance, even on standard hardware. db2vec is a high‑performance Rust tool designed for efficient CPU-based embedding generation . It parses your database dumps, generates vector embeddings using local models (Ollama, text-embeddings-inference(TEI) ) or cloud APIs (Google Gemini), and loads them into your vector database of choice – all optimized for speed without requiring a dedicated GPU. --- Core Features 🚀 Blazing Fast: Built in Rust for maximum throughput on large datasets, optimized for CPU. 🔄 Parallel Processing: Adjustable concurrency and batch‑size for embedding generation ( --num‑threads , --embedding‑concurrency , --embedding‑batch-size ). 📦 Batch Inserts: Configurable batch size ( -c, --chunk-size ) and payload limits ( -m, --max-payload-size-mb ) for efficient bulk loading into the target vector database. 🛡️ Data Filtering: Exclude sensitive tables or fields via configuration for data privacy and reduced processing time. 🔧 Highly Configurable: Fine-tune performance and behavior with extensive CLI arguments for embedding, database connections, batching, and more. 📄 Supported Dump Formats: .sql (MySQL, PostgreSQL, MSSQL, SQLite, Oracle) MSSQL: Oracle requires exporting via SQL Developer or similar into standard SQL. .surql (SurrealDB) ","default_branch":null,"files":null,"tree":[],"storefront":"/r/DevsHero","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/DevsHero/db2vec/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}