{"repo":"StarlightSearch/EmbedAnything","free":true,"listed":false,"github":"https://github.com/StarlightSearch/EmbedAnything","clone":"git clone https://github.com/StarlightSearch/EmbedAnything.git","description":"Highly Performant, Modular, Memory Safe and Production-ready Inference, Ingestion and Indexing built in Rust 🦀","language":"Rust","stars":1296,"topics":["machine-learning","indexing","rag","rust","information-retrieval","vector-database","high-performance","inference","python","onnxruntime"],"license":"Apache-2.0","category":"machine-learning","readme_excerpt":"Highly Performant, Modular and Memory Safe Ingestion, Inference and Indexing in Rust 🦀 Python docs » Rust docs » Benchmarks · FAQ · Adapters . Collaborations . Notebooks EmbedAnything is a minimalist, yet highly performant, modular, lightning-fast, lightweight, multisource, multimodal, and local embedding pipeline built in Rust. Whether you're working with text, images, audio, PDFs, websites, or other media, EmbedAnything streamlines the process of generating embeddings from various sources and seamlessly streaming (memory-efficient-indexing) them to a vector database. It supports dense, sparse, ONNX, model2vec and late-interaction embeddings, offering flexibility for a wide range of use cases. Table of Contents About The Project Built With Rust Why Candle? Getting Started Installation Usage Roadmap Contributing How to add custom model and chunk size 🚀 Key Features - No Dependency on Pytorch : Easy to deploy on cloud, comes with low memory footprint. - Highly Modular : Choose any vectorDB adapter for RAG, with 1 line 1 word of code - Backend : Supports Candle, ONNX and cloud models - MultiModality : Works with text sources like PDFs, txt, md, Images JPG and Audio, .WAV - GPU support : Hardware acceleration on GPU as well. - Chunking : In-built chunking methods like semantic, late-chunking - Vector Streaming: : Separate file processing, Indexing and Inferencing on different threads, reduces latency. - AWS S3 Bucket: : Directly import AWS S3 bucket files. - Prebult Docker Ima","default_branch":null,"files":null,"tree":[],"storefront":"/r/StarlightSearch","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/StarlightSearch/EmbedAnything/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}