{"repo":"kaivid-labs/evret","free":true,"listed":false,"github":"https://github.com/kaivid-labs/evret","clone":"git clone https://github.com/kaivid-labs/evret.git","description":"Evals framework for Information Retrieval Systems","language":"Python","stars":18,"topics":["chromadb","evals","evaluation-metrics","evaluations","information-retrieval","llm","llm-as-a-judge","metrics","milvus","ml"],"license":"MIT","category":"analytics","readme_excerpt":"🎯 Evret A focused, lightweight retriever evaluation framework with standard Information Retrieval metrics Evret brings standard Information Retrieval metrics to your recommendation, RAG and search systems. Evaluate retrievers with Hit Rate, Recall, Precision, MRR, NDCG, ERR, RBP and Average Precision in just a few lines of code. Built for simplicity, extensibility, and seamless integration with vector databases and AI frameworks. --- 🌟 Overview Evret is a modern Python framework designed for evaluating retrieval systems in Information Retrieval pipelines and search applications. It provides: - Standard IR Metrics : Hit Rate, Recall, Precision, MRR, NDCG, ERR, RBP, and Average Precision - Judge-Based Matching : Token overlap, semantic, and LLM judges for text relevance - Vector Database Support : Native adapters for Qdrant and other vector databases - Framework Integration : Adapters for LangChain and LlamaIndex --- 🚀 Quick Start Installation For optional integrations: 5-Minute Evaluation --- 🛠 Local Development Setup Use uv for local development: Install all optional integrations: Run the default test suite: See tests/README.md for coverage areas, optional integration tests, and test setup notes. --- 📊 Metrics Evret supports all standard Information Retrieval metrics: Metric Description Use Case -------- ------------- ---------- Hit Rate@k % of queries with at least one relevant doc in top-k Binary relevance, recall-focused Recall@k % of relevant docs found in top-k Comp","default_branch":null,"files":null,"tree":[],"storefront":"/r/kaivid-labs","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/kaivid-labs/evret/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}