{"repo":"feyninc/mtcb","free":true,"listed":false,"github":"https://github.com/feyninc/mtcb","clone":"git clone https://github.com/feyninc/mtcb.git","description":"The chunking benchmark","language":"Python","stars":13,"topics":["ai","chunk","chunking","judge","llm-evaluation","rag","benchmarking","observability"],"license":"Apache-2.0","category":"analytics","readme_excerpt":"🔬 mtcb ✨ The benchmark for evaluating chunking strategies in RAG pipelines. Installation • Quick Start • Benchmarks • Usage • Metrics MTCB (Massive Text Chunking Benchmark) is a standardized evaluation framework for text chunking in RAG systems. It measures how well your chunking and embedding strategy retrieves relevant passages across 9 diverse domains , from legal contracts to scientific papers. Built on top of Chonkie. 📦 Installation 🚀 Quick Start Run the lightweight nano benchmark to evaluate a chunking strategy in minutes: 🧩 Available Benchmarks Full Benchmark The full MTCB benchmark spans 9 domains with 17k questions across 3k documents: Dataset Domain Documents Questions --------- -------- ----------: ----------: 🧸 Gacha Classic Literature (Gutenberg) 100 2,878 💼 Ficha SEC Financial Filings 88 1,331 📝 Macha GitHub READMEs 445 1,812 💻 Cocha Multilingual Code 1,000 2,372 📊 Tacha Financial Tables (TAT-QA) 349 2,065 🔬 Sencha Scientific Papers (QASPER) 243 1,507 ⚖️ Hojicha Legal Contracts (CUAD) 194 1,568 🏥 Ryokucha Medical Guidelines (NICE/CDC/WHO) 241 1,351 🎓 Genmaicha MIT OCW Lecture Transcripts 250 2,037 Total 2,910 16,921 Nano Benchmark For fast iteration during development, MTCB provides a lightweight nano benchmark with 100 questions per dataset. Documents are selected to maximize question density: Dataset Domain Documents Questions --------- -------- ----------: ----------: 🧸 nano-gacha Classic Literature 5 100 💼 nano-ficha SEC Financial Filings 5 100","default_branch":null,"files":null,"tree":[],"storefront":"/r/feyninc","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/feyninc/mtcb/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}