{"repo":"run-llama/ParseBench","free":true,"listed":false,"github":"https://github.com/run-llama/ParseBench","clone":"git clone https://github.com/run-llama/ParseBench.git","description":"ParseBench - A Document Parsing Benchmark for AI Agents","language":"Python","stars":550,"topics":["benchmark","document-ai","document-parsing","evaluation","llamaindex","llm","machine-learning","ocr","pdf-parsing","table-extraction"],"license":"Apache-2.0","category":"ai-agents","readme_excerpt":"ParseBench ParseBench is a benchmark for evaluating how well document parsing tools convert PDFs into structured output that AI agents can reliably act on. It tests whether parsed output preserves the structure and meaning needed for autonomous decisions — not just whether it looks similar to a reference text. The benchmark covers 2,000 human-verified pages from real enterprise documents (insurance, finance, government), organized around five capability dimensions, each targeting a failure mode that breaks production agent workflows. Leaderboard Top 10 by Overall score. For the full sortable, filterable leaderboard, see parsebench.ai; for raw data, see leaderboard.csv. Rank Provider Category Overall Tables Charts Content Faith. Sem. Format. Visual Ground. ¢ / Page ---: --- --- ---: ---: ---: ---: ---: ---: ---: 1 LlamaParse Agentic LlamaParse 84.88 90.74 78.11 89.68 85.24 80.62 1.25¢ 2 Pulse Ultra 2 Commercial - Startup APIs 77.08 75.45 90.82 79.49 73.05 66.56 15¢ 3 LlamaParse Cost Effective LlamaParse 76.77 81.42 70.15 90.92 68.78 72.59 0.38¢ 4 KDL-Frontier-Parser-nano VLM - Open Weight 76.36 85.56 63.41 87.19 66.81 78.84 — 5 Extend (2.0) Commercial - Startup APIs 75.33 84.82 78.31 84.59 60.31 68.61 2.50¢ 6 Google Gemini 3 Flash (Thinking High) VLM - Proprietary 75.05 91.50 64.79 90.87 68.31 59.77 2.41¢ 7 Infinity-Parser2-Pro VLM - Open Weight 74.28 86.4 61.3 89.7 59.1 74.9 — 8 Extend Light (1.0) Commercial - Startup APIs 73.26 75.8 78.6 84.8 58.6 68.5 0.62¢ 9 Infinity-Parse","default_branch":null,"files":null,"tree":[],"storefront":"/r/run-llama","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/run-llama/ParseBench/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}