{"repo":"david-g-3654/homebench","free":true,"listed":false,"github":"https://github.com/david-g-3654/homebench","clone":"git clone https://github.com/david-g-3654/homebench.git","description":"Benchmark your local LLMs: speed, memory, and quality, in one command. TUI leaderboard for Ollama, LM Studio, llama.cpp, and vLLM.","language":"Python","stars":53,"topics":["benchmark","cli","evaluation","llama-cpp","llm","lm-studio","local-ai","local-llm","ollama","python"],"license":"MIT","category":"ai-agents","readme_excerpt":"homebench Benchmark the local LLMs you already have: speed, memory, and quality — as a live terminal leaderboard. homebench is a single-command TUI that discovers the models installed in your local runner ( Ollama , LM Studio , llama.cpp , vLLM , or any OpenAI-compatible server), runs a curated quality suite, measures tokens/sec , time-to-first-token , and memory footprint on your actual machine , and renders a live comparison leaderboard. That's it. No config, no API keys, no cloud. --- Why There are great tools for one half of this problem, but nothing local-first that does both: - llama-bench (inside llama.cpp) measures speed only . - lm-evaluation-harness measures quality but has no polished laptop UX and isn't built around the model runners most people actually use locally. homebench fills the gap: local-first, zero-config, UX-driven. Clone-and-run, point it at the models you already pulled, and get an at-a-glance answer to \"which of my local models is actually good, and how fast is it on this laptop?\" What it measures Metric How --- --- tok/s Output tokens ÷ generation time. Ollama reports server-side eval timing; OpenAI-compatible backends are timed client-side from the token stream. Excludes prompt processing and model load. TTFT Wall-clock time to the first streamed token (minus model-load time where the runner reports it). Memory Two numbers, labeled: Memory = resident model size the runner reports (Ollama /api/ps , LM Studio /api/v0 ); Peak = peak process-RSS growt","default_branch":null,"files":null,"tree":[],"storefront":"/r/david-g-3654","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/david-g-3654/homebench/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}