{"repo":"TIGER-AI-Lab/ClawBench","free":true,"listed":false,"github":"https://github.com/TIGER-AI-Lab/ClawBench","clone":"git clone https://github.com/TIGER-AI-Lab/ClawBench.git","description":"Open-source benchmark for browser AI agents on daily tasks.","language":"Python","stars":568,"topics":["ai-agents","benchmark","browser-automation","browser-use","dataset","evaluation","llm","web-agents","computer-use","chrome-extension"],"license":"Apache-2.0","category":"ai-agents","readme_excerpt":"More badges · featured in 37 curated lists ClawBench: Can AI Agents Complete Everyday Online Tasks? ClawBench is an open-source benchmark that evaluates AI browser agents on everyday online tasks — booking travel, ordering food, applying for jobs, managing email — across live websites. V1 lives in test-cases/v1/ , V2 in test-cases/v2/ . It measures end-to-end task success with a 5-layer recording pipeline and an agentic evaluator that compares each run against human references. Top score to date: 33.3%. We asked frontier AI agents to do what people do every day -- order food, book travel, apply for jobs, write reviews, manage projects. Even the best agent only completes about 1 in 3. --- V1: 152 everyday tasks · 143 live sites       V2: 129 tasks · 63 live sites       281 total across 163 live websites · 15 life categories The paper reports 153 (V1) and 130 (V2); two ASPCA tasks were removed after publication, so the shipping corpus is 152 and 129. Built by NAIL Group  ·  Sister project: HarnessBench — fixes the base model, varies the harness  ·  Runs on any Chrome. 中文 What are you looking for? 🏆 See scores Live leaderboard Pick a corpus (v1 / v2) 🚀 Run it on your model Quick start ↓ pip install clawbench-eval 📊 Browse 281 tasks Task explorer Search · filter · category 📄 Read the paper arXiv:2604.08523 Methodology · evaluator · results 🎬 Re-grade old runs V1 · V2 raw traces 5 layers p","default_branch":null,"files":null,"tree":[],"storefront":"/r/TIGER-AI-Lab","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/TIGER-AI-Lab/ClawBench/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}