{"repo":"QuantaAlpha/GitTaskBench","free":true,"listed":false,"github":"https://github.com/QuantaAlpha/GitTaskBench","clone":"git clone https://github.com/QuantaAlpha/GitTaskBench.git","description":"Repo-level benchmark for real-world Code Agents: from repo understanding → env setup → incremental dev/bug-fixing → task delivery, with cost-aware α metric.","language":"Python","stars":257,"topics":["claude-code","code-agent","openhands","repo-level","swe-agent","repomaster","code-llm"],"license":null,"category":"ai-agents","readme_excerpt":"🚀 GitTaskBench: A Benchmark for Code Agents Solving Real-World Tasks Through Code Repository Leveraging 📰 News - 2025.09.19 🎉 Excited to announce that our papers have been accepted to NeurIPS 2025 — RepoMaster as a Spotlight (≈3.2%) and SE-Agent as a Poster (≈24.52%)! - 2025.08.28 🎉 We open-sourced RepoMaster — an AI agent that leverages GitHub repos to solve complex real-world tasks. - 2025.08.26 🎉 We open-sourced GitTaskBench — a repo-level benchmark & tooling suite for real-world tasks. - 2025.08.10 🎉 We open-sourced SE-Agent — a self-evolution trajectory framework for multi-step reasoning. 🔗 Ecosystem : RepoMaster · GitTaskBench · SE-Agent · Team Homepage 🧭 Motivation and Goal The ultimate vision for AI agents is to enable users to accomplish real-world tasks simply by describing their needs in natural language—leaving all planning and execution to the agent, which delivers the final results autonomously. ⚠️ While existing benchmarks evaluate various agent capabilities, few focus on tasks that reflect genuine real-world practicality, especially those requiring comprehensive understanding and use of full-scale project repositories. 👋 To address this gap, we introduce GitTaskBench . Our benchmark focuses on tasks whose complexity and practical value demand leveraging repository-level code, mirroring how developers solve real problems using existing GitHub projects. Overview of GitTaskBench. 7 example real-life tasks from different modalities and their evaluations a","default_branch":null,"files":null,"tree":[],"storefront":"/r/QuantaAlpha","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/QuantaAlpha/GitTaskBench/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}