{"repo":"ukanwat/aaabench","free":true,"listed":false,"github":"https://github.com/ukanwat/aaabench","clone":"git clone https://github.com/ukanwat/aaabench.git","description":"A long-horizon benchmark harness: give a coding agent a real game engine, professional conditions and time, and ask it to build an open-world game. Harness only, no results.","language":"Shell","stars":365,"topics":["ai-agents","benchmark","game-development","llm-evaluation","mcp","unreal-engine"],"license":"MIT","category":"game-templates","readme_excerpt":"AAABench Give a coding agent a real game engine, professional conditions and time. Ask it to build an open-world game. Then go away. Built and run by Utkarsh Kanwat · 𝕏 · run it on another model and open an issue with what broke One agent. One Unreal Engine editor, driven live over MCP. A fierce brief, a shelf of production knowledge, and no human help. The agent decides everything — the geography, the districts, the roads, the buildings, the people, the traffic, the weather, the game's own screens, and what to fix when it doesn't work. Nobody points at anything for it. The one rule. Provide conditions, resources and the demand — never diagnosis, never the fix, never an answer. Whether the model notices its own mistakes is the capability being measured, so every hint is a result you can no longer claim. See HARNESS-RULES.md . What it actually measures Building a world turns out to be an unusually complete test, because it cannot be faked by pattern-matching a familiar task: - Real-world understanding. Does the model know how a city works — that deep water decides where the port goes, that industry follows the rail, that money builds uphill and upwind, that sunlight limits how tall a street can be, that a courthouse grows bail bonds around it? A world built without that knowledge looks wrong instantly, to anyone, with no expertise required. - Reasoning from causes rather than from examples. The brief demands that nothing be placed because it looked good there. Every district,","default_branch":null,"files":null,"tree":[],"storefront":"/r/ukanwat","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/ukanwat/aaabench/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}