{"repo":"itbench-hub/ITBench","free":true,"listed":false,"github":"https://github.com/itbench-hub/ITBench","clone":"git clone https://github.com/itbench-hub/ITBench.git","description":"An open source benchmarking framework for IT automation","language":"Python","stars":494,"topics":["ai","automation","hacktoberfest","it-automation","itops"],"license":"Apache-2.0","category":"workflow-automation","readme_excerpt":"ITBench Paper Leaderboard Scenarios Agents Related Benchmarks How to Cite Contributors Contacts --- 📢 Announcements Latest Updates - [May 27, 2026] Artificial Analysis and IBM Research launched ITBench-AA , the first in a new series of benchmarks evaluating frontier models on agentic enterprise IT tasks—starting with 59 SRE tasks where all evaluated models score below 50%, with FinOps and CISO tasks to follow. View the evaluation. - [January 21, 2026] IBM Research has published the Enterprise Agents and Benchmarks collection on Hugging Face, featuring ITBench alongside other enterprise AI agent ecosystems and benchmarks. View the collection. - [December 19, 2025] UC Berkeley's MAST team published a blog post analyzing ITBench SRE agent traces using MAST (Multi-Agent System Failure Taxonomy), revealing structured failure signatures that explain how and why agents fail. Read more. - [December 2, 2025] ITBench is now available on Kaggle! IBM has partnered with Kaggle to launch new AI leaderboards for enterprise tasks, including ITBench. Read more. - [November 30, 2025] A big shoutout to @phylisscity, @preespp, @tylrnguyen, @VincentCCandela, and @RMalone8 from Boston University for their contributions to ITBench, and to @Red-GV for mentoring them! - [September 18, 2025] Our paper STRATUS: A Multi-agent System for Autonomous Reliability Engineering of Modern Clouds was accepted at NeurIPS 2025. Read the paper. - [July 17, 2025] ITBench was presented as an oral at ICML 2025 (Oral ","default_branch":null,"files":null,"tree":[],"storefront":"/r/itbench-hub","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/itbench-hub/ITBench/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}