{"repo":"eigent-ai/toolathlon_gym","free":true,"listed":false,"github":"https://github.com/eigent-ai/toolathlon_gym","clone":"git clone https://github.com/eigent-ai/toolathlon_gym.git","description":"Toolathlon-Gym for testing AI agents real-world tool-use capabilities across diverse MCP servers.","language":"Python","stars":145,"topics":[],"license":"Apache-2.0","category":"mcp-servers","readme_excerpt":"Toolathlon-GYM — 503 multi-tool tasks backed by a local PostgreSQL database, no external APIs required Training and evaluating LLM agents on real-world tool use is hard. Most existing datasets are either too narrow in tool coverage, too small in scale, or depend on live external APIs that change over time. We introduce Toolathlon-GYM, a large-scale, self-contained environment with 503 tasks, 25 MCP servers, and a rich mock database. It runs entirely locally, with no external API calls required at running time. Toolathlon-GYM is built on and extends the infrastructure from Toolathlon by HKUST-NLP. The task format, evaluation framework, MCP server interfaces, and database schema design all originate from the Toolathlon project. This dataset applies the same format at larger scale, producing a substantially bigger and more diverse task pool for training and evaluation. Each task asks an agent to complete an end-to-end goal such as pulling data from a mock enterprise database, producing a spreadsheet report, scheduling a calendar event, and sending a summary email, using a fixed set of MCP (Model Context Protocol) servers as tools. Every task is fully automated: a preprocess/main.py script sets up the initial workspace state before the run, the agent executes using the provided tools, and an evaluation/main.py script checks the outputs against reference groundtruth. No human graders or live external services are involved. The dataset is designed to stress-test agent capabilities ","default_branch":null,"files":null,"tree":[],"storefront":"/r/eigent-ai","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/eigent-ai/toolathlon_gym/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}