{"repo":"modelscope/MCPBench","free":true,"listed":false,"github":"https://github.com/modelscope/MCPBench","clone":"git clone https://github.com/modelscope/MCPBench.git","description":"The evaluation benchmark on MCP servers","language":"Python","stars":251,"topics":["benchmark","database","mcp","mcp-server","websearch"],"license":"Apache-2.0","category":"mcp-servers","readme_excerpt":"🦊 MCPBench: A Benchmark for Evaluating MCP Servers [![Documentation][docs-image]][docs-url] [![Package License][package-license-image]][package-license-url] 中文 English MCPBench is an evaluation framework for MCP Servers. It supports the evaluation of three types of servers: Web Search, Database Query and GAIA, and is compatible with both local and remote MCP Servers. The framework primarily evaluates different MCP Servers (such as Brave Search, DuckDuckGo, etc.) in terms of task completion accuracy, latency, and token consumption under the same LLM and Agent configurations. Here is the evaluation report. The implementation refers to LangProBe: a Language Programs Benchmark.\\ Big thanks to Qingxu Fu for the initial implementation! 📋 Table of Contents - 🔥 News - 🛠️ Installation - 🚀 Quick Start - Launch MCP Server - Launch Evaluation - 🧂 Datasets and Experiments - 🚰 Cite 🔥 News + Sep. 1, 2025 🌟 Modelscope AI hackathon will be hold on Sep. 23rd, ref: https://modelscope.cn/active/aihackathon-mcp-agent + Apr. 29, 2025 🌟 Update the code for evaluating the MCP Server Package within GAIA. + Apr. 14, 2025 🌟 We are proud to announce that MCPBench is now open-sourced. 🛠️ Installation The framework requires Python version = 3.11, nodejs and jq. 🚀 Quick Start Please first determine the type of MCP server you want to use: - If it is a remote host (accessed via SSE , such as ModelScope, Smithery, or localhost), you can directly conduct the evaluation. - If it is started locally ","default_branch":null,"files":null,"tree":[],"storefront":"/r/modelscope","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/modelscope/MCPBench/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}