{"repo":"Kylejeong2/mcpvals","free":true,"listed":false,"github":"https://github.com/Kylejeong2/mcpvals","clone":"git clone https://github.com/Kylejeong2/mcpvals.git","description":"An MCP Evaluation Library","language":"TypeScript","stars":45,"topics":["evals","mcp"],"license":"Apache-2.0","category":"mcp-servers","readme_excerpt":"MCPVals A comprehensive evaluation library for Model Context Protocol (MCP) servers. Test and validate your MCP servers with complete MCP specification coverage including Tools with deterministic metrics, security validation, and optional LLM-based evaluation. Status : MVP – API stable , minor breaking changes possible before 1.0.0 --- 0. Quick Start 1. Installation 2. Create a config file Create a config file (e.g., mcp-eval.config.ts ): 3. Run Evaluation --- 1. Core Concepts MCPVals provides comprehensive testing for all MCP specification primitives: 1. Tool Health Testing : Directly calls individual tools with specific arguments to verify their correctness, performance, and error handling. This is ideal for unit testing and regression checking. 2. Workflow Evaluation : Uses a large language model (LLM) to interpret natural language prompts and execute a series of tool calls to achieve a goal. This tests the integration of your MCP primitives from an LLM's perspective. --- 2. Installation & Runtime Requirements 1. Node.js ≥ 18 – we rely on native fetch , EventSource , and fs/promises . 2. pnpm / npm / yarn – whichever you prefer, MCPVals is published as an ESM‐only package. 3. MCP Server – a local stdio binary or a remote Streaming-HTTP endpoint. 4. Anthropic API Key – Required for workflow execution (uses Claude to drive tool calls). Set via ANTHROPIC API KEY environment variable. 5. (Optional) OpenAI key – Only required if using the LLM judge feature. Set via OPENAI API K","default_branch":null,"files":null,"tree":[],"storefront":"/r/Kylejeong2","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Kylejeong2/mcpvals/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}