{"repo":"darkrishabh/agent-skills-eval","free":true,"listed":false,"github":"https://github.com/darkrishabh/agent-skills-eval","clone":"git clone https://github.com/darkrishabh/agent-skills-eval.git","description":"A test runner for agentskills.io-style AI agent skills","language":"TypeScript","stars":684,"topics":["agent-evals","agent-skills","agentskills","ai-agents","cli","jsonl","llm-evals","llm-evaluation","openai-compatible","typescript"],"license":"MIT","category":"ai-agents","readme_excerpt":"agent-skills-eval A test runner for Agent Skills. Write a SKILL.md , drop in some evals, and find out — empirically — whether your skill actually makes the model better at the task. Documentation · Quickstart · SDK · agentskills.io --- Why this exists Agent Skills — the open standard from Anthropic for giving agents domain knowledge — make it easy to ship a SKILL.md and assume your agent is now better at the task. The hard part is proving it. agent-skills-eval is the missing piece. It runs your skill against the same prompts twice — once with skill loaded into context, once without skill (baseline) — has a judge model grade both outputs, and gives you a side-by-side report. If the skill doesn't make a measurable difference, you'll see it. If it does, you have receipts. It's the test framework for the Agent Skills ecosystem, separated from any specific agent runtime so it works wherever your skills do. Quickstart That's it. Point it at a folder of skills, give it a target model and a judge model, and it produces a workspace with full artifacts and a static HTML report. Open iteration-1/report/index.html and you have a real, evidence-backed answer to \"is my skill working?\" What you get --- --- with skill vs without skill Every eval runs both ways so you can see the actual lift from the skill — or its absence. Judge-graded outputs Use any chat model as a judge. Pass/fail with cited assertions, not vibes. TypeScript SDK + CLI One-liner CLI for CI, full SDK for custom pipelines, c","default_branch":null,"files":null,"tree":[],"storefront":"/r/darkrishabh","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/darkrishabh/agent-skills-eval/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}