{"repo":"edonadei/caliper","free":true,"listed":false,"github":"https://github.com/edonadei/caliper","clone":"git clone https://github.com/edonadei/caliper.git","description":"Know if your agent skill actually works. A lightweight evaluation harness that tracks a success rate across Claude Code, Codex, Pi, and Hermes.","language":"Python","stars":39,"topics":["ai-agents","claude-code","cli","codex","evals","skills","pi","evaluation","reliability","hermes"],"license":"MIT","category":"ai-agents","readme_excerpt":"Caliper: Know if your agent skill actually works Caliper is a lightweight evaluation harness for agent skills. Write a short spec of what \"good\" looks like, run it, and get a success rate you can track. Works with the agent you already use: Claude Code, Codex, Pi, or Hermes . Caliper installs the skill where the agent looks for skills and lets the agent choose. Teach your agent to evaluate: Or run it yourself: You write a spec, a YAML file describing what \"working\" means. Either hand-write it or have /grill-skill generate it for you. --ablate runs the same tasks with that skill removed , and caliper compare diffs the two runs task by task: --- Agent skills are hard to test. A skill that works on your machine, on this prompt, today, might fail tomorrow after a model update or a one-line prompt edit. Caliper makes reliability measurable: define what success looks like, run the skill repeatedly, and get a success rate you can track over time. Use Caliper to answer questions like: - Is my agent still working the same with this new model? - Did my prompt edit improved the skill? - Does my skill fire when it should, and stay quiet when it needs to not trigger? - Is the skill worth the context? Or would the base agent pass without it? - Does it still pass the workflows it passed last week? - Which agent (Claude Code, Codex, Pi, or Hermes) runs this skill more reliably? --- Quick start Path A: Agentic (let your agent drive) 1. Install the skills 2. Generate a spec interactively In yo","default_branch":null,"files":null,"tree":[],"storefront":"/r/edonadei","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/edonadei/caliper/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}