{"repo":"pinecone-io/cultivar","free":true,"listed":false,"github":"https://github.com/pinecone-io/cultivar","clone":"git clone https://github.com/pinecone-io/cultivar.git","description":"Use cultivar to test your Agent Skills, run them in sandboxes, and across different agents.","language":"Python","stars":37,"topics":["agent","agent-skills","benchmarking","claude-code","codex","copilot","cursor","evals","gemini-cli","pinecone"],"license":"MIT","category":"cli-tools","readme_excerpt":"cultivar A CLI tool to help you write tests for skills, test them across agents, and iterate until they work, from the Pinecone DevRel team. Test how well skills work against tasks, across agents, locally and remotely. Customize sandboxes for how agents should start, and graders for how agents should work. Use traces to iteratively refine skills and optimize them against tasks. Benchmark against skills, docs, and baselines. And, even run in parallel simulatenously for faster execution. Two ways to run it: the cultivar CLI directly, or install the bundled skill and let your coding agent drive it: Same engine either way — but when an agent drives it you won't watch the live run / --remote sandbox dashboard as directly as in your own terminal. Keep the skill project-scoped (not -g ); it's never auto-tested. Prerequisites - Python 3.11+ and uv - An Anthropic API key (the grader runs locally; agents in the sandbox use Modal-injected keys) - A Modal account if you want --remote runs (recommended for parallelism + isolation). This is the recommended experience! Install Modal setup (one-time) --remote runs each eval in an isolated Modal sandbox — recommended for parallelism and clean auth state. Skip this section if you only need local runs. The first --remote run builds the sandbox image ( 3–5 min). Subsequent runs use the cached image ( 5–10 s cold start). Defaults you can override via env var : Env var Default What it controls --- --- --- CULTIVAR MODAL SECRET eval-sandbox-secrets","default_branch":null,"files":null,"tree":[],"storefront":"/r/pinecone-io","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/pinecone-io/cultivar/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}