{"repo":"genesiscode2026/genesis-agent-skill-eval","private":true,"description":null,"default_branch":"main","stars":0,"forks":0,"pushed_at":"2026-09-17T10:23:19Z","languages":{"Python":11285},"files":16,"tree":[".gitignore","CHANGELOG.md","LICENSE","QUICKSTART.md","README.md","SECURITY.md","VERSION","bin/genesis-skill-eval","genesis_skill_eval/__init__.py","genesis_skill_eval/__main__.py","genesis_skill_eval/evaluator.py","pyproject.toml","tests/fixtures/sample_skill/SKILL.md","tests/fixtures/sample_skill/calc.py","tests/fixtures/scenarios.json","tests/test_evaluator.py"],"sizes":{".gitignore":103,"CHANGELOG.md":664,"LICENSE":1072,"QUICKSTART.md":424,"README.md":2341,"SECURITY.md":273,"VERSION":6,"bin/genesis-skill-eval":212,"genesis_skill_eval/__init__.py":190,"genesis_skill_eval/__main__.py":2411,"genesis_skill_eval/evaluator.py":6679,"pyproject.toml":640,"tests/fixtures/sample_skill/SKILL.md":164,"tests/fixtures/sample_skill/calc.py":344,"tests/fixtures/scenarios.json":538,"tests/test_evaluator.py":1793},"readme":"# genesis-agent-skill-eval\n\n> **Deterministic Agent Skill evaluation & benchmarking harness with schema and runtime assertions.**\n\n[![Version](https://img.shields.io/badge/version-1.0.0-blue.svg)](VERSION)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n[![Zero Dependencies](https://img.shields.io/badge/dependencies-0-brightgreen.svg)](pyproject.toml)\n\n---\n\n## The Problem\n\nAs Agent Skills ecosystems expand across Claude Code, Gemini, and Antigravity, developers lack standardized tooling to benchmark and test their skills before distribution:\n- Skills often ship with broken frontmatter or malformed metadata.\n- Automation scripts fail silently or hang indefinitely without timeouts.\n- Edge-case arguments cause non-deterministic crashes that break autonomous agent runs.\n\n## The Solution\n\n`genesis-agent-skill-eval` provides a deterministic testing and benchmarking harness for Agent Skills. It validates `SKILL.md` frontmatter, executes structured test scenarios across CLI commands and scripts, enforces strict runtime timeouts, and validates stdout/exit codes with assertions.\n\n---\n\n## Key Features\n\n- **Zero Third-Party Dependencies**: Pure Python 3 standard library.\n- **Specification Conformance**: Validates `SKILL.md` frontmatter, required metadata keys, and instruction markdown.\n- **Scenario Assertion Engine**: Tests command exit codes, substring containment, regex matching, and valid JSON outputs.\n- **Runtime Timeout Protection**: Halts hanging tasks automatically after configurable execution timeouts.\n- **Performance Benchmarking**: Measures millisecond execution latency and overall scenario pass rates.\n\n---\n\n## Quickstart\n\n```bash\n# Clone the repository\ngit clone git@github.com:genesiscode2026/genesis-agent-skill-eval.git\ncd genesis-agent-skill-eval\n\n# Benchmark an Agent Skill against a scenarios suite\n./bin/genesis-skill-eval \\\n  --skill /path/to/my-skill \\\n  --scenarios ./scenarios.json\n\n# Output machine-readable JSON report for CI\n./bin/genesis-skill-eval --skill ./my-skill --scenarios ./tests.json --json\n```\n\n---\n\n## Commercial Distribution\n\n- **Product Name**: `genesis-agent-skill-eval`\n- **Catalog ID**: `P16`\n- **Price**: $59\n- **Seller**: `genesiscode2026`\n- **Authorized Payout Rail**: USDC on Base (`0xC6F86e170411182114FcCdb28793dC76B5e8D144`)\n- **License**: MIT\n","price_usd":"59.00","clone_checkout":"https://api.checkout402.com/c/chk_LWLUHt5XpgsL31QA","rehearse_checkout":"https://api.checkout402.com/c/chk_LWLUHt5XpgsL31QA/test","preview":"/r/genesiscode2026/genesis-agent-skill-eval/f/<path> serves file CONTENTS for the seller's preview paths; other paths answer with the price","how_to_buy":"pay clone_checkout with x402 (pay402 does it in one call); the goods arrive as payload_base64, a tar.gz of the full repo. Rehearse first at rehearse_checkout: same flow, simulated money.","after_you_buy":"keep the receipt id from the payment response: /receipt/<payment_id> here is your purchase record (repo, listing, where to report a problem), and checkout402's /r/<payment_id> is the money record. Tell your human both URLs."}