{"repo":"sshh12/agent-pr-replay","free":true,"listed":false,"github":"https://github.com/sshh12/agent-pr-replay","clone":"git clone https://github.com/sshh12/agent-pr-replay.git","description":"Agent PR Replay takes merged PRs from any repository, reverse-engineers the task prompt, runs Claude Code against it, and compares what the agent did versus what humans actually shipped. The result is targeted, empirical guidance.","language":"Python","stars":32,"topics":["claude","claude-code","coding-agents","context-engineering","skills"],"license":"MIT","category":"ai-agents","readme_excerpt":"Agent PR Replay Discover how AI coding agents actually navigate your codebase, and where they diverge from human developers. The best way to improve an agent's ability to work in a codebase is to observe its default behavior, measure the gap against real human solutions, and steer it based on evidence. Agent PR Replay takes merged PRs from any repository, reverse-engineers the task prompt, runs Claude Code against it, and compares what the agent did versus what humans actually shipped. The result is targeted, empirical guidance. See example reports → How It Works 1. Ground Truth : Start with real merged PRs as human-validated solutions 2. Replay Setup : Checkout the repo at the PR's base commit, reverse-engineer a task prompt from the diff 3. Run Agent : Execute Claude Code with the prompt (same starting point, same task, different solver) 4. Compare : Diff what Claude changed vs what humans shipped; identify systematic gaps 5. Synthesize : Generate targeted guidance (CLAUDE.md, AGENTS.md, skills) based on observed behavioral patterns Installation Prerequisites: - Python 3.11+ - GitHub CLI ( gh ), authenticated via gh auth login - Claude Code CLI ( claude ), installed and authenticated Warning : This tool uses the Claude API and will incur costs ( $4 per --top-k value). It runs code in git worktrees. Only use with trusted repositories or within sandboxed environments. Or with uv: Development setup Usage Run Analysis View Statistics Generate Report Synthesize findings into a s","default_branch":null,"files":null,"tree":[],"storefront":"/r/sshh12","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/sshh12/agent-pr-replay/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}