{"repo":"adepeju4/attest","free":true,"listed":false,"github":"https://github.com/adepeju4/attest","clone":"git clone https://github.com/adepeju4/attest.git","description":"Evidence-grounded evaluation for AI agents — verifies each claim against the agent's real tool outputs (constrained, evidence-grounded model judgment, not holistic LLM-judge guesswork), with confidence intervals.","language":"Python","stars":17,"topics":["agent-evaluation","ai-agents","anthropic","evals","evaluation","gemini","llm","llmops","observability","openai"],"license":"MIT","category":"ai-agents","readme_excerpt":"attest A reality-checker for AI agents. It grades an agent's answer against what its tools actually returned — so it catches made-up facts, misused tools, and security slips that a \"does this look good?\" check waves through. The problem The more common ways grading an AI agent ask another AI: \"is this answer good?\" That's easy to fool. A confident, well-written answer can sail through even when a specific detail buried inside it is wrong — because the grader is reacting to the story , not checking the facts . Research bears this out: just rewriting an agent's reasoning — while leaving what it actually did unchanged — can push an AI judge's false-positive rate up by as much as 90% ( Gaming the Judge , Khalifa et al., 2026). attest takes the opposite approach: never trust what the agent says it did — check it against the receipts. Every tool the agent used produced a real output. attest treats those outputs as the source of truth and verifies the answer against them. What it checks attest looks at a run — a record of one task: what the user asked, which tools the agent called, what those tools returned, and the final answer. It then answers four plain questions: - Did it make things up? It breaks the answer into individual statements and checks each one against the real tool outputs. If the tools say Berlin is bigger than Paris but the answer says the opposite, that statement gets flagged — with the exact line of evidence that proves it wrong. - Did it use its tools properly? D","default_branch":null,"files":null,"tree":[],"storefront":"/r/adepeju4","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/adepeju4/attest/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}