{"repo":"paradigmxyz/evmbench","free":true,"listed":false,"github":"https://github.com/paradigmxyz/evmbench","clone":"git clone https://github.com/paradigmxyz/evmbench.git","description":"Collab with OpenAI. A benchmark and harness for finding and exploiting smart contract bugs","language":"TypeScript","stars":448,"topics":["agents","ai","audit","blockchain","eth","ethereum","evm","security","solidity","testing"],"license":"Apache-2.0","category":"blockchain-web3","readme_excerpt":"evmbench is a benchmark and agent harness for finding and exploiting smart contract bugs. How it works Security Key services Repo layout Quickstart (local dev) This repository contains a companion interface to the evmbench detect evaluation (code). For reference, we include the evaluation code as a pinned submodule at frontier-evals/ . Upload contract source code, select an agent, and receive a structured vulnerability report rendered in the UI. How it works Architecture End-to-end flow 1. User uploads a zip of contract files via the frontend. The UI sends the archive, selected model key, and (optionally) an OpenAI API key to /v1/jobs/start . 2. The backend creates a job record in Postgres, stores a secret bundle in the Secrets Service, and publishes a message to RabbitMQ. 3. The Instancer consumes the job and starts a worker (Docker locally; Kubernetes backend is optional). 4. The worker fetches its bundle from the Secrets Service, unpacks the uploaded zip to audit/ , then runs Codex in \"detect-only\" mode: - prompt: backend/worker runner/detect.md (copied to $HOME/AGENTS.md inside the container) - model map: backend/worker runner/model map.json (maps UI model keys to Codex model IDs) - command wrapper: backend/worker runner/run codex detect.sh 5. The agent writes submission/audit.md . The worker validates that the output contains parseable JSON with {\"vulnerabilities\": [...]} and then uploads it to the Results Service. 6. The frontend polls job status and renders the report ","default_branch":null,"files":null,"tree":[],"storefront":"/r/paradigmxyz","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/paradigmxyz/evmbench/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}