{"repo":"Jwuthri/Tracely","free":true,"listed":false,"github":"https://github.com/Jwuthri/Tracely","clone":"git clone https://github.com/Jwuthri/Tracely.git","description":"Trace-native CI/CD for AI agents — production failures become regression tests that block the PR. Auto-detect, cluster, freeze into hermetic cases, replay in CI for $0.","language":"Python","stars":643,"topics":["ai-agents","ci-cd","evals","llm-evaluation","llm-observability","llm-ops","agent","agent-observability","clickhouse","evaluation"],"license":"MIT","category":"deployment-docker-iac","readme_excerpt":"Tracely Production failures become regression tests. Trace-native CI/CD for AI agents. Tracely grades every agent trace as it lands, clusters the failures into issues, freezes the bad runs into hermetic replayable cases — and blocks the pull request that would ship them again. Website · Docs · Agent skill · Guided tour · 2-min demo · Design dossier Self-host the whole stack in one click — API, worker, UI, Postgres, ClickHouse, Redis and MinIO: --- Why another agent-observability tool? Because observability stops at the dashboard. You can see that your agent broke — then what? Every eval tool asks you to hand-author a dataset : sit down, invent questions, write ideal answers, keep them current as the product changes. That dataset is a guess about what might break. Production already handed you the real thing: a trace of the exact run that failed, with the exact input, the exact tool calls, the exact model responses. The recorded run is the test. Tracely freezes that trace into a hermetic regression case and replays it on every PR. Everything else — quality scores, failure clusters, suggested fixes, CI verdicts, trends — is derived from the trace . There are no hand-authored datasets. Dataset-first tools Tracely --- --- --- Where tests come from You write them Promoted from real failing traces Fidelity to production A guess The exact failing run, byte for byte Cost to replay in CI Live model calls $0 — recorded tool/LLM fixtures What happens on regression A dashboard number mov","default_branch":null,"files":null,"tree":[],"storefront":"/r/Jwuthri","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Jwuthri/Tracely/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}