{"repo":"litmux4ai/litmux","free":true,"listed":false,"github":"https://github.com/litmux4ai/litmux","clone":"git clone https://github.com/litmux4ai/litmux.git","description":"⚡ Unit tests for AI. Test prompts, compare models, save money.","language":"Python","stars":105,"topics":["ai","cli","cost-optimization","developer-tools","evaluation","llm","open-source","prompt-engineering","python","testing"],"license":"MIT","category":"dev-tools","readme_excerpt":"Litmux Unit tests for AI. Test prompts, compare models, catch regressions. --- Why Every team shipping AI features hits the same three problems: 1. No testing standard. REST has Postman, frontends have Cypress. LLM calls have manual spot-checking. 2. Prompt regression is invisible. A one-word change can silently break 15% of edge cases. 3. Model selection is vibes. \"We use GPT-4o because it's good\" — but is it $15k/month better than Gemini Flash? Litmux gives you a YAML config, pass/fail assertions, and a cost report. That's it. --- Quick Start No database, no cloud account, no Docker. --- Core Commands litmux run — unit tests for prompts litmux eval — bulk evaluation against datasets litmux generate — AI-generated test datasets litmux cost — cost projection across models Finds the cheapest model that passes your tests. litmux compare — side-by-side model outputs --- Cloud (Optional, Free) Sync results to a hosted dashboard for history, trends, and team visibility. The CLI works fully offline. Cloud is opt-in. --- Assertion Types Type Description ------ ------------- contains Output contains substring not-contains Output does not contain substring regex Output matches regex pattern json-valid Output is valid JSON json-schema Output has required JSON keys cost-less-than Cost below threshold (USD) latency-less-than Latency below threshold (ms) llm-judge LLM scores output 1–10 against criteria --- CI/CD --- Configuration Environment Variables Variable Purpose ---------- --------","default_branch":null,"files":null,"tree":[],"storefront":"/r/litmux4ai","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/litmux4ai/litmux/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}