{"repo":"sodiumsun/snackcache","free":true,"listed":false,"github":"https://github.com/sodiumsun/snackcache","clone":"git clone https://github.com/sodiumsun/snackcache.git","description":"Drop-in caching proxy for OpenAI and Anthropic APIs - stop paying for the same answer twice!","language":"Python","stars":43,"topics":["anthropic","cache","llm","openai","proxy","python"],"license":"MIT","category":"ai-agents","readme_excerpt":"SnackCache Semantic caching proxy for OpenAI and Anthropic APIs. Stop paying for the same answer twice. Results • How It Works • Installation • Usage • Roadmap --- SnackCache is a caching layer for OpenAI and Anthropic APIs. It reduces your LLM API costs by caching responses and returning them for similar queries. Works with existing SDKs. No code changes beyond the base URL. The insight: Most developers use similar patterns - common system prompts, standard instructions, popular use cases. When you get a cache hit, it might be from: - Your own previous request - Another developer with the same normalized prompt - A pre-seeded common pattern With SnackCache, you don't need to pay for those tokens twice. --- Results In our testing with real development workflows: Metric Without SnackCache With SnackCache -------- ------------------- ----------------- API calls 1,000 312 Tokens used 847,000 264,000 Cost (GPT-4o) $12.70 $3.96 Avg latency 1.2s 0.08s 68% fewer API calls. 69% cost reduction. 15x faster on cache hits. Cache hits return in 80ms (local embedding lookup) vs 1-2 seconds for API calls. Why semantic matching matters Exact-match caching only helps with identical prompts. Real usage has variations: Semantic matching increased our cache hit rate from 23% to 68% in testing. --- How It Works Two-layer cache: 1. Exact match - Hash lookup for identical prompts (instant) 2. Semantic match - Embedding similarity for similar prompts Uses all-MiniLM-L6-v2 embeddings ( 23M params, ru","default_branch":null,"files":null,"tree":[],"storefront":"/r/sodiumsun","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/sodiumsun/snackcache/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}