{"repo":"GuglielmoCerri/khazad","free":true,"listed":false,"github":"https://github.com/GuglielmoCerri/khazad","clone":"git clone https://github.com/GuglielmoCerri/khazad.git","description":"Transparent, transport-layer semantic cache for LLM API calls, powered by Redis 8 Vector Sets.","language":"Python","stars":33,"topics":["anthropic","embeddings","gemini","llm","openai","python","redis","semantic-cache","vector-search"],"license":"MIT","category":"ai-agents","readme_excerpt":"Khazad — You shall not pass. Transparent, transport-layer semantic cache for LLM API calls powered by Redis Vector Sets. 50% fewer API calls · 96% faster on hits · 50% lower spend · 100% transparent Illustrative figures at a 0.50 hit rate (280ms cache replay vs. 7900ms upstream call). Your numbers depend on traffic. Khazad intercepts LLM HTTP traffic at the transport layer and serves semantically equivalent requests from a Redis vector cache, with zero changes to your application code . How it works Key properties: - Model-aware — each (provider, model) pair gets its own vector set, so a gpt-4o answer is never served to a gpt-4o-mini call, no matter how similar the prompt. Set cache scope=\"host\" to scope by provider host only , letting every model or deployment on the same provider share one cache (different providers stay isolated — see Configuration). - Conversation-aware — the whole message list (system, user, assistant) is embedded, not just the last user turn. Two different conversations ending with the same follow-up question (\"What about its population?\") never collide. - Streaming both ways — cache hits replay as real SSE streams (sync and async clients); cache misses that stream are captured chunk-by-chunk with no added latency and reassembled into a canonical JSON response, so a streamed answer can later serve a non-streamed request and vice versa. Aborted streams are never cached. Why use it Semantic caching trades exactness for cost and latency. Know the trade bef","default_branch":null,"files":null,"tree":[],"storefront":"/r/GuglielmoCerri","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/GuglielmoCerri/khazad/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}