{"repo":"msmarkgu/RelayFreeLLM","free":true,"listed":false,"github":"https://github.com/msmarkgu/RelayFreeLLM","clone":"git clone https://github.com/msmarkgu/RelayFreeLLM.git","description":"A restful API designed to route user prompts to various AI model providers.","language":"Python","stars":178,"topics":["api-gateway","free","llm","python","rate-limit","api-aggregator","fastapi","openai","quota-tracking","auto-failover"],"license":"MIT","category":"ai-agents","readme_excerpt":"RelayFreeLLM One endpoint. More free AI than any single provider. Less rate limit headaches. Don't want to pay $$/month to use AI Models? RelayFreeLLM is an open-source gateway that combines multiple free-tier providers into a single OpenAI-compatible API — so you get aggregately more free inference with automatic failover. Gemini · Groq · Mistral · DeepSeek · NVIDIA · Cerebras · Cloudflare · Ollama No code changes. No retry logic. No 429 errors breaking your app. --- The Free Tier Problem → The RelayFreeLLM Solution --- What You Get Feature Why It Matters --------- ---------------- OpenAI-compatible Drop-in for your existing code. LangChain, LlamaIndex, any SDK. Automatic Failover Provider down? One model hit limits? We try the next one automatically. Zero downtime. Session Affinity Pin conversations to a provider via X-Session-ID for context caching benefits. 4-Mode Context Management Static, Dynamic, Reservoir, Adaptive — with extractive summarization to preserve long conversations. Consistent Output Style Universal style guidance + response normalizers eliminate provider-specific quirks. Opt out per-request via X-Use-ServerSide-System-Prompt: false . Intent-Based Routing model type=coding , model scale=large , model name=deepseek — tell us what you need, not which API to call. [NEW] Image-aware Routing Upload images in chat; the dispatcher automatically routes to vision-capable models. Explicit text-only model pick? Falls back within the same provider, then any provider. ","default_branch":null,"files":null,"tree":[],"storefront":"/r/msmarkgu","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/msmarkgu/RelayFreeLLM/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}