{"repo":"JawherKl/llm-api-gateway","free":true,"listed":false,"github":"https://github.com/JawherKl/llm-api-gateway","clone":"git clone https://github.com/JawherKl/llm-api-gateway.git","description":"Scalable API gateway that aggregates calls to multiple LLMs (OpenAI, Hugging Face, Groq, Anthropic, Gemini, etc.), includes caching, rate limiting, logging, monitoring and production-ready deployment.","language":"Go","stars":23,"topics":["anthropic","caching","docker","gemini","golang","groq","huggingface","k8s","llm","openai"],"license":null,"category":"ai-agents","readme_excerpt":"🚀 Unified LLM API Gateway --- ✨ Overview Unified LLM API Gateway is a scalable, extensible platform that aggregates and normalises calls to multiple LLM backends (OpenAI, Hugging Face, Groq, Anthropic, Gemini, and more). It provides a unified API with built-in caching, rate limiting, authentication, logging, metrics, and production-ready deployment manifests for Docker and Kubernetes. --- --- 🏗️ Architecture - API Gateway (Go): - Accepts client requests - Handles authentication, routing and request transformation - Aggregates/fans-out to LLM backends - LLM Adapters (microservices): - Wrap each provider’s API (OpenAI, Hugging Face, etc.) with a unified internal interface - Cache Layer: - Redis for result caching (prompt+params as cache key) - Rate Limiter: - Redis-based leaky-bucket or token-bucket (shared across instances) - Auth & Quotas: - API keys / JWT, per-key quotas (Redis or DB) - Observability: - Structured logs (JSON), Prometheus metrics, traces - Deployment: - Docker images, Helm charts, Kubernetes manifests, CI builds --- 📁 Monorepo Layout --- ⚡ Quickstart 1. Start all services: 2. Gateway API: http://localhost:3020/gateway/query 3. Admin Dashboard: http://localhost:3040 --- 🔌 Supported Providers - [x] OpenAI (GPT-3.5, GPT-4, GPT-4o, etc.) - [x] Hugging Face Inference API - [x] Groq - [x] OpenRouter - [x] Anthropic (Claude) - [x] Gemini (Google) - [ ] More coming soon! --- 🛡️ Features - Unified API: One endpoint for all LLMs - Authentication: API key/JWT middl","default_branch":null,"files":null,"tree":[],"storefront":"/r/JawherKl","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/JawherKl/llm-api-gateway/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}