{"repo":"croit/llm-gateway","free":true,"listed":false,"github":"https://github.com/croit/llm-gateway","clone":"git clone https://github.com/croit/llm-gateway.git","description":"Self-hosted, OpenAI-compatible LLM gateway that turns any backend into an agent. Point an OpenAI SDK at it and the model runs tools mid-completion. From web search, code sandbox, document rendering, RAG, MCP connectors all behind a built-in chat UI. Multi-backend routing with health checks + failover, OIDC + RBAC, per-user tokens.","language":"Rust","stars":16,"topics":["ai-agents","chat-ai","image-ai","llm","llm-gateway","mcp","oidc","openai-api","openai-api-chatbot","rag"],"license":"AGPL-3.0","category":"ai-agents","readme_excerpt":"LLM Gateway One endpoint for all your LLM backends — that also makes them agentic. LLM Gateway is an OpenAI-API-compatible reverse proxy: point any OpenAI SDK at it and it routes across your self-hosted and cloud models (health checks, failover, stable aliases), then runs tools mid-completion — web search, code sandbox, document rendering, RAG, per-user MCP connectors — so plain clients get tool use with zero agent code of their own. Team-ready with OIDC login, per-user tokens, and RBAC, plus a built-in chat UI for people who don't speak curl. Ships as a single self-hosted binary (Rust, SQLite) — no compose file, no vector DB, no separate frontend. Contents - What it does - Tools the model can call - The built-in web UI - Scheduled actions - Conversation compaction - Voice conversation - Integrations (per-user MCP connectors) - Quick start (local development) - Configuration - Chat attachments (S3) - RAG (codebase search) - Agent Skills - Using the gateway - HTTP endpoints - Production deployment (container + systemd) - Docker Compose - Documentation - Contributing - License What it does - OpenAI-compatible API — POST /v1/chat/completions (streaming + non-streaming), POST /v1/embeddings , POST /v1/images/generations + POST /v1/images/edits , POST /v1/audio/transcriptions , POST /v1/audio/speech (text-to-speech, when a speech pool is configured), and GET /v1/models . Point any OpenAI SDK at it. - Multi-backend routing — named upstream pools ( chat / transcription / embedding /","default_branch":null,"files":null,"tree":[],"storefront":"/r/croit","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/croit/llm-gateway/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}