{"repo":"Northwood-Systems/millwright","free":true,"listed":false,"github":"https://github.com/Northwood-Systems/millwright","clone":"git clone https://github.com/Northwood-Systems/millwright.git","description":"Self-hosted LLM router. Cost effective, deterministic, and fast. Secure and private by default.","language":"Rust","stars":45,"topics":["ai-gateway","bedrock","gateway","litellm","llm","llm-gateway","llmops","openai","openai-proxy","rust"],"license":"Apache-2.0","category":"ai-agents","readme_excerpt":"Millwright The self-hosted LLM router for policy, cache affinity, and spend control in one Rust Binary. What is Millwright Millwright is an open-source, self hosted LLM router that runs between your AI applications and the model providers you choose. It accepts OpenAI Chat Completions and Anthropic Messages, then routes each request to an OpenAI-compatible API, Anthropic, or Amazon Bedrock. You define which models belong in the cheap , mid , and frontier roles. Millwright selects the lowest estimated-cost healthy route the resolved role permits and, when given a session ID, preserves provider/model affinity for prompt-cache reuse. Millwright is a router, not an agent orchestrator. It does not spawn agents, schedule work, inspect prompts, or rewrite context. Why Millwright LLM costs rise quickly when every request reaches the most expensive model and provider changes break prompt-cache reuse. Millwright makes the lower-cost path explicit: - Control model spend — reserve frontier models for the work your policy says needs them. - Protect prompt caches — role-scoped affinity keeps each session on a confirmed provider and model while its cache window is warm. - Keep concurrent agents moving — one session can maintain independent cheap , mid , and frontier lanes without serializing requests. - Change providers without changing application code — route compatible traffic across providers behind one endpoint. - Keep routing inspectable — response headers identify the chosen route; t","default_branch":null,"files":null,"tree":[],"storefront":"/r/Northwood-Systems","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/Northwood-Systems/millwright/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}