{"repo":"greynewell/infermux","free":true,"listed":false,"github":"https://github.com/greynewell/infermux","clone":"git clone https://github.com/greynewell/infermux.git","description":"Route inference across providers.","language":"Go","stars":95,"topics":["golang","mist-stack","ai-infrastructure","cost-tracking","model-routing","ai-gateway","anthropic","api-gateway","inference","inference-routing"],"license":"MIT","category":"machine-learning","readme_excerpt":"infermux Inference router. Part of the MIST stack. Install Provider interface Route Tracks tokens and cost per request. Reports spans to TokenTrace. HTTP API gRPC API InferMux serves the same router over gRPC ( infermux.v1.InferMuxService ), defined in proto/infermux/v1/infermux.proto . The server ships with the standard gRPC health service, server reflection (works with grpcurl out of the box), keepalive enforcement, panic recovery, structured per-RPC logging, and graceful drain on SIGINT/SIGTERM. Go client: The client retries UNAVAILABLE (transient provider failure) up to 3 attempts with exponential backoff via gRPC service config, and never retries NOT FOUND or INVALID ARGUMENT . Caller deadlines propagate through the server into provider calls. Error contract: Condition gRPC code --- --- Empty messages, bad role, temperature out of range INVALID ARGUMENT No provider for the requested model NOT FOUND Resolved provider failed upstream (retryable) UNAVAILABLE Caller deadline elapsed DEADLINE EXCEEDED Integration tests cover the full wire path (real TCP, real server, real client), including retry behavior, deadline propagation, error mapping, and health checks: Regenerate protobuf stubs: CLI","default_branch":null,"files":null,"tree":[],"storefront":"/r/greynewell","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/greynewell/infermux/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}