{"repo":"traceloop/hub","free":true,"listed":false,"github":"https://github.com/traceloop/hub","clone":"git clone https://github.com/traceloop/hub.git","description":"High-scale LLM gateway, written in Rust. OpenTelemetry-based observability included","language":"Rust","stars":225,"topics":["artificial-intelligence","datascience","generative-ai","llm","llmops","ml","model-monitoring","observability","open-source","opentelemetry"],"license":"Apache-2.0","category":"ai-agents","readme_excerpt":"Open-source, high-performance LLM gateway written in Rust. Connect to any LLM provider with a single API. Observability Included. Get started » Slack Docs Traceloop Hub is a next-gen high-performance LLM gateway written in Rust that centralizes control and tracing of all LLM calls. It provides a unified OpenAI-compatible API for connecting to multiple LLM providers with observability built-in. Features - Multi-Provider Support : OpenAI, Anthropic, Azure OpenAI, Google VertexAI, AWS Bedrock - OpenAI Compatible API : Drop-in replacement for OpenAI API calls - Two Deployment Modes : - YAML Mode : Simple static configuration with config files - Database Mode : Dynamic configuration with PostgreSQL and Management API - Built-in Observability : OpenTelemetry tracing and Prometheus metrics - High Performance : Written in Rust with async/await support - Hot Reload : Dynamic configuration updates (Database mode) - Pipeline System : Extensible request/response processing - Unified Architecture : Single crate structure with integrated Management API Quick Start Using Docker Using Cargo Architecture The project uses a unified single-crate architecture: Configuration Modes YAML Mode Perfect for simple deployments and development environments. Features: - Static configuration via config.yaml - No external dependencies - Simple provider and model setup - No management API - Single port (3000) Example config.yaml: Database Mode Ideal for production environments requiring dynamic configuratio","default_branch":null,"files":null,"tree":[],"storefront":"/r/traceloop","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/traceloop/hub/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}