{"repo":"ParisNeo/lollms_hub","free":true,"listed":false,"github":"https://github.com/ParisNeo/lollms_hub","clone":"git clone https://github.com/ParisNeo/lollms_hub.git","description":"A proxy server for multiple ollama instances with Key security","language":"Python","stars":642,"topics":["ollama","proxy","security","rate-limiting","api-gateway","user-management","load-balancer","federation","agent","personal"],"license":"Apache-2.0","category":"networking-infra","readme_excerpt":"🧊 LoLLMs Hub Fortress: The Universal AI Gateway LoLLMs Hub is a secure, high-performance orchestration layer that unifies disparate AI backends into a single, managed API. It serves as a fortress for your compute resources, providing enterprise-grade security, intelligent routing, and advanced workflow automation. 🏢 Unified Multi-Backend Architecture LoLLMs Hub is designed for extreme scale and flexibility. It treats different inference engines as swappable compute nodes: Ollama : Native support for local model management and serving. vLLM : High-throughput serving for production-grade LLM clusters. llama.cpp : Lightweight, GGUF-based local serving via the internal Binary Hub. OpenAI-Compatible APIs : Seamlessly proxy and load-balance cloud providers. 🏢 Hierarchical Architecture (The Master Hub Pattern) LoLLMs Hub is designed for extreme scale. Because its API is 100% compatible with Ollama and OpenAI, you can chain multiple Hubs together. Multi-Node Cluster Flow Why use a Slave Hub? 1. Local Management: The Slave Hub handles spawning local processes on the worker machine. 2. Simplified Master: The Master Hub only sees one \"Remote Server\" entry instead of dozens of raw instances. 3. Security: Only the Master needs public access; Slaves can stay on a private VPN/VLAN. --- 🧠 Advanced AI Orchestration: Routers & Ensembles Smart Routers: Hierarchical Model Routing Create virtual \"traffic controllers\" that intelligently route requests to the best model for the job: Routing Str","default_branch":null,"files":null,"tree":[],"storefront":"/r/ParisNeo","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/ParisNeo/lollms_hub/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}