{"repo":"neilthomas89440-crypto/inferna-next","free":true,"listed":false,"github":"https://github.com/neilthomas89440-crypto/inferna-next","clone":"git clone https://github.com/neilthomas89440-crypto/inferna-next.git","description":"Self-hosted GPU cluster orchestration: deploy and serve AI models on your own hardware","language":"Python","stars":30,"topics":[],"license":"Apache-2.0","category":"self-hosted-apps","readme_excerpt":"Inferna Next Self-hosted GPU cluster orchestration: deploy and serve open-source AI models on your own hardware with one click — a \"Model-as-a-Service\" for machines you control. Inferna Next is a control plane for GPU clusters. A central server exposes a web UI and REST/gRPC APIs; lightweight worker agents run on each GPU host and manage inference engine containers (vLLM, SGLang). Deploy a model from a curated catalog, and the platform picks the worker, the GPU, the engine, and the tuning profile (low latency or high throughput) for you. Everything runs in Docker. Features - Central web UI to manage multiple GPU clusters — on-prem or cloud - Model catalog with one-click deploy (Qwen, Llama, Phi, DeepSeek, BGE, Whisper, Qwen2.5-VL) - Automatic engine selection — vLLM and SGLang, tuned for latency or throughput profiles - Multi-vendor GPU support — NVIDIA (NVML), AMD (ROCm), plus a mock mode for demos - Enterprise basics — user auth (JWT), role-based access control, monitoring dashboards (Prometheus + Grafana) - Docker-first — single docker compose command to run the whole stack Architecture - Server — FastAPI control plane: auth/RBAC, clusters, workers, model catalog, instance lifecycle, scheduling, Prometheus metrics. Also hosts the gRPC registration/sync service. - Worker — polls the server every 5 s, reports GPU/system state and instance health, and reconciles local engine containers (pull → create → start → health-probe → stop/remove). - Sync protocol — unary gRPC polling ","default_branch":null,"files":null,"tree":[],"storefront":"/r/neilthomas89440-crypto","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/neilthomas89440-crypto/inferna-next/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}