{"repo":"GaeaRuiW/kube-llmops","free":true,"listed":false,"github":"https://github.com/GaeaRuiW/kube-llmops","clone":"git clone https://github.com/GaeaRuiW/kube-llmops.git","description":null,"language":"Go","stars":101,"topics":["ai","deep-learning","fine-tuning","grafana","helm","k8s","keycloak","kubernetes","litellm","llama-cpp"],"license":"Apache-2.0","category":"deployment-docker-iac","readme_excerpt":"English 中文 🌐 Project Site kube-llmops 🏆 The most comprehensive Kubernetes-native LLMOps Platform — One command to deploy, manage, monitor, and optimize your entire LLM infrastructure. [!NOTE] v1.0.0 released -- the v1.0 trio is complete: Operator (LLMPlatform / ModelDeployment / FineTuneRun CRDs) + kubectl-llmops CLI (15+ commands, kubectl llmops ) + Headlamp Dashboard (Kubernetes UI + kube-llmops-portal plugin). See CHANGELOG for details. What is kube-llmops? kube-llmops is an opinionated, batteries-included Helm chart that deploys a complete LLM operations stack on Kubernetes: - Model Serving -- vLLM, llama.cpp, or TEI, auto-selected based on model format (engine auto-detection from source name) - CLI ( kubectl-llmops ) -- kubectl plugin with 15+ imperative commands: deploy , list , status , scale , canary , logs , test , port-forward , finetune , rag , platform , migrate , ... - AI Gateway -- LiteLLM for unified OpenAI-compatible API, key management, rate limiting, budget control - Observability -- Prometheus + Grafana (11 dashboards + 8 alert rules) + Langfuse v3 LLM tracing + node-exporter + kube-state-metrics - Logging -- Fluent Bit + Loki, queryable in Grafana Explore - Autoscaling -- KEDA scales vLLM pods based on queue depth, TTFT P95, and TPOT P95; supports scale-to-zero with fallback - Security -- Keycloak SSO for Grafana/Langfuse, LLM-Guard prompt injection defense, NetworkPolicy isolation - RAG Infrastructure -- Dify platform + pgvector + TEI embedding/rerankin","default_branch":null,"files":null,"tree":[],"storefront":"/r/GaeaRuiW","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/GaeaRuiW/kube-llmops/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}