{"repo":"pmady/keda-gpu-scaler","free":true,"listed":false,"github":"https://github.com/pmady/keda-gpu-scaler","clone":"git clone https://github.com/pmady/keda-gpu-scaler.git","description":"KEDA External gRPC Scaler for GPU workloads - native NVML metrics via DaemonSet, no Prometheus required","language":"Go","stars":115,"topics":["autoscaling","daemonset","gpu","grpc","inference","keda","kubernetes","nvidia","nvml","vllm"],"license":"Apache-2.0","category":"deployment-docker-iac","readme_excerpt":"KEDA GPU Scaler Scale Kubernetes GPU workloads from real hardware metrics. No DCGM. No PromQL. Optional Prometheus metrics built in. A KEDA External Scaler that reads NVIDIA GPU metrics directly from NVML C-bindings and autoscales your vLLM, Triton, and custom inference deployments including scale-to-zero. Why This Exists Kubernetes HPA watches CPU and memory. It can't see GPU utilization. Your vLLM pod shows 8% CPU while the GPU is at 100%. The usual fix is dcgm-exporter → Prometheus → KEDA, but that's 5 components and 15-30s of latency. This project reads GPU metrics directly from NVML and serves them to KEDA over gRPC. 2 components, 2-4 second latency. Why Not a Native KEDA Scaler? Putting GPU support inside KEDA core doesn't work: 1. CGO Constraint : NVIDIA's Go bindings ( go-nvml ) require CGO ENABLED=1 . KEDA builds with CGO ENABLED=0 . 2. Node-Level Hardware Access : The KEDA operator runs as a central pod. NVML requires local GPU device access via libnvidia-ml.so , which only a DaemonSet on GPU nodes can provide. 3. Independent Release Cycle : Ship GPU scaling improvements without waiting for KEDA release cycles. This design is documented in KEDA issue #7538. --- Architecture High Level Overview Detailed Data Flow 1. DaemonSet — Runs on nodes labeled with nvidia.com/gpu.present: \"true\" . 2. NVML Bindings — Directly reads Streaming Multiprocessor (SM) utilization and Frame Buffer Memory via go-nvml C-bindings. 3. gRPC Interface — Implements externalscaler.ExternalScale","default_branch":null,"files":null,"tree":[],"storefront":"/r/pmady","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/pmady/keda-gpu-scaler/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}