{"repo":"InfraWhisperer/llmtop","free":true,"listed":false,"github":"https://github.com/InfraWhisperer/llmtop","clone":"git clone https://github.com/InfraWhisperer/llmtop.git","description":"htop for your LLM inference cluster","language":"Go","stars":20,"topics":["cli","gpu","inference","kubernetes","llm","monitoring","sglang","terminal","tui","vllm"],"license":null,"category":"cli-tools","readme_excerpt":"htop for your LLM inference cluster Real-time terminal dashboard for vLLM , SGLang , LMCache , NVIDIA NIM , and NVIDIA Dynamo inference clusters. --- --- Install Or grab a binary from GitHub Releases, or: Quick Start What It Does - Real-time KV cache, queue depth, TTFT/ITL latency, token throughput across all workers - GPU resource view ( g ) — utilization, VRAM, temperature, power via DCGM exporter - Model-grouped view ( m ) — aggregate stats by model with drill-down - Kubernetes-native — auto-discovers pods, scrapes through API server proxy, no port-forwards needed - Works with NVIDIA Dynamo — filters frontends, labels prefill/decode workers automatically Backend Support Backend Metrics Auto-detect Notes --------- --------- ------------- ------- vLLM ✅ Full ✅ Yes vllm: metric prefix SGLang ✅ Full ✅ Yes sglang: metric prefix LMCache ✅ Cache ✅ Yes lmcache metric prefix NIM ✅ Full ✅ Yes Unprefixed vLLM metrics at /v1/metrics Dynamo ✅ Full ✅ Yes Auto-filters frontends, labels decode/prefill workers TGI ✅ Full ✅ Yes tgi metric prefix, no KV cache metrics TensorRT-LLM ✅ Full ✅ Yes trtllm prefix at /prometheus/metrics Triton ✅ Full ✅ Yes nv inference / nv trt llm on port 8002 llama.cpp ✅ Full ✅ Yes llamacpp: prefix, requires --metrics flag LiteLLM ✅ Full ✅ Yes litellm prefix, proxy-level metrics Ollama ⚡ Basic ✅ Yes JSON /api/ps — model name + online status Keyboard Shortcuts Key Action ----- -------- s Cycle sort column f Cycle backend filter d Detail view g GPU view m Model-grou","default_branch":null,"files":null,"tree":[],"storefront":"/r/InfraWhisperer","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/InfraWhisperer/llmtop/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}