{"repo":"okuvshynov/cubestat","free":true,"listed":false,"github":"https://github.com/okuvshynov/cubestat","clone":"git clone https://github.com/okuvshynov/cubestat.git","description":"Horizon chart for CPU/GPU/Neural Engine utilization monitoring. Supports Apple M1-M4, Nvidia GPUs, AMD GPUs","language":"Python","stars":28,"topics":["command-line-tool","gpu","monitoring","neural-engine","horizon","nvidia-gpu","apple-silicon","rocm"],"license":"MIT","category":"analytics","readme_excerpt":"system monitoring horizon charts for terminal cubestat is a command-line utility to monitor system metrics in horizon chart format. It was originally created for Apple M1/M2 devices, but supports Linux with NVIDIA GPU as well, including Google Colab environment. Numerous tools exist for tracking system metrics, yet horizon charts stand out due to their good information density which enables the display of many time-series data on a single screen. Let's start with an example: https://github.com/okuvshynov/cubestat/assets/661042/8e1e405e-ca61-4ffb-bedb-e04eb33f8bc2 In the clip above we see Mixtral-8x7b inference on MacBook Air with FF layers offloaded to SSD. We can notice somewhat low GPU util, 2Gb/s+ of data read from disk, as we have to fetch the weights, but plenty of free RAM (And we are actually able to serve almost 100Gb model on 24Gb machine with fp16 precision, even if very slow). We can also clearly see moment of change from model loading (cpu util, disk writes for model preprocessing) to model inference (disk reads, gpu util going up, cpu going down) Currently cubestat reports: 1. CPU utilization - configurable per core ('by core'), cluster of cores on Apple M1+: Efficiency/Performance ('by cluster') or all. Is shown as percentage. 2. GPU utilization per card/chip. Is shown in percentage. Works for Apple's M1/M2 SoC and NVIDIA GPUs. For NVIDIA GPU can show VRAM usage as well. In case of multi-GPU can show individual GPUs or aggregated average. 3. ANE (Neural Engine) ","default_branch":null,"files":null,"tree":[],"storefront":"/r/okuvshynov","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/okuvshynov/cubestat/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}