{"repo":"grgalex/nvshare","free":true,"listed":false,"github":"https://github.com/grgalex/nvshare","clone":"git clone https://github.com/grgalex/nvshare.git","description":"Practical GPU Sharing Without Memory Size Constraints","language":"C","stars":316,"topics":["gpu","kubernetes"],"license":"Apache-2.0","category":"deployment-docker-iac","readme_excerpt":"nvshare : Practical GPU Sharing Without Memory Size Constraints nvshare is a GPU sharing mechanism that allows multiple processes (or containers running on Kubernetes) to securely run on the same physical GPU concurrently, each having the whole GPU memory available. You can watch a quick explanation plus a demonstration at https://www.youtube.com/watch?v=9n-5sc5AICY. To achieve this, it transparently enables GPU page faults using the system RAM as swap space. To avoid thrashing, it uses nvshare-scheduler , which manages the GPU and gives exclusive GPU access to a single process for a given time quantum (TQ), which has a default duration of 30 seconds. This functionality solely depends on the Unified Memory API provided by the NVIDIA kernel driver. It is highly unlikely that an update to NVIDIA's kernel drivers would interfere with the viability of this project as it would require disabling Unified Memory. The de-facto way (Nvidia's device plugin) of handling GPUs on Kubernetes is to assign them to containers in a 1-1 manner. This is especially inefficient for applications that only use a GPU in bursts throughout their execution, such as long-running interactive development jobs like Jupyter notebooks. I've written a Medium article on the challenges of GPU sharing on Kubernetes, it's worth a read. Indicative Use Cases - Run 2+ processes/containers with infrequent GPU bursts on the same GPU (e.g., interactive apps, ML inference) - Run 2+ non-interactive workloads (e.g., ML trai","default_branch":null,"files":null,"tree":[],"storefront":"/r/grgalex","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/grgalex/nvshare/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}