{"repo":"saturncloud/dask-saturn","free":true,"listed":false,"github":"https://github.com/saturncloud/dask-saturn","clone":"git clone https://github.com/saturncloud/dask-saturn.git","description":"Python library for interacting with Dask clusters in Saturn","language":"Python","stars":12,"topics":["python","dask","distributed-computing","data-science","cloud","data-engineering"],"license":"BSD-3-Clause","category":"data-pipelines","readme_excerpt":"dask-saturn Python library for interacting with Dask clusters in Saturn Cloud. Dask-Saturn mimics the API of Dask-Kubernetes, but allows the user to interact with clusters created within Saturn Cloud. Start cluster From within a Jupyter notebook, you can start a cluster: By default this will start a dask cluster with the same settings that you have already set in the Saturn UI or in a prior notebook. To start the cluster with a certain number of workers using the n workers option. Similarly, you can set the scheduler size , worker size , and worker is spot . Note: If the cluster is already running then you can't change the settings. Attempting to do so will raise a warning. Use the shudown on close option to set up a cluster that is tied to the client kernel. This functions like a regular dask LocalCluster , when your jupyter kernel dies or is restarted, the dask cluster will close. Adjust number of workers Once you have a cluster you can interact with it via the jupyter widget, or using the scale and adapt methods. For example, to manually scale up to 20 workers: To create an adaptive cluster that controls its own scaling: Interact with client To submit tasks to the cluster, you sometimes need access to the Client object. Instantiate this with the cluster as the only argument: Close cluster To terminate all resources associated with a cluster, use the close method: Change settings To update the settings (such as n workers , worker size , worker is spot , nthreads ) on an exi","default_branch":null,"files":null,"tree":[],"storefront":"/r/saturncloud","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/saturncloud/dask-saturn/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}