{"repo":"saturncloud/prefect-saturn","free":true,"listed":false,"github":"https://github.com/saturncloud/prefect-saturn","clone":"git clone https://github.com/saturncloud/prefect-saturn.git","description":"Python client for using Prefect Cloud with Saturn Cloud","language":"Python","stars":17,"topics":["prefect","dask","cloud","data-engineering","python","workflow-engine","workflow"],"license":"BSD-3-Clause","category":"workflow-automation","readme_excerpt":"prefect-saturn prefect-saturn is a Python package that makes it easy to run Prefect Cloud flows on a Dask cluster with Saturn Cloud. For a detailed tutorial, see \"Fault-Tolerant Data Pipelines with Prefect Cloud \". Installation prefect-saturn is available on PyPi. prefect-saturn can be installed directly from GitHub Getting Started prefect-saturn is intended for use inside a Saturn Cloud environment, such as a Jupyter notebook. Customize Dask You can customize the size and behavior of the Dask cluster used to run prefect flows. prefect saturn.PrefectCloudIntegration.register flow with saturn() accepts to arguments to accomplish this: dask cluster kwargs : keyword arguments to pass to the constructor dask saturn.SaturnCluster . dask adapt kwargs : keyword arguments used to configure \"Adaptive Scaling\" For example, the code below tells Saturn that this flow should run on a Dask cluster with 3 xlarge workers, and that prefect should shut down the cluster once the flow run has finished. Contributing See CONTRIBUTING.md for documentation on how to test and contribute to prefect-saturn .","default_branch":null,"files":null,"tree":[],"storefront":"/r/saturncloud","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/saturncloud/prefect-saturn/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}