{"repo":"francois-rozet/dawgz","free":true,"listed":false,"github":"https://github.com/francois-rozet/dawgz","clone":"git clone https://github.com/francois-rozet/dawgz.git","description":"Unleash the true power of scheduling","language":"Python","stars":39,"topics":["directed-acyclic-graph","workflow","scheduling","python","hpc","slurm","reproducible-science"],"license":"MIT","category":"workflow-automation","readme_excerpt":"Directed Acyclic Workflow Graph Scheduling dawgz provides a lightweight and intuitive Python interface to declare, schedule and execute job workflows. It can also delegate execution to resource management backends such as Slurm, which means you can write, configure, and submit your workflows without ever leaving Python. Installation The dawgz package is available on PyPi and can be installed with pip . Alternatively, if you need the latest features, you can install it from source. Getting started In dawgz , a job is a Python function. The dawgz.job decorator allows to declare the resources a job requires and capture its arguments. A job's dependencies within the workflow are declared with the dawgz.Job.after method. After declaration, the dawgz.schedule function takes care of scheduling the jobs and their dependencies, with a selected execution backend. For more information, check out the interface and the examples. Follows a small example demonstrating how one could use dawgz to calculate π (very roughly) using the Monte Carlo method. We define two jobs, generate and estimate . Five instances of generate are declared that will be executed concurrently. The estimate job has all generate instances as dependencies, meaning that it will only start after they have completed successfuly. Alternatively, on a Slurm HPC cluster, changing the backend to \"slurm\" results in the following job queue. In addition to the Python interface, dawgz provides a simple command-line interface to li","default_branch":null,"files":null,"tree":[],"storefront":"/r/francois-rozet","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/francois-rozet/dawgz/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}