{"repo":"facebookincubator/submitit","free":true,"listed":false,"github":"https://github.com/facebookincubator/submitit","clone":"git clone https://github.com/facebookincubator/submitit.git","description":"Python 3.8+ toolbox for submitting jobs to Slurm","language":"Python","stars":1633,"topics":["slurm","python","clusters"],"license":"MIT","category":"dev-tools","readme_excerpt":"Submit it! What is submitit? Submitit is a lightweight tool for submitting Python functions for computation within a Slurm cluster. It basically wraps submission and provide access to results, logs and more. Slurm is an open source, fault-tolerant, and highly scalable cluster management and job scheduling system for large and small Linux clusters. Submitit allows to switch seamlessly between executing on Slurm or locally. An example is worth a thousand words: performing an addition From inside an environment with submitit installed: The Job class also provides tools for reading the log files ( job.stdout() and job.stderr() ). If what you want to run is a command, turn it into a Python function using submitit.helpers.CommandFunction , then submit it. By default stdout is silenced in CommandFunction , but it can be unsilenced with verbose=True . Find more examples here!!! Submitit is a Python 3.8+ toolbox for submitting jobs to Slurm. It aims at running python function from python code. Install Quick install, in a virtualenv/conda environment where pip is installed (check which pip ): - stable release: - stable release using conda : - main branch: You can try running the MNIST example to check that everything is working as expected (requires sklearn). Documentation See the following pages for more detailled information: - Examples: for a bunch of examples dealing with errors, concurrency, multi-tasking etc... - Structure and main objects: to get a better understanding of how su","default_branch":null,"files":null,"tree":[],"storefront":"/r/facebookincubator","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/facebookincubator/submitit/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}