{"repo":"AgnostiqHQ/covalent-slurm-plugin","free":true,"listed":false,"github":"https://github.com/AgnostiqHQ/covalent-slurm-plugin","clone":"git clone https://github.com/AgnostiqHQ/covalent-slurm-plugin.git","description":"Executor plugin interfacing Covalent with Slurm","language":"Python","stars":26,"topics":["python","workflow","etl","hpc","parallelization","pipelines","python3","covalent","quantum-computing","machinelearning"],"license":"Apache-2.0","category":"data-pipelines","readme_excerpt":"  Covalent Slurm Plugin Covalent is a Pythonic workflow tool used to execute tasks on advanced computing hardware. This executor plugin interfaces Covalent with HPC systems managed by Slurm. For workflows to be deployable, users must have SSH access to the Slurm login node, writable storage space on the remote filesystem, and permissions to submit jobs to Slurm. Installation To use this plugin with Covalent, simply install it using pip : On the remote system, the Python version in the environment you plan to use must match that used when dispatching the calculations. Additionally, the remote system's Python environment must have the base covalent package installed (e.g. pip install covalent ). Usage The following shows an example of a Covalent configuration that is modified to support Slurm: The first stanza describes default connection parameters for a user who can connect to the Slurm login node using, for example: The second and third stanzas describe default parameters for #SBATCH directives and default parameters passed directly to srun , respectively. This example generates a script containing the following preamble: and subsequent workflow submission with: To use the configuration settings, an electron’s executor must be specified with a string argument, in this case: Alternatively, passing a SlurmExecutor instance enables custom behavior scoped to specific tasks. Here, the executor's prerun commands and postrun commands parameters can be used to list shell comman","default_branch":null,"files":null,"tree":[],"storefront":"/r/AgnostiqHQ","claimed":false,"request_supported":{"post":"https://gitbuyer.com/r/AgnostiqHQ/covalent-slurm-plugin/request-supported","requests":0},"note":"indexed from public GitHub; nothing is for sale on this page. Clone it from GitHub. Paid listings live at /search."}