====== Job Dependencies ====== Slurm supports dependencies between jobs, allowing users to define simple workflows and pipelines. To create a depenency, you can use the following sbatch flag: -d, --dependency= where ''dependency_list'' specifies which jobs to depend on and what kind of dependency it is. This is an example of a simple dependency: sbatch --dependency=afterok:11536887 This will submit a job, which can only start once the job with the ID 11536887 finished successfully. A list of possible dependency types can be found in the following table and in the [[https://slurm.schedmd.com/sbatch.html#OPT_dependency|sbatch man-page]]. ^type^Job runs after...^ | after:job_id | job starts or is cancelled | | afterany:job_id | Terminated in any way | | afternotok:job_id | Terminated non-zero | | afterok:job_id | finished normally | | aftercorr:job_id | After corresponding task in array | | singleton | Only one of a kind may run at a time | Multiple jobs can be specified by appending with a colon as delimiter, e.g. ''-d after:123:124:125''. ===== Scripting with Job Dependencies ===== Manually creating complex job dependencies on the command line can be cumbersome and error-prone. It is advised to set up scripts to do this task for you. ''sbatch'' has the ''--parsable'' flag to make this task easier: When using this flag, only the job ID will be returned and it can be used to capture the job ID into a variable. ===== Examples ===== Capture ''sbatch'' output into variable and use it in the next job: # submit a first job, extract job id jobid=$(sbatch --parsable job1.sbatch) # submit a second job with dependency: starts only if the previous job terminates successfully) jobid=$(sbatch --parsable --dependency=afterok:$jobid job2.sbatch) # submit a third job with dependency: starts only if the previous job terminates successfully) jobid=$(sbatch --parsable --dependency=afterok:$jobid job3.sbatch) ---- Submit scripts, each with a with dependency on the previous one: #!/bin/bash ID=$(sbatch --parsable $1) shift for script in "$@"; do ID=$(sbatch --parsable --dependency=afterany:${ID} $script) done Usage: ''./sub_dep.sh scriptA.sh scriptB.sh scriptC.sh scriptD.sh''