====== Job Dependencies ======
Slurm supports dependencies between jobs, allowing users to define simple workflows and pipelines. To create a depenency, you can use the following sbatch flag:
-d, --dependency=
where ''dependency_list'' specifies which jobs to depend on and what kind of dependency it is. This is an example of a simple dependency:
sbatch --dependency=afterok:11536887
This will submit a job, which can only start once the job with the ID 11536887 finished successfully.
A list of possible dependency types can be found in the following table and in the [[https://slurm.schedmd.com/sbatch.html#OPT_dependency|sbatch man-page]].
^type^Job runs after...^
| after:job_id | job starts or is cancelled |
| afterany:job_id | Terminated in any way |
| afternotok:job_id | Terminated non-zero |
| afterok:job_id | finished normally |
| aftercorr:job_id | After corresponding task in array |
| singleton | Only one of a kind may run at a time |
Multiple jobs can be specified by appending with a colon as delimiter, e.g. ''-d after:123:124:125''.
===== Scripting with Job Dependencies =====
Manually creating complex job dependencies on the command line can be cumbersome and error-prone. It is advised to set up scripts to do this task for you. ''sbatch'' has the ''--parsable'' flag to make this task easier: When using this flag, only the job ID will be returned and it can be used to capture the job ID into a variable.
===== Examples =====
Capture ''sbatch'' output into variable and use it in the next job:
# submit a first job, extract job id
jobid=$(sbatch --parsable job1.sbatch)
# submit a second job with dependency: starts only if the previous job terminates successfully)
jobid=$(sbatch --parsable --dependency=afterok:$jobid job2.sbatch)
# submit a third job with dependency: starts only if the previous job terminates successfully)
jobid=$(sbatch --parsable --dependency=afterok:$jobid job3.sbatch)
----
Submit scripts, each with a with dependency on the previous one:
#!/bin/bash
ID=$(sbatch --parsable $1)
shift
for script in "$@"; do
ID=$(sbatch --parsable --dependency=afterany:${ID} $script)
done
Usage: ''./sub_dep.sh scriptA.sh scriptB.sh scriptC.sh scriptD.sh''