Differences
This shows you the differences between two versions of the page.
Next revision | Previous revision | ||
en:services:application_services:high_performance_computing:running_jobs_slurm:dependencies [2022/03/09 13:07] – created mboden | en:services:application_services:high_performance_computing:running_jobs_slurm:dependencies [2022/03/09 13:40] (current) – [Job Dependencies] mboden | ||
---|---|---|---|
Line 1: | Line 1: | ||
+ | ====== Job Dependencies ====== | ||
+ | Slurm supports dependencies between jobs, allowing users to define simple workflows and pipelines. To create a depenency, you can use the following sbatch flag: | ||
+ | < | ||
+ | where '' | ||
+ | <code bash> | ||
+ | This will submit a job, which can only start once the job with the ID 11536887 finished successfully. | ||
+ | A list of possible dependency types can be found in the following table and in the [[https:// | ||
+ | |||
+ | ^type^Job runs after...^ | ||
+ | | after: | ||
+ | | afterany: | ||
+ | | afternotok: | ||
+ | | afterok: | ||
+ | | aftercorr: | ||
+ | | singleton | Only one of a kind may run at a time | | ||
+ | |||
+ | Multiple jobs can be specified by appending with a colon as delimiter, e.g. '' | ||
+ | |||
+ | ===== Scripting with Job Dependencies ===== | ||
+ | Manually creating complex job dependencies on the command line can be cumbersome and error-prone. It is advised to set up scripts to do this task for you. '' | ||
+ | |||
+ | ===== Examples ===== | ||
+ | |||
+ | Capture '' | ||
+ | <code bash> | ||
+ | # submit a first job, extract job id | ||
+ | jobid=$(sbatch --parsable job1.sbatch) | ||
+ | |||
+ | # submit a second job with dependency: starts only if the previous job terminates successfully) | ||
+ | jobid=$(sbatch --parsable --dependency=afterok: | ||
+ | |||
+ | # submit a third job with dependency: starts only if the previous job terminates successfully) | ||
+ | jobid=$(sbatch --parsable --dependency=afterok: | ||
+ | |||
+ | ---- | ||
+ | |||
+ | Submit scripts, each with a with dependency on the previous one: | ||
+ | <file bash sub_dep.sh># | ||
+ | |||
+ | ID=$(sbatch --parsable $1) | ||
+ | shift | ||
+ | for script in " | ||
+ | ID=$(sbatch --parsable --dependency=afterany: | ||
+ | done</ | ||
+ | Usage: '' |