[slurm-users] Managing Multiple Dependencies

[Renfro, Michael]

There may be prettier ways, but this gets the job done. Captures the output from each sbatch command to get a job ID, colon separates the ones in the second group, and removes the trailing colon before submitting the last job:


#!/bin/bash
JOB1=$(sbatch job1.sh | awk '{print $NF}')
echo "Submitted job 1, id ${JOB1}"
JOB2N=""
JOB2N_FILES="job2.sh job3.sh"
for job in ${JOB2N_FILES}; do
  JOB2N="${JOB2N}$(sbatch --dependency=afterok:$JOB1 $job | awk '{print $NF}'):"
  echo "Submitted $job, list is now ${JOB2N}"
done
JOB2N=$(echo ${JOB2N} | sed 's/:$//g')
echo "Submitting last job"
sbatch --dependency=afterok:$JOB2N joblast.sh



From: slurm-users <slurm-users-bounces at lists.schedmd.com> on behalf of Jason Simms <simmsj at lafayette.edu>
Date: Tuesday, March 2, 2021 at 1:18 PM
To: Slurm User Community List <slurm-users at lists.schedmd.com>
Subject: [slurm-users] Managing Multiple Dependencies

External Email Warning

This email originated from outside the university. Please use caution when opening attachments, clicking links, or responding to requests.

________________________________
Hello all,

I am relatively new to the nuances of handling complex dependencies in Slurm, so I'm hoping the hive mind can help. I have a user wanting to accomplish the following:

  *   submit one job
  *   submit multiple jobs that are dependent on the output from the first job (so they just need to launch once the first job has completed)
  *   submit one final job dependent on all the previous jobs completing
Is there a way to do this cleanly? So it's a three stage process. I have ideas in my head of writing Slurm JobIDs to a file, reading them out, and managing dependencies that way, but perhaps there is a more efficient way (or perhaps not!).

Warmest regards,
Jason

--
Jason L. Simms, Ph.D., M.P.H.
Manager of Research and High-Performance Computing
XSEDE Campus Champion
Lafayette College
Information Technology Services
710 Sullivan Rd | Easton, PA 18042
Office: 112 Skillman Library
p: (610) 330-5632
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20210302/37b2d818/attachment.htm>