<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:0 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Century Gothic";
panose-1:2 11 5 2 2 2 2 2 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1676297615;
mso-list-template-ids:-1647569986;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:\F0B7 ;
mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:\F0A7 ;
mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:\F0A7 ;
mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:\F0A7 ;
mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:\F0A7 ;
mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:\F0A7 ;
mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:\F0A7 ;
mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:\F0A7 ;
mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">There may be prettier ways, but this gets the job done. Captures the output from each sbatch command to get a job ID, colon separates the ones in the second group, and removes the trailing colon before submitting the last job:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="mso-element:para-border-div;border-top:double windowtext 2.25pt;border-left:none;border-bottom:double windowtext 2.25pt;border-right:none;padding:1.0pt 0in 1.0pt 0in">
<p class="MsoNormal" style="border:none;padding:0in"><o:p> </o:p></p>
<p class="MsoNormal" style="border:none;padding:0in">#!/bin/bash<o:p></o:p></p>
<p class="MsoNormal" style="border:none;padding:0in">JOB1=$(sbatch job1.sh | awk '{print $NF}')<o:p></o:p></p>
<p class="MsoNormal" style="border:none;padding:0in">echo "Submitted job 1, id ${JOB1}"<o:p></o:p></p>
<p class="MsoNormal" style="border:none;padding:0in">JOB2N=""<o:p></o:p></p>
<p class="MsoNormal" style="border:none;padding:0in">JOB2N_FILES="job2.sh job3.sh"<o:p></o:p></p>
<p class="MsoNormal" style="border:none;padding:0in">for job in ${JOB2N_FILES}; do<o:p></o:p></p>
<p class="MsoNormal" style="border:none;padding:0in"> JOB2N="${JOB2N}$(sbatch --dependency=afterok:$JOB1 $job | awk '{print $NF}'):"<o:p></o:p></p>
<p class="MsoNormal" style="border:none;padding:0in"> echo "Submitted $job, list is now ${JOB2N}"<o:p></o:p></p>
<p class="MsoNormal" style="border:none;padding:0in">done<o:p></o:p></p>
<p class="MsoNormal" style="border:none;padding:0in">JOB2N=$(echo ${JOB2N} | sed 's/:$//g')<o:p></o:p></p>
<p class="MsoNormal" style="border:none;padding:0in">echo "Submitting last job"<o:p></o:p></p>
<p class="MsoNormal" style="border:none;padding:0in">sbatch --dependency=afterok:$JOB2N joblast.sh<o:p></o:p></p>
<p class="MsoNormal" style="border:none;padding:0in"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">slurm-users <slurm-users-bounces@lists.schedmd.com> on behalf of Jason Simms <simmsj@lafayette.edu><br>
<b>Date: </b>Tuesday, March 2, 2021 at 1:18 PM<br>
<b>To: </b>Slurm User Community List <slurm-users@lists.schedmd.com><br>
<b>Subject: </b>[slurm-users] Managing Multiple Dependencies<o:p></o:p></span></p>
</div>
<p align="center" style="margin:0in;text-align:center;background:white"><b><span style="font-size:12.0pt;color:red;background:white">External Email Warning</span></b><o:p></o:p></p>
<p align="center" style="mso-margin-top-alt:0in;margin-right:12.0pt;margin-bottom:0in;margin-left:12.0pt;text-align:center;background:white">
<b><span style="font-size:12.0pt;color:red">This email originated from outside the university. Please use caution when opening attachments, clicking links, or responding to requests.</span></b><o:p></o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="0" width="100%" align="center">
</div>
<div>
<div>
<p class="MsoNormal">Hello all, <o:p></o:p></p>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">I am relatively new to the nuances of handling complex dependencies in Slurm, so I'm hoping the hive mind can help. I have a user wanting to accomplish the following:<o:p></o:p></p>
</div>
<div>
<ul type="disc">
<li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
submit one job<o:p></o:p></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
submit multiple jobs that are dependent on the output from the first job (so they just need to launch once the first job has completed)<o:p></o:p></li><li class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1">
submit one final job dependent on all the previous jobs completing<o:p></o:p></li></ul>
<div>
<p class="MsoNormal">Is there a way to do this cleanly? So it's a three stage process. I have ideas in my head of writing Slurm JobIDs to a file, reading them out, and managing dependencies that way, but perhaps there is a more efficient way (or perhaps not!).<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Warmest regards,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">Jason<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal">-- <o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal"><b><span style="font-size:10.5pt;font-family:"Century Gothic",sans-serif;color:#822433">Jason L. Simms, Ph.D., M.P.H.</span></b><span style="font-size:10.5pt;font-family:Helvetica;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Century Gothic",sans-serif;color:black">Manager of Research and High-Performance Computing</span><span style="font-size:10.5pt;font-family:Helvetica;color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Century Gothic",sans-serif;color:black">XSEDE Campus Champion<br>
</span><span style="font-size:10.5pt;font-family:"Century Gothic",sans-serif;color:gray">Lafayette College<br>
Information Technology Services<br>
710 Sullivan Rd | Easton, PA 18042<br>
Office: 112 Skillman Library<br>
p: (610) 330-5632</span><span style="font-size:10.5pt;font-family:Helvetica;color:black"><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>