<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">Hi Hermann,<div class=""><br class=""></div><div class="">I happened to read along this conversation and was just solving this issue today. I added this part to the epilog script to make it work:</div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""># Add job report to stdout</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class="">StdOut=$(/usr/bin/scontrol show job=$SLURM_JOB_ID | /usr/bin/grep StdOut | /usr/bin/xargs | /usr/bin/awk 'BEGIN { FS = "=" } ; { print $2 }')</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""><br class=""></span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class="">NODELIST=($(/usr/bin/scontrol show hostnames))</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""><br class=""></span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""># Only add to StdOut file if it exists and if we are the first node</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class="">if [ "$(/usr/bin/hostname -s)" = "${NODELIST[0]}" -a ! -z "${StdOut}" ]</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class="">then</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""> echo "################################# JOB REPORT ##################################" >> $StdOut</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""> /usr/bin/seff $SLURM_JOB_ID >> $StdOut</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class=""> echo "###############################################################################" >> $StdOut</span></font></div><div class=""><font face="Monaco" class=""><span style="font-style: normal;" class="">fi</span></font></div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=""> Contrary to what it says </span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class="">in the slurm docs</span><span style="caret-color: rgb(0, 0, 0); color: rgb(0, 0, 0);" class=""> </span><a href="https://slurm.schedmd.com/prolog_epilog.html" class="">https://slurm.schedmd.com/prolog_epilog.html</a> I was not able to use the env var SLURM_JOB_STDOUT, so I had to fetch it via scontrol. In addition I had to make sure it is only called by the „leading“ node as the epilog script will be called by ALL nodes of a multinode job and they would all call <i class="">seff </i><span style="font-style: normal;" class="">and clutter up the output</span>. Last thing was to check if StdOut is not of length zero (i.e. it exists). Interactive jobs would otherwise cause the node to drain.</div><div class=""><br class=""></div><div class="">Maybe this helps. </div><div class=""><br class=""></div><div class="">Kind regards</div><div class="">Sebastian</div><div class=""><br class=""></div><div class="">PS: goslmailer looks quite nice with its recommendations! Will definitely look into it.</div><div class=""><br class=""><div class="">
<div class="">--</div><div class="">Westfälische Wilhelms-Universität (WWU) Münster</div><div class="">WWU IT</div><div class="">Sebastian Potthoff (eScience / HPC)</div>
</div>
<div><br class=""><blockquote type="cite" class=""><div class="">Am 15.09.2022 um 18:07 schrieb Hermann Schwärzler <<a href="mailto:hermann.schwaerzler@uibk.ac.at" class="">hermann.schwaerzler@uibk.ac.at</a>>:</div><br class="Apple-interchange-newline"><div class=""><div class="">Hi Ole,<br class=""><br class="">On 9/15/22 5:21 PM, Ole Holm Nielsen wrote:<br class=""><blockquote type="cite" class="">On 15-09-2022 16:08, Hermann Schwärzler wrote:<br class=""><blockquote type="cite" class="">Just out of curiosity: how do you insert the output of seff into the out-file of a job?<br class=""></blockquote>Use the "smail" tool from the slurm-contribs RPM and set this in slurm.conf:<br class="">MailProg=/usr/bin/smail<br class=""></blockquote><br class="">Maybe I am missing something but from what I can tell smail sends an email and does *not* change or append to the .out file of a job...<br class=""><br class="">Regards,<br class="">Hermann<br class=""><br class=""></div></div></blockquote></div><br class=""></div></body></html>