<div dir="ltr">Hello, <br><br>I have written a script in my prolog.sh that cancels any slurm job if the parameter gres=gpu is not present. This is the script i added to my prolog.sh<br><br><font face="monospace">if [ $SLURM_JOB_PARTITION == "gpu" ]; then<br>        if [ ! -z "${GPU_DEVICE_ORDINAL}" ]; then<br>                echo "GPU ID used is ID: $GPU_DEVICE_ORDINAL "<br>                list_gpu=$(echo "$GPU_DEVICE_ORDINAL" | sed -e "s/,//g")<br>                Ngpu=$(expr length $list_gpu)<br>        else<br>                echo "No GPU selected"<br>                Ngpu=0<br>        fi</font><div><span style="font-family:monospace"><br></span></div><div><span style="font-family:monospace">       # if  0 gpus were allocated, cancel the job</span><br></div><div><font face="monospace">        if [ "$Ngpu" -eq "0" ]; then<br>              scancel ${SLURM_JOB_ID}                                          fi<br>fi</font><br><br>What the code does is look at the number of gpus allocated, and if it is 0, cancel the job ID. It working fine if a user use sbatch submit.sh (and the submit.sh do not have the value --gres=gpu:1). However, when requesting an interactive session without gpus, the job is getting killed and the job hangs for 5-6 mins before getting killed. <br><pre class="gmail-c-mrkdwn__pre" style="box-sizing:inherit;margin-top:4px;margin-bottom:4px;padding:8px;font-size:12px;line-height:1.50001;font-variant-ligatures:none;white-space:pre-wrap;word-break:normal;border-radius:4px;color:rgb(29,28,29);font-family:Monaco,Menlo,Consolas,"Courier New",monospace">jlo@mfe01:~ $ srun --partition=gpu --pty bash --login
srun: job 4631872 queued and waiting for resources
srun: job 4631872 has been allocated resources
srun: Force Terminated job 4631872 ...the killing hangs for 5-6minutes</pre>Is there anything wrong with my script? Why only when scancel an interactive session, I am seeing this hanging. I would like to remove the hanging</div><div>Thanks<br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><p class="MsoNormal" style="background-image:initial;background-position:initial;background-repeat:initial"><b><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:#9c1d21">Fritz Ratnasamy</span></b><span style="font-size:10.5pt;font-family:"Helvetica Neue";color:#333333"></span></p>

<p class="MsoNormal" style="background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:#9c1d21">Data Scientist</span><span style="font-size:10.5pt;font-family:"Helvetica Neue";color:#333333"></span></p>

<p class="MsoNormal" style="background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:10.5pt;font-family:"Arial",sans-serif;color:#9c1d21">Information Technology</span><span style="font-size:10.5pt;font-family:"Helvetica Neue";color:#333333"></span></p>

<p class="MsoNormal" style="background-image:initial;background-position:initial;background-repeat:initial"><span style="color:rgb(103,110,115);font-family:Times,serif;font-size:10.5pt">The University of Chicago</span></p><p class="MsoNormal" style="background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:10.5pt;font-family:"Helvetica Neue";color:#333333"></span></p>

<p class="MsoNormal" style="background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:10.5pt;font-family:"Times",serif;color:#676e73">Booth School of Business</span><span style="font-size:10.5pt;font-family:"Helvetica Neue";color:#333333"></span></p>

<p class="MsoNormal" style="background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:10.5pt;font-family:"Times",serif;color:#676e73">5807 S. Woodlawn</span><span style="font-size:10.5pt;font-family:"Helvetica Neue";color:#333333"></span></p>

<p class="MsoNormal" style="background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:10.5pt;font-family:"Times",serif;color:#676e73">Chicago,</span><span style="font-size:10.5pt;font-family:"Helvetica Neue";color:#333333"> </span><span style="font-size:10.5pt;font-family:"Times",serif;color:#676e73">Illinois</span><span style="font-size:10.5pt;font-family:"Helvetica Neue";color:#333333"> </span><span style="font-size:10.5pt;font-family:"Times",serif;color:#676e73">60637</span></p>

<p class="MsoNormal" style="background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:10.5pt;font-family:"Times",serif;color:#676e73">Phone: +(1) 773-834-4556</span><span style="font-size:10.5pt;font-family:"Helvetica Neue";color:#333333"></span></p></div></div></div></div></div>