<div dir="ltr">A feature that many slurm users might like is sbatch --time-min. Using both --time-min and --time a user can specify the range of acceptable wall times limits. This can make it much easier to keep jobs running right up to the maintenance reservation. e.g.:<div><br></div><div>sbatch --time-min=30:00 --time=48:00:00 script.sh</div><div><br></div><div>would allow the job to schedule for any time-slot between 30 minutes and 2 days in length. If the user has some mechanism for job chaining or similar, this can allow them to make the most of backfill opportunities.</div><div><br></div><div>-Doug</div><div><br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><font face="courier new, monospace" size="1">----</font></div><div><font face="arial, helvetica, sans-serif"><font size="2">Doug Jacobsen, Ph.D.</font><br></font></div><div><font face="arial, helvetica, sans-serif" size="1">NERSC Computer Systems Engineer</font></div><div><font size="1"><font face="arial, helvetica, sans-serif"><a href="http://www.nersc.gov" target="_blank">National Energy Research Scientific Computing Center</a></font><br></font></div><div><font size="1"><a href="mailto:dmjacobsen@lbl.gov" style="font-family:arial,helvetica,sans-serif" target="_blank">dmjacobsen@lbl.gov</a><span style="font-family:arial,helvetica,sans-serif"></span></font><br></div><div><font size="1"><br></font></div><div><font size="1"><font face="courier new, monospace"><span style="color:rgb(136,136,136)">------------- __o</span><br style="color:rgb(136,136,136)"><span style="color:rgb(136,136,136)">---------- _ '\<,_</span><br style="color:rgb(136,136,136)"><span style="color:rgb(136,136,136)">----------(_)/ (_)__________________________</span></font><br></font></div><div style="font-family:arial;font-size:small"><br></div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr">On Fri, May 11, 2018 at 7:27 AM Paul Edmon <<a href="mailto:pedmon@cfa.harvard.edu">pedmon@cfa.harvard.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">In the past we used the LUA job submit plugin to block jobs that would <br>
intersect maintenance reservations. I would look at that.<br>
<br>
-Paul Edmon-<br>
<br>
<br>
On 05/11/2018 08:19 AM, Bill Wichser wrote:<br>
> The problem is that reservations can be in there yet have no effect on <br>
> the submitted job if they would run before the reservation takes <br>
> place. One can pull the starting time simply using something like this<br>
><br>
> scontrol show res -o | awk '{print $2}'<br>
><br>
> with output<br>
><br>
> StartTime=2018-06-12T06:00:00<br>
> StartTime=2018-06-12T06:00:00<br>
><br>
> You'd need more code around that, obviously, to determine if this <br>
> starttime might hold up the job.<br>
><br>
> Bill<br>
><br>
><br>
> On 05/10/2018 04:23 PM, Prentice Bisbal wrote:<br>
>> Dear Slurm Users,<br>
>><br>
>> We've started using maintenance reservations. As you would expect, <br>
>> this caused some confusion for users who were wondering why their <br>
>> jobs were queuing up and not running. Some of my users provide a <br>
>> public service of sorts that automatically submits jobs to our <br>
>> cluster. They would like to have their submission framework <br>
>> automatically detect if there's a reservation that may interfere with <br>
>> their jobs, and act accordingly.<br>
>><br>
>> What is the best way to do this? Typically, in my shell scripts, I <br>
>> have some command that tests something, and then check exit code <br>
>> returned by the command. For example to check if my name is in file <br>
>> 'foo.txt', I'd do something like this:<br>
>><br>
>> grep -iq prentice foo.txt<br>
>> retval=$?<br>
>> if [ $retval -eq 0 ]; then<br>
>> echo "Prentice found"<br>
>> else<br>
>> echo "Prentice not found"<br>
>> fi<br>
>> unset retval<br>
>><br>
>> Or something like that. I was also thinking this might work, too:<br>
>><br>
>> num_res=$(scontrol -o show res | wc -l)<br>
>> if [ $num_res -eq 0 ]; then<br>
>> echo "No reservations found"<br>
>> else<br>
>> echo "$num_res reservation(s) found"<br>
>> fi<br>
>><br>
>> Are there any better or other ways that you would recommend? Also, if <br>
>> there's more than one, is are they listed in any kind of order in the <br>
>> scontrol or sinfo output (soonest first, soonest last, etc.)? From <br>
>> the man page, it looks like 'scontrol show reservation' doesn't <br>
>> provide any sorting.<br>
>><br>
>> Prentice<br>
>><br>
>><br>
>><br>
>><br>
>><br>
><br>
<br>
<br>
</blockquote></div>