<html>
  <head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8">
  </head>
  <body text="#000000" bgcolor="#FFFFFF">
    <p>Just to revisit this, for jobs that are queued, but prevented
      from running, will have a more useful reason in 18.08, which will
      address one of my issues with reservation collisions. <br>
    </p>
    <p><a class="moz-txt-link-freetext" href="https://bugs.schedmd.com/show_bug.cgi?id=5138">https://bugs.schedmd.com/show_bug.cgi?id=5138</a><br>
    </p>
    <p><a class="moz-txt-link-freetext" href="https://bugs.schedmd.com/show_bug.cgi?id=4987">https://bugs.schedmd.com/show_bug.cgi?id=4987</a></p>
    <pre class="moz-signature" cols="72">Prentice Bisbal
Lead Software Engineer
Princeton Plasma Physics Laboratory
<a class="moz-txt-link-freetext" href="http://www.pppl.gov">http://www.pppl.gov</a></pre>
    <div class="moz-cite-prefix">On 05/11/2018 10:36 AM, Douglas
      Jacobsen wrote:<br>
    </div>
    <blockquote type="cite"
cite="mid:CAHaWJFEB8-MnOrvLey3YSn5NHkxQVswQbiQGP5vFmqMDpy5JYw@mail.gmail.com">
      <div dir="ltr">A feature that many slurm users might like is
        sbatch --time-min.  Using both --time-min and --time a user can
        specify the range of acceptable wall times limits.  This can
        make it much easier to keep jobs running right  up to the
        maintenance reservation.  e.g.:
        <div><br>
        </div>
        <div>sbatch --time-min=30:00 --time=48:00:00 script.sh</div>
        <div><br>
        </div>
        <div>would allow the job to schedule for any time-slot between
          30 minutes and 2 days in length.  If the user has some
          mechanism for job chaining or similar, this can allow them to
          make the most of backfill opportunities.</div>
        <div><br>
        </div>
        <div>-Doug</div>
        <div><br clear="all">
          <div>
            <div dir="ltr" class="gmail_signature"
              data-smartmail="gmail_signature">
              <div dir="ltr">
                <div>
                  <div dir="ltr">
                    <div><font size="1" face="courier new, monospace">----</font></div>
                    <div><font face="arial, helvetica, sans-serif"><font
                          size="2">Doug Jacobsen, Ph.D.</font><br>
                      </font></div>
                    <div><font size="1" face="arial, helvetica,
                        sans-serif">NERSC Computer Systems Engineer</font></div>
                    <div><font size="1"><font face="arial, helvetica,
                          sans-serif"><a href="http://www.nersc.gov"
                            target="_blank" moz-do-not-send="true">National
                            Energy Research Scientific Computing Center</a></font><br>
                      </font></div>
                    <div><font size="1"><a
                          href="mailto:dmjacobsen@lbl.gov"
                          style="font-family:arial,helvetica,sans-serif"
                          target="_blank" moz-do-not-send="true">dmjacobsen@lbl.gov</a><span
                          style="font-family:arial,helvetica,sans-serif"></span></font><br>
                    </div>
                    <div><font size="1"><br>
                      </font></div>
                    <div><font size="1"><font face="courier new,
                          monospace"><span
                            style="color:rgb(136,136,136)">-------------
                            __o</span><br style="color:rgb(136,136,136)">
                          <span style="color:rgb(136,136,136)">----------
                            _ '\<,_</span><br
                            style="color:rgb(136,136,136)">
                          <span style="color:rgb(136,136,136)">----------(_)/
                             (_)__________________________</span></font><br>
                      </font></div>
                    <div style="font-family:arial;font-size:small"><br>
                    </div>
                  </div>
                </div>
              </div>
            </div>
          </div>
          <br>
        </div>
      </div>
      <br>
      <div class="gmail_quote">
        <div dir="ltr">On Fri, May 11, 2018 at 7:27 AM Paul Edmon <<a
            href="mailto:pedmon@cfa.harvard.edu" moz-do-not-send="true">pedmon@cfa.harvard.edu</a>>
          wrote:<br>
        </div>
        <blockquote class="gmail_quote" style="margin:0 0 0
          .8ex;border-left:1px #ccc solid;padding-left:1ex">In the past
          we used the LUA job submit plugin to block jobs that would <br>
          intersect maintenance reservations.  I would look at that.<br>
          <br>
          -Paul Edmon-<br>
          <br>
          <br>
          On 05/11/2018 08:19 AM, Bill Wichser wrote:<br>
          > The problem is that reservations can be in there yet have
          no effect on <br>
          > the submitted job if they would run before the
          reservation takes <br>
          > place. One can pull the starting time simply using
          something like this<br>
          ><br>
          > scontrol show res -o | awk '{print $2}'<br>
          ><br>
          > with output<br>
          ><br>
          > StartTime=2018-06-12T06:00:00<br>
          > StartTime=2018-06-12T06:00:00<br>
          ><br>
          > You'd need more code around that, obviously, to determine
          if this <br>
          > starttime might hold up the job.<br>
          ><br>
          > Bill<br>
          ><br>
          ><br>
          > On 05/10/2018 04:23 PM, Prentice Bisbal wrote:<br>
          >> Dear Slurm Users,<br>
          >><br>
          >> We've started using maintenance reservations. As you
          would expect, <br>
          >> this caused some confusion for users who were
          wondering why their <br>
          >> jobs were queuing up and not running. Some of my
          users provide a <br>
          >> public service of sorts that automatically submits
          jobs to our <br>
          >> cluster. They would like to have their submission
          framework <br>
          >> automatically detect if there's a reservation that
          may interfere with <br>
          >> their jobs, and act accordingly.<br>
          >><br>
          >> What is the best way to do this? Typically, in my
          shell scripts, I <br>
          >> have some command that tests something, and then
          check exit code <br>
          >> returned by the command. For example to check if my
          name is in file <br>
          >> 'foo.txt', I'd do something like this:<br>
          >><br>
          >> grep -iq prentice foo.txt<br>
          >> retval=$?<br>
          >> if [ $retval -eq 0 ]; then<br>
          >>      echo "Prentice found"<br>
          >> else<br>
          >>      echo "Prentice not found"<br>
          >> fi<br>
          >> unset retval<br>
          >><br>
          >> Or something like that. I was also thinking this
          might work, too:<br>
          >><br>
          >> num_res=$(scontrol -o show res  | wc -l)<br>
          >> if [ $num_res -eq 0 ]; then<br>
          >>      echo "No reservations found"<br>
          >> else<br>
          >>      echo "$num_res reservation(s) found"<br>
          >> fi<br>
          >><br>
          >> Are there any better or other ways that you would
          recommend? Also, if <br>
          >> there's more than one, is are they listed in any kind
          of order in the <br>
          >> scontrol or sinfo output (soonest first, soonest
          last, etc.)? From <br>
          >> the man page, it looks like 'scontrol show
          reservation' doesn't <br>
          >> provide any sorting.<br>
          >><br>
          >> Prentice<br>
          >><br>
          >><br>
          >><br>
          >><br>
          >><br>
          ><br>
          <br>
          <br>
        </blockquote>
      </div>
    </blockquote>
    <br>
  </body>
</html>