<div dir="ltr">In case any of you is interested, my team integrated DMTCP checkpoint library with Slurm, thus allowing this preemption to be done without loosing any computation (and some other fancy stuff). An important use case for us is, in fact, these VIP queues :)<div><br></div><div>Here is a link to the documentation, including tests, configuration and howtos: <a href="https://github.com/ciemat-tic/codec/wiki/Slurm-DMTCP">https://github.com/ciemat-tic/codec/wiki/Slurm-DMTCP</a></div><div><br></div><div>cheers, </div><div><br></div><div>Manuel</div><div><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">2018-01-25 17:18 GMT+01:00 Brian Novogradac <span dir="ltr"><<a href="mailto:Brian.Novogradac@utoronto.ca" target="_blank">Brian.Novogradac@utoronto.ca</a>></span>:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div id="m_1759297044253635457divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif" dir="ltr">
<p style="margin-top:0;margin-bottom:0">Thank you for your input!</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">Would any of you have an example on how you setup the conf file for the queue using the QOS method? I'm reading the QOS docs as I type.</p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<p style="margin-top:0;margin-bottom:0">Brian</p>
<div style="color:rgb(0,0,0)">
<hr style="display:inline-block;width:98%">
<div id="m_1759297044253635457divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> slurm-users <<a href="mailto:slurm-users-bounces@lists.schedmd.com" target="_blank">slurm-users-bounces@lists.<wbr>schedmd.com</a>> on behalf of Loris Bennett <<a href="mailto:loris.bennett@fu-berlin.de" target="_blank">loris.bennett@fu-berlin.de</a>><br>
<b>Sent:</b> Thursday, January 25, 2018 11:09 AM<br>
<b>To:</b> Slurm User Community List<br>
<b>Subject:</b> Re: [slurm-users] Creating priority quotas</font>
<div> </div>
</div>
<div class="m_1759297044253635457BodyFragment"><font size="2"><span style="font-size:11pt">
<div class="m_1759297044253635457PlainText"><span class="">Hi Brian,<br>
<br>
QOS is probably the right way to go. You can set up a QOS 'vip', which<br>
can preempt other QOS. We don't preempt, but use multifactor priority<br>
with various QOS with different priority values. In that case, VIP jobs<br>
won't start immediately, but just get pushed to the front of the queue.<br>
On the other hand, no CPU-time is lost due to low-status jobs being<br>
terminated early due to preemption (although if the low-status jobs are<br>
able to do some form of checkpointing that will be less of an issue).<br>
Depending on how pushy your VIPs are, they might go for a preemptionless<br>
solution, too.<br>
<br>
Cheers,<br>
<br>
Loris<br>
<br>
John Hearns <<a href="mailto:hearnsj@googlemail.com" target="_blank">hearnsj@googlemail.com</a>> writes:<br>
<br>
> Brian, not my area of expertise. Do you want 'premption' - ie the VIP user runs something and other jobs are pre-empted?<br>
> <a href="https://slurm.schedmd.com/preempt.html" id="m_1759297044253635457LPlnk874844" target="_blank">
https://slurm.schedmd.com/<wbr>preempt.html</a>
</span><div id="m_1759297044253635457LPBorder_GT_15168970156580.5491772374206965" style="margin-bottom:20px;overflow:auto;width:100%;text-indent:0px">
<table id="m_1759297044253635457LPContainer_15168970156560.12777855690447493" cellspacing="0" style="width:90%;background-color:rgb(255,255,255);overflow:auto;padding-top:20px;padding-bottom:20px;margin-top:20px;border-top:1px dotted rgb(200,200,200);border-bottom:1px dotted rgb(200,200,200)">
<tbody>
<tr valign="top" style="border-spacing:0px">
<td id="m_1759297044253635457TextCell_15168970156570.6638353882898431" colspan="2" style="vertical-align:top;padding:0px;display:table-cell">
<div id="m_1759297044253635457LPRemovePreviewContainer_15168970156570.6263620981649101"></div>
<div id="m_1759297044253635457LPTitle_15168970156570.5401225639196234" style="color:rgb(93,178,255);font-weight:400;font-size:21px;font-family:wf_segoe-ui_light,"Segoe UI Light","Segoe WP Light","Segoe UI","Segoe WP",Tahoma,Arial,sans-serif;line-height:21px">
<a id="m_1759297044253635457LPUrlAnchor_15168970156570.6213726096033383" href="https://slurm.schedmd.com/preempt.html" style="text-decoration:none" target="_blank">Slurm Workload Manager</a></div>
<div id="m_1759297044253635457LPMetadata_15168970156570.4221466078929985" style="margin:10px 0px 16px;color:rgb(102,102,102);font-weight:400;font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif;font-size:14px;line-height:14px">
<a href="http://slurm.schedmd.com" target="_blank">slurm.schedmd.com</a></div>
<div id="m_1759297044253635457LPDescription_15168970156580.5322751300015303" style="display:block;color:rgb(102,102,102);font-weight:400;font-family:wf_segoe-ui_normal,"Segoe UI","Segoe WP",Tahoma,Arial,sans-serif;font-size:14px;line-height:20px;max-height:100px;overflow:hidden">
Preemption. Slurm supports job preemption, the act of stopping one or more "low-priority" jobs to let a "high-priority" job run. Job preemption is implemented as a ...</div>
</td>
</tr>
</tbody>
</table>
</div><span class="">
<br>
<br>
><br>
> On 25 January 2018 at 16:27, Brian Novogradac <<a href="mailto:Brian.Novogradac@utoronto.ca" target="_blank">Brian.Novogradac@utoronto.ca</a>> wrote:<br>
><br>
> I'm new to Slurm, and looking for some assistance.<br>
><br>
> I need to create various queues. The one i am having issues with is a "VIP" queue<br>
><br>
> I want to create a queue for a specific node that overides all jobs on that node when a "VIP" uses the queue.<br>
><br>
> We are using SSSD for our authentication system to the login node.<br>
><br>
> I'm looking at the QOS docs and am stumped.<br>
><br>
> Any help or direction much appreciated.<br>
><br>
> Brian Novogradac<br>
><br>
><br>
-- <br>
Dr. Loris Bennett (Mr.)<br>
ZEDAT, Freie Universität Berlin Email <a href="mailto:loris.bennett@fu-berlin.de" target="_blank">loris.bennett@fu-berlin.de</a><br>
<br>
</span></div>
</span></font></div>
</div>
</div>
</div>
</blockquote></div><br></div>