<div dir="ltr"><div>I have for instance two nodes:<br></div><div><br></div><div>1) 30 CPUs</div><div>2) 70 CPUs</div><div><br></div><div>so in total I have 100 CPUs.</div><div><br></div><div>Two partition </div><div><br></div><div>1) low priority</div><div>2) high priority</div><div><br></div><div>I also have two data processing pipelines, the first one uses low priority partition and it can use all CPUs available. The second one uses high priority partition and trigger analysis sometimes. I implemented preemption so when the high priority pipeline start analysis slurm suspend low priority jobs and start high priority jobs. How can I limit the CPUs of second pipeline for instance at 50 (independently from node) ? in this way I have always minimum 50 CPUs that work for low level pipeline.</div><div><br></div><div>The second step could be limit the number of CPUs but in relation with the total number of available CPUs , otherwise if the second node goes down, I have 30 CPUs, but with the previous limit of 50, the high priority pipeline will use all those 30 CPUs. </div><div><br></div><div>If I want to preserve some CPUs for database and web operation I need to set a lower CPUs number in slurm.conf on node row?</div><div><br></div><div>Thank you!</div><div><br></div><div><br></div><div><br></div><div class="gmail_extra"><br><div class="gmail_quote">2018-01-05 12:15 GMT+01:00 Ade Fewings <span dir="ltr"><<a href="mailto:ade.fewings@hpcwales.co.uk" target="_blank">ade.fewings@hpcwales.co.uk</a>></span>:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-style:solid;border-left-color:rgb(204,204,204);padding-left:1ex">
<div lang="EN-GB">
<div class="gmail-m_-7190956711610440096WordSection1">
<p class="gmail-MsoNormal"><span>Are the partitions dynamic? i.e. is the desire to limit based on partitions that float across a larger number of nodes and so the desire is to limit a particular partition to maximum amount of those
nodes?<u></u><u></u></span></p>
<p class="gmail-MsoNormal"><span><u></u> <u></u></span></p>
<p class="gmail-MsoNormal"><span>~~<br>
Ade<u></u><u></u></span></p>
<p class="gmail-MsoNormal"><span><u></u> <u></u></span></p>
<p class="gmail-MsoNormal"><span><u></u> <u></u></span></p>
<p class="gmail-MsoNormal"><span><u></u> <u></u></span></p>
<p class="gmail-MsoNormal"><b><span lang="EN-US">From:</span></b><span lang="EN-US"> slurm-users [mailto:<a href="mailto:slurm-users-bounces@lists.schedmd.com" target="_blank">slurm-users-bounces@<wbr>lists.schedmd.com</a>]
<b>On Behalf Of </b>Nicolò Parmiggiani<br>
<b>Sent:</b> 05 January 2018 09:56<br>
<b>To:</b> <a href="mailto:slurm-users@googlegroups.com" target="_blank">slurm-users@googlegroups.com</a><br>
<b>Cc:</b> <a href="mailto:slurm-users@lists.schedmd.com" target="_blank">slurm-users@lists.schedmd.com</a><br>
<b>Subject:</b> Re: [slurm-users] Limit number of CPU in a partition<u></u><u></u></span></p><div><div class="gmail-h5">
<p class="gmail-MsoNormal"><u></u> <u></u></p>
<div>
<p class="gmail-MsoNormal">Hi,<u></u><u></u></p>
<div>
<p class="gmail-MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="gmail-MsoNormal">can someone help me? How can I limit the maximum number of CPUs that a partition can use.<u></u><u></u></p>
</div>
<div>
<p class="gmail-MsoNormal"><u></u> <u></u></p>
</div>
<div>
<p class="gmail-MsoNormal">Thank You.<u></u><u></u></p>
</div>
</div>
<div>
<p class="gmail-MsoNormal"><u></u> <u></u></p>
<div>
<p class="gmail-MsoNormal">2018-01-02 18:28 GMT+01:00 Nicolò Parmiggiani <<a href="mailto:nicolo.parmiggiani@gmail.com" target="_blank">nicolo.parmiggiani@gmail.com</a>><wbr>:<u></u><u></u></p>
<blockquote style="border-style:none none none solid;border-left-width:1pt;border-left-color:rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<div>
<div>
<p class="gmail-MsoNormal"><span style="font-size:9.5pt">I have only one server and two data analysis pipelines, one for standard jobs and other one for high priority job that can be triggered sometimes. <u></u><u></u></span></p>
</div>
<div>
<p class="gmail-MsoNormal"><span style="font-size:9.5pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="gmail-MsoNormal"><span style="font-size:9.5pt">My first solution was to split the CPU of the server in two partition, one for each pipeline. <u></u><u></u></span></p>
</div>
<div>
<p class="gmail-MsoNormal"><span style="font-size:9.5pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="gmail-MsoNormal"><span style="font-size:9.5pt">A more complex (but i suppose better) solution could be to use one partition and manage the priority of the two pipeline and when i get jobs from high priority pipeline those jobs must start immediately, temporarily
blocking standard jobs.<u></u><u></u></span></p>
</div>
<div>
<p class="gmail-MsoNormal"><span style="font-size:9.5pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="gmail-MsoNormal"><span style="font-size:9.5pt">What solution do you recommend? And how can I implement it?<u></u><u></u></span></p>
</div>
<div>
<p class="gmail-MsoNormal"><span style="font-size:9.5pt"><u></u> <u></u></span></p>
</div>
<div>
<p class="gmail-MsoNormal"><span style="font-size:9.5pt">Thank You.<u></u><u></u></span></p>
</div>
</div>
<div>
<div>
<div>
<p class="gmail-MsoNormal"><u></u> <u></u></p>
<div>
<p class="gmail-MsoNormal">2018-01-02 13:33 GMT+01:00 Ole Holm Nielsen <<a href="mailto:Ole.H.Nielsen@fysik.dtu.dk" target="_blank">Ole.H.Nielsen@fysik.dtu.dk</a>>:<u></u><u></u></p>
<blockquote style="border-style:none none none solid;border-left-width:1pt;border-left-color:rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<p class="gmail-MsoNormal">On 01/02/2018 12:59 PM, Nicolò Parmiggiani wrote:<u></u><u></u></p>
<blockquote style="border-style:none none none solid;border-left-width:1pt;border-left-color:rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<p class="gmail-MsoNormal">My problem is that i have for instance 100 CPU, and i want to create two partition each with 50 CPU maximum usage. In this way i can submit job to both partitions independently.<u></u><u></u></p>
</blockquote>
<p class="gmail-MsoNormal" style="margin-bottom:12pt"><br>
I wonder what you really want to achieve? Why do you want to divide your 100 CPUs (nodes or CPU cores?) into 2 partitions? The queue will be much more flexible if identical nodes constitute 1 partition.<br>
<br>
If you really insist on splitting up your nodes into 2 partitions, you can configure that in slurm.conf, for example with 100 nodes named a001..a100:<br>
<br>
PartitionName=partition1 Nodes=a[001-050]<br>
PartitionName=partition2 Nodes=a[051-100]<br>
<br>
If you wish, you can look at my Slurm Wiki pages for inspiration:<br>
<a href="https://wiki.fysik.dtu.dk/niflheim/SLURM" target="_blank">https://wiki.fysik.dtu.dk/<wbr>niflheim/SLURM</a><br>
<a href="https://wiki.fysik.dtu.dk/niflheim/Slurm_configuration" target="_blank">https://wiki.fysik.dtu.dk/<wbr>niflheim/Slurm_configuration</a><u></u><u></u></p>
<blockquote style="border-style:none none none solid;border-left-width:1pt;border-left-color:rgb(204,204,204);padding:0cm 0cm 0cm 6pt;margin:5pt 0cm 5pt 4.8pt">
<p class="gmail-MsoNormal">2018-01-02 11:29 GMT+01:00 Nicolò Parmiggiani <<a href="mailto:nicolo.parmiggiani@gmail.com" target="_blank">nicolo.parmiggiani@gmail.com</a> <mailto:<a href="mailto:nicolo.parmiggiani@gmail.com" target="_blank">nicolo.parmiggiani@<wbr>gmail.com</a>>>:<br>
<br>
Hi,<br>
<br>
how can i limit the number of CPU that a partition can use?<br>
<br>
For instance when a partition reach its maximum CPUs number you can<br>
submit new job but they are put in queue.<u></u><u></u></p>
</blockquote>
<p class="gmail-MsoNormal"><span style="color:rgb(136,136,136)"><br>
<span class="gmail-m_-7190956711610440096m-1187504046345617134hoenzb">/Ole</span><br>
<br>
<span class="gmail-m_-7190956711610440096m-1187504046345617134hoenzb">-- </span><br>
<span class="gmail-m_-7190956711610440096m-1187504046345617134hoenzb">You received this message because you are subscribed to a topic in the Google Groups "slurm-users" group.</span><br>
<span class="gmail-m_-7190956711610440096m-1187504046345617134hoenzb">To unsubscribe from this topic, visit </span>
</span><a href="https://groups.google.com/d/topic/slurm-users/lLpRTC1rvKA/unsubscribe" target="_blank">https://groups.google.com/d/<wbr>topic/slurm-users/lLpRTC1rvKA/<wbr>unsubscribe</a><span class="gmail-m_-7190956711610440096m-1187504046345617134hoenzb"><span style="color:rgb(136,136,136)">.</span></span><span style="color:rgb(136,136,136)"><br>
<span class="gmail-m_-7190956711610440096m-1187504046345617134hoenzb">To unsubscribe from this group and all its topics, send an email to
</span></span><a href="mailto:slurm-users%2Bunsubscribe@googlegroups.com" target="_blank">slurm-users+unsubscribe@<wbr>googlegroups.com</a><span class="gmail-m_-7190956711610440096m-1187504046345617134hoenzb"><span style="color:rgb(136,136,136)">.</span></span><span style="color:rgb(136,136,136)"><br>
<span class="gmail-m_-7190956711610440096m-1187504046345617134hoenzb">For more options, visit </span></span><a href="https://groups.google.com/d/optout" target="_blank">https://groups.google.com/d/<wbr>optout</a><span class="gmail-m_-7190956711610440096m-1187504046345617134hoenzb"><span style="color:rgb(136,136,136)">.</span></span><u></u><u></u></p>
</blockquote>
</div>
<p class="gmail-MsoNormal"><u></u> <u></u></p>
</div>
</div>
</div>
</blockquote>
</div>
<p class="gmail-MsoNormal"><u></u> <u></u></p>
</div>
</div></div></div>
<hr>
<p> <a href="http://www.hpcwales.co.uk" target="_blank"><img src="http://www.hpcwales.co.uk/sites/default/files/hpcwales/Emailsig1-16.jpg" alt="HPC Wales - www.hpcwales.co.uk" width="201" height="136"></a>
</p>
<hr>
<p align="left"><span style="color:rgb(77,77,77);font-family:Arial;font-size:small">The contents of this email and any files transmitted with it are confidential and intended solely for the named addressee only. Unless you are the named addressee (or authorised
to receive this on their behalf) you may not copy it or use it, or disclose it to anyone else. If you have received this email in error, please notify the sender by email or telephone. All emails sent by High Performance Computing Wales have been checked
using an Anti-Virus system. We would advise you to run your own virus check before opening any attachments received as we will not in any event accept any liability whatsoever, once an email and/or attachment is received.
<br>
<br>
High Performance Computing Wales is a private limited company incorporated in Wales on 8 March 2010 as company number 07181701.
<br>
<br>
Our registered office is at Finance Office, Bangor University, Cae Derwen, College Road, Bangor, Gwynedd. LL57 2DG. UK.
<br>
<br>
High Performance Computing Wales is part funded by the European Regional Development Fund through the Welsh Government.
</span></p>
</div><div class="gmail-HOEnZb"><div class="gmail-h5">
<p></p>
-- <br>
You received this message because you are subscribed to a topic in the Google Groups "slurm-users" group.<br>
To unsubscribe from this topic, visit <a href="https://groups.google.com/d/topic/slurm-users/lLpRTC1rvKA/unsubscribe" target="_blank">https://groups.google.com/d/<wbr>topic/slurm-users/lLpRTC1rvKA/<wbr>unsubscribe</a>.<br>
To unsubscribe from this group and all its topics, send an email to <a href="mailto:slurm-users+unsubscribe@googlegroups.com" target="_blank">slurm-users+unsubscribe@<wbr>googlegroups.com</a>.<br>
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank">https://groups.google.com/d/<wbr>optout</a>.<br>
</div></div></blockquote></div><br></div></div>