<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
<div class="">Something to try . . </div>
<div class=""><br class="">
</div>
<div class="">If you restart “slurmctld” does the new QOS apply?</div>
<div class=""><br class="">
</div>
<div class="">We had a situation where slurmdbd was running as a different user than slurmctld and hence sacctmgr changes weren’t being reflected in slurmctld.</div>
<div class=""><br class="">
</div>
<div class=""> -greg</div>
<br class="">
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 27 Apr 2020, at 12:57, Simon Andrews <<a href="mailto:simon.andrews@babraham.ac.uk" class="">simon.andrews@babraham.ac.uk</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div class="WordSection1" style="page: WordSection1; caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;">
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
I’m trying to use QoS limits to dynamically change the number of CPUs a user is allowed to use on our cluster. As far as I can see I’m setting the appropriate GrpTRES=cpu value and I can read that back, but then jobs are being stopped before the user has reached
that limit.<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
In squeue I see loads of lines like:<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
166599 normal nf-BISMARK_(288) auser PD 0:00 1 (QOSMaxCpuPerUserLimit)<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
..but if I run:<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
squeue -t running -p normal --format="%.12u %.2t %C "<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Then the total for that user is 288 cores, but in the QoS configuration they should be allowed more. If I run:<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
sacctmgr show user WithAssoc format=user%12,GrpTRES<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
..then I get:<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
auser cpu=512<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
What am I missing? Why is ‘auser’ not being allowed to use all 512 of their allowed CPUs before the QOS limit is kicking in?<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Thanks for any help you can offer.<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
Simon.<o:p class=""></o:p></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif;" class="">
<o:p class=""> </o:p></div>
</div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
<o:p class=""></o:p><o:p class=""></o:p><span style="font-size: 8pt; font-family: Arial, sans-serif; color: rgb(31, 73, 125);" class="">The Babraham Institute, Babraham Research Campus, Cambridge CB22 3AT<span class="Apple-converted-space"> </span></span><i class=""><span style="font-size: 8pt; font-family: Arial, sans-serif; color: rgb(54, 95, 145);" class="">Registered
Charity No. 1053902.</span></i><span style="font-size: 8pt; font-family: Arial, sans-serif; color: rgb(31, 73, 125);" class=""><o:p class=""></o:p></span></div>
<div style="margin: 0cm 0cm 0.0001pt; font-size: 11pt; font-family: Calibri, sans-serif; caret-color: rgb(0, 0, 0); font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; text-decoration: none;" class="">
<span style="font-size: 8pt; font-family: Arial, sans-serif; color: rgb(31, 73, 125);" class="">The information transmitted in this email is directed only to the addressee. If you received this in error, please contact the sender and delete this email from
your system. The contents of this e-mail are the views of the sender and do not necessarily represent the views of the Babraham Institute.<span class="Apple-converted-space"> </span></span><span style="font-size: 8pt; font-family: Arial, sans-serif; color: rgb(54, 95, 145);" class="">Full
conditions at:<span class="Apple-converted-space"> </span></span><a href="http://www.babraham.ac.uk/terms" style="color: rgb(149, 79, 114); text-decoration: underline;" class=""><span style="font-size: 8pt; font-family: Arial, sans-serif; color: rgb(54, 95, 145);" class="">www.babraham.ac.uk</span></a></div>
</div>
</blockquote>
</div>
<br class="">
</body>
</html>