<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0in;
margin-right:0in;
margin-bottom:0in;
margin-left:.5in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:794253169;
mso-list-type:hybrid;
mso-list-template-ids:-1081730492 1756111666 67698691 67698693 67698689 67698691 67698693 67698689 67698691 67698693;}
@list l0:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Calibri",sans-serif;
mso-fareast-font-family:Calibri;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Symbol;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:"Courier New";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-.25in;
font-family:Wingdings;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style>
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Thanks Thomas,<o:p></o:p></p>
<p class="MsoNormal"> That's helpful and a bit more tenable than what I thought was going to be required. I have a few additional questions. Based on my reading of the docs, it seems that GrpTRESmin is set on the account and then each user needs
to have the partition set there. This brings up a couple of questions for me:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<ul style="margin-top:0in" type="disc">
<li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">How can an account have multiple time GrpTRESmin values for separate partitions? I'm guessing those have to be separate accounts then?<o:p></o:p></li><li class="MsoListParagraph" style="margin-left:0in;mso-list:l0 level1 lfo1">All of limits that I applied per queue in pbs are all in qos settings in slurm so I could dispense with the additional partitions but I also need to limit some classes of jobs to particular
sets of nodes and I didn't see any way to accomplish that besides partitions.<o:p></o:p></li></ul>
<p class="MsoNormal" style="margin-left:.25in"><o:p> </o:p></p>
<p class="MsoNormal" style="margin-left:.25in"><o:p> </o:p></p>
<p class="MsoNormal">Thanks again!<o:p></o:p></p>
<p class="MsoNormal">Todd<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:12.0pt;color:black">From: </span></b><span style="font-size:12.0pt;color:black">slurm-users <slurm-users-bounces@lists.schedmd.com> on behalf of "Thomas M. Payerle" <payerle@umd.edu><br>
<b>Reply-To: </b>Slurm User Community List <slurm-users@lists.schedmd.com><br>
<b>Date: </b>Wednesday, December 12, 2018 at 1:45 PM<br>
<b>To: </b>Slurm User Community List <slurm-users@lists.schedmd.com><br>
<b>Subject: </b>Re: [slurm-users] Noob slurm question<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<div>
<div>
<p class="MsoNormal">Slurm accounting is based on the notion of "associations". An association is a set of cluster, partition, allocation account, and user. I think most sites do the accounting so that it is a single limit applied to all partitions, etc.
but you can use sacctmgr to apply limits at any association level. Normally you would set GrpTRESmin at the required level. The GrpTRESmin values apply at the association you set them and on all child associations.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">So while most sites would do something like e.g. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> set GrpTRESmin=cpu=N for allocation acct Acct1<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">thereby allowing members of Acct1 to use (as a group) N cpu-minutes combined across all partitions, you could also do something like<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> set GrpTRESmin=cpu=N for allocation Acct1 and<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> set GrpTRESmin=cpu=A for allocation Acct1 and partitionA<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> set GrpTRESmin=cpu=B for allocation Acct1 and partitionB<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">In this scenario, users of Acct1 can use at most A cpu-min on partitionA and B on paritionB, subject to combined usage on all partitions (A, B, and anything else) does not exceed N.
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Underneath the covers, Slurm and PBS accounting behave a bit differently --- IIRC in PBS you assign "credits" to accounts which then get debited as jobs run. In Slurm, each association tracks usage as jobs run, and you can configure limits
on the usage at various levels. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">The tools for reporting usage of allocation accounts in Slurm leave something to be desired; sshare is the underlying tool but not very user friendly, and I find sbank leaves a lot to be desired.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">I have some Perl libraries interfacing with sshare, etc. on CPAN (ihttps://<a href="http://metacpan.org/pod/Slurm::Sshare">metacpan.org/pod/Slurm::Sshare</a>) which include a basic sbalance command script. You would likely need to modify
the script for your situation (it assumes a situation more like the first example above), but that should not be too bad.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">On Wed, Dec 12, 2018 at 1:58 PM Merritt, Todd R - (tmerritt) <<a href="mailto:tmerritt@email.arizona.edu">tmerritt@email.arizona.edu</a>> wrote:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Hi all,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> I'm new to slurm. I've used PBS extensively and have set up an accounting system that gives groups/account a fixed number of hours per month on a per queue/partition
basis. It decrements that time allocation with every job run and then resets it to the original value at the start of the next month. I had hoped that slurm would do this natively but it doesn't seem like it does. I came across sbank which sounds like it would
implement this but it also seems like it would span partitions and not allow separate limits per partition. Is this something that has already been implemented or could be done in an easier way than what I'm trying?<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"> <o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Thanks,<o:p></o:p></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto">Todd<o:p></o:p></p>
</div>
</div>
</blockquote>
</div>
<p class="MsoNormal"><br clear="all">
<br>
-- <o:p></o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal">Tom Payerle <br>
DIT-ACIGS/Mid-Atlantic Crossroads <a href="mailto:payerle@umd.edu" target="_blank">
payerle@umd.edu</a><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">5825 University Research Park (301) 405-6135<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">University of Maryland<br>
College Park, MD 20740-3831<o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>