<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=gb2312"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:等线;
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"\@等线";
panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
{font-family:"MS UI Gothic";
panose-1:2 11 6 0 7 2 5 8 2 4;}
@font-face
{font-family:"\@MS UI Gothic";}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:windowtext;}
span.EmailStyle20
{mso-style-type:personal-reply;
font-family:等线;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=ZH-CN link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span lang=EN-US style='font-size:10.5pt;font-family:等线'>Well, ‘</span><span lang=EN-US>sacctmgr modify cluster name=***’ is exactly what we want, and inspired by this command, we found that ‘sacctmgr show cluster’ can clearly list all the cluster associations.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>But during test we found another problem. When limitation is defined both on cluster level and user level, the smaller one will take effect, user association did not take precedence of low level one. For example:<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'>> sacctmgr show association format=cluster,account,user,grptres,qos<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'> Cluster Account User GrpTRES QOS<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'>---------- ---------- ---------- ------------- --------------------<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'> sjtupi root gres/gpu=1 normal<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'> sjtupi acct-hpc normal<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'> sjtupi acct-hpc hpczty gres/gpu=2 normal<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>Cluster association defined 1-gpu limitation and User association defined 2-gpu limitation, and then 2-gpu job be blocked:<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'>> scontrol show job 6567880<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'>JobId=6567880 JobName=test<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'> UserId=hpczty(3861) GroupId=hpczty(3861) MCS_label=N/A<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'> Priority=127 Nice=0 Account=acct-hpc QOS=normal<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'> <b>JobState=PENDING Reason=AssocGrpGRES</b> Dependency=(null)<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'> Requeue=0 Restarts=0 BatchFlag=1 Reboot=0 ExitCode=0:0<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'> …<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'> NumNodes=1-1 NumCPUs=1 NumTasks=1 CPUs/Task=1 ReqB:S:C:T=0:0:*:*<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'> TRES=cpu=1,mem=7G,node=1,billing=1,<b>gres/gpu=2</b><o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'> Socks/Node=* NtasksPerN:B:S:C=0:0:*:* CoreSpec=*<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'> MinCPUsNode=1 MinMemoryCPU=7G MinTmpDiskNode=0<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'> Features=(null) DelayBoot=00:00:00<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:"Courier New"'> …<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>According to official document <a href="https://slurm.schedmd.com/resource_limits.html">https://slurm.schedmd.com/resource_limits.html</a> , User association at hierarchy 3 should have higher priority than Cluster association at hierarchy 5. Is this a bug or document wrote wrong?<o:p></o:p></span></p><p class=MsoNormal style='text-indent:11.25pt'><span lang=EN-US style='font-size:10.5pt;font-family:等线'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span style='font-family:等线'>发件人<span lang=EN-US>:</span></span></b><span lang=EN-US style='font-family:等线'> Paul Brunk <pbrunk@uga.edu> <br></span><b><span style='font-family:等线'>发送时间<span lang=EN-US>:</span></span></b><span lang=EN-US style='font-family:等线'> 2022</span><span style='font-family:等线'>年<span lang=EN-US>2</span>月<span lang=EN-US>10</span>日<span lang=EN-US> 10:28<br></span><b>收件人<span lang=EN-US>:</span></b><span lang=EN-US> Slurm User Community List <slurm-users@lists.schedmd.com><br></span><b>主题<span lang=EN-US>:</span></b><span lang=EN-US> Re: [slurm-users] What is the 'Root/Cluster association' level in Resource Limits document mean?<o:p></o:p></span></span></p></div></div><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>Hi:<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US>You can use e.g. 'sacctmgr show -s users', and you'll see each user's<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>cluster assocation as one of the output columns. If the name were<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>'yourcluster', then you could do: sacctmgr modify cluster<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US>name=yourcluster set grpTres="node=8".<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><div><div><p class=MsoNormal><span lang=EN-US style='color:black'>== <o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:black'>Paul Brunk, system administrator</span><span lang=EN-US><o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:black'>Georgia Advanced Resource Computing Center<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='color:black'>Enterprise IT Svcs, the University of Georgia<o:p></o:p></span></p></div></div><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p><p class=MsoNormal style='margin-bottom:12.0pt'><span lang=EN-US>On 2/8/22, 2:33 AM, "slurm-users" <<a href="mailto:slurm-users-bounces@lists.schedmd.com">slurm-users-bounces@lists.schedmd.com</a>> wrote:<o:p></o:p></span></p><div><p class=MsoNormal><span lang=EN-US>…[H]ow to check or modify this “cluster association”? Using command sacctmgr show association, I can only list all users’ association.<o:p></o:p></span></p></div><div><p class=MsoNormal><span lang=EN-US><o:p> </o:p></span></p></div><div><p class=MsoNormal><span lang=EN-US>Considering the scene in which we want to set a default node number limitation for all users, command such as sacctmgr modify user set grptres="node=8" do can set the limitation on all users at once, but it will cover the original per-user limitation on some specific account. So it may not be an satisfying solution. If the “cluster association” exists, it may be exactly what we want. So how to set the “cluster association”?<o:p></o:p></span></p></div></div></body></html>