<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi Paul,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Try</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">sacctmgr modify qos gputest set flags=DenyOnLimit</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);"><br>
</span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">Sean</span><br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> slurm-users <slurm-users-bounces@lists.schedmd.com> on behalf of Paul Raines <raines@nmr.mgh.harvard.edu><br>
<b>Sent:</b> Saturday, 29 May 2021 12:48<br>
<b>To:</b> slurm-users@lists.schedmd.com <slurm-users@lists.schedmd.com><br>
<b>Subject:</b> [EXT] [slurm-users] rejecting jobs that exceed QOS limits</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">External email: Please exercise caution<br>
<br>
<br>
I want to dedicate one of our GPU servers for testing where<br>
users are only allowed to run 1 job at a time using 1 GPU and<br>
8 cores of the server. So I put one server in a partition on its<br>
own and setup a QOS for it as follows:<br>
<br>
sacctmgr add qos gputest<br>
sacctmgr modify qos gputest set priority=20<br>
sacctmgr modify qos gputest set MaxJobsPerUser=1<br>
sacctmgr modify qos gputest set MaxTRESPerUser=cpu=8,gres/gpu=1<br>
sacctmgr show qos format=name,priority,MaxTRESPerUser,MaxJobsPerUser<br>
<br>
In slurm.conf I have:<br>
<br>
AccountingStorageEnforce=safe,qos<br>
AccountingStorageTRES=Billing,CPU,Energy,Mem,Node,FS/Disk,Pages,VMem,gres/gpu<br>
EnforcePartLimits=ALL<br>
<br>
<br>
This works but when I submit a job asking for 2 more more GPUs, instead<br>
of being immediate rejected it queues but never runs. Same if I<br>
ask for more than 8 cores<br>
<br>
Is there a way to get it immediately rejected?<br>
<br>
</div>
</span></font></div>
</body>
</html>