[slurm-users] MaxSubmitJobsPerUser does not work as expected

[Aravindh Sampathkumar]

Hi.

I'm trying to set *MaxSubmitJobsPerUser *to a QOS in expectation that it will limit a user from submitting more than a certain number of jobs at a time. However it seems to limit the user at a much smaller number of jobs.

I ran the following command to set the limit.
sacctmgr modify qos normal set MaxSubmitJobsPerUser=144

I can see that it is set..

# sacctmgr show qos
 Name Priority GraceTime Preempt PreemptMode Flags UsageThres UsageFactor GrpTRES GrpTRESMins GrpTRESRunMin GrpJobs GrpSubmit GrpWall MaxTRES MaxTRESPerNode MaxTRESMins MaxWall MaxTRESPU MaxJobsPU MaxSubmitPU MaxTRESPA MaxJobsPA MaxSubmitPA MinTRES
---------- ---------- ---------- ---------- ----------- ---------------------------------------- ---------- ----------- ------------- ------------- ------------- ------- --------- ----------- ------------- -------------- ------------- ----------- ------------- --------- ----------- ------------- --------- ----------- -------------
 normal 0 00:00:00 cluster 1.000000 cpu=72,mem=7+ 144

However, when a user tries to submit a job, they see the error below.

:[~] > srun -n4 --pty bash
srun: error: QOSMaxSubmitJobPerUserLimit
srun: error: Unable to allocate resources: Job violates accounting/QOS policy (job submit limit, user's size and/or time limits)

I checked and made sure that they are well under the set limit of 144. They only have 38 jobs in the system.

Do any of you use this QOS Limit? What am I missing here? 


Thanks for your time! 
-- 
 Aravindh Sampathkumar
 aravindh at fastmail.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20190206/4e61af51/attachment.html>