[slurm-users] [External] Is it possible to set a default QOS per partition?

[Loris Bennett]

Stack Korora <stackkorora at disroot.org> writes:

> On 3/1/21 4:26 PM, Prentice Bisbal wrote:
>> Two things:
>>
>> 1. So your users are okay with specifying a partition, but specifying a QOS is
>> a bridge too far?
>
> *sigh* Yeah. It's been requested several times. I can't defend it, but if it
> makes them happy...then they will find something else to complain
> about. (kidding... mostly :-D )
>
>>
>> 2. Have your job_submit.lua script filter the jobs into the correct QOS. You
>> can check the partition and set the QOS accordingly.
>>
>> First, you need to have this set in your slurm.conf:
>>
>> JobSubmitPlugins=job_submit/lua
>>
>> But I'm pretty sure that's the default setting.
>>
>> Since it looks like your partitions and corresponding QOSes have the same
>> names, you can just add this line to the slurm_job_submit function body in
>> your job_submit.lua script:
>>
>>   job_desc.qos = job_desc.partition
>>
>> And voila! Problem solved.
>>
>> After editing job_submit.lua, you'll need to restart slurmctld for the changes
>> to take effect. Also, it's a good idea to 'tail -f' slurmctld.log while
>> restarting - any errors with the syntax will be printed there, and if there's
>> any errors in that file, slumctld won't start. 
>
> Thank you! A great idea. I will give this a try.

I have often come across the idea of assigning QOS automatically on the
basis of other job parameters, but have never really understood the use
case.  We use QOS to allow priority to be increased for a limit amount
of resources:

  $ sqos
        Name   Priority     MaxWall MaxJobs MaxSubmit     MaxTRESPU
  ---------- ---------- ----------- ------- --------- -------------
      hiprio     100000    03:00:00      50       100       cpu=160
        prio       1000  3-00:00:00     500      1000       cpu=480
    standard          0 14-00:00:00    3000     10000       cpu=960

and use job_submit.lua to force the users to select a QOS.

Can anyone explain idea behind the automatic QOS assignment approach to
me?

Cheers,

Loris

-- 
Dr. Loris Bennett (Hr./Mr.)
ZEDAT, Freie Universität Berlin         Email loris.bennett at fu-berlin.de