[slurm-users] one job at a time - how to set?

[Renfro, Michael]

You can adjust or enforce almost anything about a job with a job_submit.lua [1 (search for “JobSubmitPlugins”), 2].

Assuming you want this node in a single partition, you can set ExclusiveUser in a partition definition in slurm.conf. That would at least keep other users off the node, but wouldn’t prevent a single user from running multiple jobs on the node.

Past that, you can force a QOS on the partition [3], and use that to set limits on how many jobs a user can have running [4]. That might be just a MaxJobs=1 for the QOS.

[1] https://slurm.schedmd.com/archive/slurm-15.08.13/slurm.conf.html
[2] https://github.com/SchedMD/slurm/blob/slurm-15-08-13-1/contribs/lua/job_submit.lua
[3] https://slurm.schedmd.com/archive/slurm-15.08.13/qos.html
[4] https://slurm.schedmd.com/archive/slurm-15.08.13/resource_limits.html

> On Apr 29, 2020, at 3:19 PM, Rutger Vos <rutger.vos at naturalis.nl> wrote:
> 
> External Email Warning
> This email originated from outside the university. Please use caution when opening attachments, clicking links, or responding to requests.
> Hi Michael,
> 
> thanks very much for your swift reply. So here we would have to convince the users they'd have to specify this when submitting, right? I.e. 'sbatch --exclusive myjob.sh', if I understand correctly. Would there be a way to simply enforce this, i.e. at the slurm.conf level or something?
> 
> Thanks again!
> 
> Rutger
> 
> On Wed, Apr 29, 2020 at 10:06 PM Renfro, Michael <Renfro at tntech.edu> wrote:
> That’s a *really* old version, but https://slurm.schedmd.com/archive/slurm-15.08.13/sbatch.html indicates there’s an exclusive flag you can set.
> 
>> On Apr 29, 2020, at 1:54 PM, Rutger Vos <rutger.vos at naturalis.nl> wrote:
>> .
>> Hi,
>> 
>> for a smallish machine that has been having degraded performance we want to implement a policy where only one job (submitted with sbatch) is allowed to run and any others submitted after it are supposed to wait in line.
>> 
>> I assumed this was straightforward but I can't seem to figure it out. Can I set that up in slurm.conf or in some other way? Thank you very much for your help. BTW we are running slurm 15.08.7 if that is at all relevant.
>> 
>> Best wishes,
>> 
>> Dr. Rutger A. Vos
>> Researcher / Bioinformatician
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> +31717519600 - +31627085806
>> rutger.vos at naturalis.nl - www.naturalis.nl
>> Darwinweg 2, 2333 CR Leiden
>> Postbus 9517, 2300 RA Leiden
>> 
>>  
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
> 
> 
> -- 
> 
> Met vriendelijke groet,
> 
> Dr. Rutger A. Vos
> Researcher / Bioinformatician
> 
> 
> 
> 
> 
> 
> 
> +31717519600 - +31627085806
> rutger.vos at naturalis.nl - www.naturalis.nl
> Darwinweg 2, 2333 CR Leiden
> Postbus 9517, 2300 RA Leiden
> 
>  
> 
> 
> 
> 
> 
> 
> 
> 
>