[slurm-users] Strange error, submission denied

[Henkel, Andreas]

Hi Marcus,

What just came to my mind: if you don’t set —ntasks isn’t the default just 1? All examples I know using ntasks-per-node also set ntasks with ntasks >= ntasks-per-node. 

Best,
Andreas 

> Am 14.02.2019 um 06:33 schrieb Marcus Wagner <wagner at itc.rwth-aachen.de>:
> 
> Hi all,
> 
> I have narrowed this down a little bit.
> 
> the really astonishing thing is, that if I use
> 
> --ntasks=48
> 
> I can submit the job, it will be scheduled onto one host:
> 
>    NumNodes=1 NumCPUs=48 NumTasks=48 CPUs/Task=1 ReqB:S:C:T=0:0:*:*
>    TRES=cpu=48,mem=182400M,node=1,billing=48
> 
> but as soon as I change --ntasks to --ntasks-per-node (which should be the same, as --ntasks=48 schedules onto one host), I get the error:
> 
> sbatch: error: CPU count per node can not be satisfied
> sbatch: error: Batch job submission failed: Requested node configuration is not available
> 
> 
> Is there no one else, who observes this behaviour?
> Any explanations?
> 
> 
> Best
> Marcus
> 
> 
>> On 2/13/19 1:48 PM, Marcus Wagner wrote:
>> Hi all,
>> 
>> I have a strange behaviour here.
>> We are using slurm 18.08.5-2 on CentOS 7.6.
>> 
>> Let me first describe our computenodes:
>> NodeName=ncm[0001-1032]  CPUs=48  Sockets=4 CoresPerSocket=12 ThreadsPerCore=2 RealMemory=185000 Feature=skx8160,hostok,hpcwork                        Weight=10541 State=UNKNOWN
>> 
>> we have the following config set:
>> 
>> $>scontrol show config | grep -i select
>> SelectType              = select/cons_res
>> SelectTypeParameters    = CR_CORE_MEMORY,CR_ONE_TASK_PER_CORE
>> 
>> 
>> So, I have 48 cores on one node. According to the manpage of sbatch, I should be able to do the following:
>> 
>> #SBATCH --ntasks=48
>> #SBATCH --ntasks-per-node=48
>> 
>> But I get the following error:
>> sbatch: error: Batch job submission failed: Requested node configuration is not available
>> 
>> 
>> Has anyone an explanation for this?
>> 
>> 
>> Best
>> Marcus
>> 
> 
> -- 
> Marcus Wagner, Dipl.-Inf.
> 
> IT Center
> Abteilung: Systeme und Betrieb
> RWTH Aachen University
> Seffenter Weg 23
> 52074 Aachen
> Tel: +49 241 80-24383
> Fax: +49 241 80-624383
> wagner at itc.rwth-aachen.de
> www.itc.rwth-aachen.de
> 
>