[slurm-users] sacct: error

[Marcus Wagner]

Hi Chris,

this is not correct. From the slurm.conf manpage:

CPUs:
Number of logical processors on the node (e.g. "2").  CPUs and Boards 
are mutually exclusive. It can be set to the total number of sockets, 
cores or threads. This can be useful when you want to schedule only  
the  cores on a hyper-threaded node.  If CPUs is omitted, it will be set 
equal to the product of Sockets, CoresPerSocket, and ThreadsPerCore.  
The default value is 1.

If one only wants to schedule by cores, even if hyperthreading is 
enabled, you need to set CPUs to Sockets*CoresPerSocket. See also this 
part from SelectTypeParameters:

CR_Core_Memory:
Cores  and  memory  are  consumable resources.  On nodes with 
hyper-threads, each thread is counted as a CPU to satisfy a job's 
resource requirement, but multiple jobs are not allocated threads on the 
samecore.  The count of CPUs allocated to a job may be rounded up to 
account for every CPU on an allocated core.  Setting a value for 
DefMemPerCPU is strongly recommended.
Especially the part "every CPU on an allocated core".

To me it looks like CPUs is the synonym for hardware threads.


Best
Marcus


On 05/05/2018 02:42 PM, Chris Samuel wrote:
> On Saturday, 5 May 2018 2:45:19 AM AEST Eric F. Alemany wrote:
>
>> With Ray suggestion i have a error message for each nodes. Here i am giving
>> you only one error message from a node.
>> sacct: error: NodeNames=radonc01 CPUs=32 doesn't match
>> Sockets*CoresPerSocket*ThreadsPerCore (16), resetting CPUs
>> The interesting thing is if you follow the
>> Sockets*CoresPerSocket*ThreadsPerCore formula 2x8x2 = 32  however look above
>> and it says (16) - Strange, no ?
> No, Slurm is right.   CPUS != threads.  You've got 16 CPU cores, each with 2
> threads.  So in this configuration you can schedule 16 tasks per node and each
> task can use 2 threads.
>
> What does "slurmd -C" say on that node?
>
> All the best,
> Chris

-- 
Marcus Wagner, Dipl.-Inf.

IT Center
Abteilung: Systeme und Betrieb
RWTH Aachen University
Seffenter Weg 23
52074 Aachen
Tel: +49 241 80-24383
Fax: +49 241 80-624383
wagner at itc.rwth-aachen.de
www.itc.rwth-aachen.de