[slurm-users] srun and --cpus-per-task
hermann.schwaerzler at uibk.ac.at
Fri Mar 25 09:53:59 UTC 2022
ok, I see. I also just re-read the documentation to find this in the
description of the "CPUs" option:
"This can be useful when you want to schedule only the cores on a
hyper-threaded node. If CPUs is omitted, its default will be set equal
to the product of Boards, Sockets, CoresPerSocket, and ThreadsPerCore."
Do you happen to know if there is a difference between setting CPUs
explicitely like you do it and not setting it but using "ThreadsPerCore=1"?
My guess is that there is no difference and in both cases only the
physical cores are "handed out to jobs". But maybe I am wrong?
On 3/25/22 8:49 AM, Bjørn-Helge Mevik wrote:
> For what it's worth, we have a similar setup, with one crucial
> difference: we are handing out physical cores to jobs, not hyperthreads,
> and we are *not* seeing this behaviour:
> $ srun --cpus-per-task=1 -t 10 --mem-per-cpu=1g -A nn9999k -q devel echo foo
> srun: job 5371678 queued and waiting for resources
> srun: job 5371678 has been allocated resources
> $ srun --cpus-per-task=3 -t 10 --mem-per-cpu=1g -A nn9999k -q devel echo foo
> srun: job 5371680 queued and waiting for resources
> srun: job 5371680 has been allocated resources
> We have
> and node definitions like
> NodeName=DEFAULT CPUs=40 Sockets=2 CoresPerSocket=20 ThreadsPerCore=2 RealMemory=182784 Gres=localscratch:330G Weight=1000
> (so we set CPUs to the number of *physical cores*, not *hyperthreads*).
More information about the slurm-users