[slurm-users] cpu-bind not used ?

[Ionel GARDAIS]

Hi list,

We are migrating from SGE to Slurm.
I'm having hard-time to make process pinning work.

Whatever --cpu-bind is set to, I can not get the placement I want.
It works as expected for cores, threads, boards and sockets, but as soon as I pass a map_cpu or mask_cpu it goes on its own.
Nodes are 1 board, 2 sockets, 4 cores/socket, 1 thread per core.

Tests were made with Slurm 20.11.0.
I've built 20.11.1 this morning.

Am I missing something ?

$ srun --exclusive --cpu-bind=v,cores -N 1-1 -n 2 testmpi.sh
cpu-bind=MASK - mania-8, task  0  0 [3126]: mask 0x1 set
cpu-bind=MASK - mania-8, task  1  1 [3127]: mask 0x4 set

$ srun --exclusive --cpu-bind=v,threads -N 1-1 -n 2 testmpi.sh
cpu-bind=MASK - mania-8, task  0  0 [3261]: mask 0x1 set
cpu-bind=MASK - mania-8, task  1  1 [3262]: mask 0x4 set

$ srun --exclusive --cpu-bind=v,boards -N 1-1 -n 4 testmpi.sh
cpu-bind=MASK - mania-8, task  0  0 [3515]: mask 0x1 set
cpu-bind=MASK - mania-8, task  1  1 [3516]: mask 0x4 set
cpu-bind=MASK - mania-8, task  3  3 [3518]: mask 0x40 set
cpu-bind=MASK - mania-8, task  2  2 [3517]: mask 0x10 set

$ srun --exclusive --cpu-bind=v,sockets -N 1-1 -n 4 testmpi.sh
cpu-bind=MASK - mania-8, task  0  0 [3646]: mask 0x55 set
cpu-bind=MASK - mania-8, task  1  1 [3647]: mask 0x55 set
cpu-bind=MASK - mania-8, task  2  2 [3648]: mask 0x55 set
cpu-bind=MASK - mania-8, task  3  3 [3649]: mask 0x55 set

$ srun --exclusive --cpu-bind=v,mask_cpu:0xa -N 1-1 -n 2 testmpi.sh
cpu-bind=MASK - mania-8, task  0  0 [2816]: mask 0x5 set
cpu-bind=MASK - mania-8, task  1  1 [2819]: mask 0x5 set

$ srun --exclusive --cpu-bind=v,mask_cpu:0x0 -N 1-1 -n 4 testmpi.sh
cpu-bind=MASK - mania-8, task  0  0 [3800]: mask 0x55 set
cpu-bind=MASK - mania-8, task  1  1 [3801]: mask 0x55 set
cpu-bind=MASK - mania-8, task  3  3 [3803]: mask 0x55 set
cpu-bind=MASK - mania-8, task  2  2 [3802]: mask 0x55 set

$ srun --exclusive --cpu-bind=v,none -N 1-1 -n 4 testmpi.sh
cpu-bind=MASK - mania-8, task  0  0 [5053]: mask 0x55 set
cpu-bind=MASK - mania-8, task  1  1 [5054]: mask 0x55 set
cpu-bind=MASK - mania-8, task  2  2 [5056]: mask 0x55 set
cpu-bind=MASK - mania-8, task  3  3 [5057]: mask 0x55 set

$ srun --exclusive --cpu-bind=v,none,mask_cpu:0x33,0x33,0x33,0x33 -N 1-1 -n 4 testmpi.sh
cpu-bind=MASK - mania-8, task  0  0 [5191]: mask 0x55 set
cpu-bind=MASK - mania-8, task  1  1 [5192]: mask 0x55 set
cpu-bind=MASK - mania-8, task  2  2 [5193]: mask 0x55 set
cpu-bind=MASK - mania-8, task  3  3 [5194]: mask 0x55 set


Thanks,

--

Ionel

--
Beicip-Franlab SA - 232 av. Napoleon Bonaparte - BP 2132-92502 Rueil-Malmaison Cedex
Capital: EUR 6 000 000 - TVA FR 54 679 804 047 - RCS Nanterre 679 804 047
This message and any attachments (the message) are confidential and intended solely for the addressees.
Any unauthorised use, dissemination or reproduction is strictly prohibited.
The sender does not accept liability for any errors or omissions in the contents of this message arising as a result of e-mail transmission.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20201212/6efb01c0/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: slurm.conf
Type: application/octet-stream
Size: 3119 bytes
Desc: slurm.conf
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20201212/6efb01c0/attachment-0002.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: cgroup.conf
Type: application/octet-stream
Size: 55 bytes
Desc: cgroup.conf
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20201212/6efb01c0/attachment-0003.obj>