[slurm-users] cpu allocation on a NUMA node

[Lincy]

Hi,
   Just for later reference, it turns out I have only to add thing like
this in the submission script:
###SBATCH --ntasks-per-socket=4
export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK}
export OMP_PROC_BIND="spread"      ## make cores in order inside a task
srun --cpu-bind=v,cores -m block:block ./code

Thanks,
Chris

On Fri, Apr 19, 2019 at 9:22 AM Lincy <lincy.tw at gmail.com> wrote:

>  Hi all,
>           I have a cpu allocation problem on the 2 socket 18-core machine,
> with CPUSpecList=17,35 reserved.
> Ideally, if I submit a job with two 17-core tasks in a node, it allocates
> each task on a single NUMA node.
>
> # *srun -N1 -n2 --cpus-per-task=**17**  numactl  -s *physcpubind: *0 1 2
> 3 4 5 6 7 8 9 10 11 12 13 14 15 16* cpubind: 0 membind: 0 1 physcpubind: *18
> 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34* cpubind: 1 membind: 0 1
> However, a job with two 16-core tasks show a bad allocation (cover two
> numa nodes) even if I enforce it by --cpu-bind=mask_cpu:0xffff,0x3fffc0000
>
> # *srun -N1 -n2 --cpus-per-task=**16**  numactl  -s*, physcpubind: *0 1 2
> 3 4 5 6 7 8 9 10 11 12 13 14 15* cpubind: 0 membind: 0 1 physcpubind: *16
> 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32* cpubind: 0 1 membind: 0 1
> How should I allocate it on 0-15, 18-33 for example?
>
> Many thanks,
> Chris
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20190420/7dae9160/attachment.html>