[slurm-users] Intel MPI startup
cap at nsc.liu.se
Tue Apr 30 11:59:54 UTC 2019
On Mon, 29 Apr 2019 15:47:49 +0000
Michael Robbert <mrobbert at mines.edu> wrote:
> I was curious what startup method other sites are using with Intel
We (nsc.liu.se nat. level academic HPC) have used IntelMPI+slurm as a
primary platform for several generations.
Our setup to this day is "mpiexec.hydra -bootstrap slurm".
* it works well
* it binds well including with slurm cgroups
* it performs well
For those of you not familiar with the approach it basically goes like
site wrapper -> mpirun -> mpiexec.hydra -> srun pmi_proxy
That is, the intel pmi_proxy ends up under the slurmstepd and then
launches and pinns the ranks/threads.
When I say it performs well I include experiences up to configs such as
3200 ranks with 16 threads each and 64000 ranks pure MPI.
Disclaimer: IntelMPI-2019 is "problematic" and not yet successfully
More information about the slurm-users