<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body>
<br>
Can someone please clarify the "best practices" for building OpenMPI
compatible with Slurm?<br>
<br>
<a class="moz-txt-link-freetext" href="https://slurm.schedmd.com/mpi_guide.html#open_mpi">https://slurm.schedmd.com/mpi_guide.html#open_mpi</a> tells me what I
_can_ do but I'm unclear as to what I _should_ do. <br>
<br>
I've built OpenMPI 4.1.5 with: --with-pmix
--with-libevent=internal --with-hwloc=internal --with-slurm. If I
run an MPI program on my cluster (slurm 18.08.8) with "srun -N2 foo"
it seems to work fine. (slurm.conf has MpiDefault=pmix).<br>
<br>
If I "srun --mpi=openmpi -N2 foo" it chokes with:<br>
<br>
<blockquote>OPAL_ERROR: Unreachable in file
../../../../../opal/mca/pmix/pmix3/pmix3x_client.c at line 112<br>
-------------------------------------------------------------------------------------------------------------------<br>
This application appears to have been direct launched using
"srun",<br>
but OMPI was not build with SLURM's PMI support and therefore
cannot<br>
execute. There are several options for building PMI support under<br>
SLURM, depending upon the SLURM version you are using:<br>
<br>
version 16.05 or later: you can use SLURM's PMIx support. THis<br>
require that you configure and uild SLURM --with-pmix.<br>
.<br>
.<br>
.<br>
</blockquote>
<br>
So I guess the question is, what is the "right" way to build OpenMPI
with Slurm. Is the fact that my non-Slurm pmix works "correct" or
am I just getting lucky that the various software I have just
happens to be compatible. If I build OpenMPI am I supposed to use
Slurm's pmix/libevent/hwloc or is that optional. If it's optional
when/why might I choose to do so. If I need Slurm's versions is
there some way to find which pmix/libevent/hwloc my current Slurm
install is using? Note: my sysadmins are not going to be helpful as
they think Slurm 18 and OpenMPI 4.0.2a is adequate for users' needs
:^(. <br>
<br>
I like the idea of _not_ tying my OpenMPI to the installed Slurm
just in case our support people ever decide to upgrade system
software.<br>
<br>
Thanks.<br>
<p><br>
</p>
</body>
</html>