[slurm-users] [17.11.1] no good pmi intention goes unpunished

[rhc at open-mpi.org]

Hmmm - I think there may be something a little more subtle here. If you build your app and link it against “libpmi2”, and that library is actually the one from PMIx, then it won’t work with Slurm’s PMI2 plugin because the communication protocols are completely different.

So the fact is that if you want to use PMIx backward compatibility, you (a) need to link against either libpmix or the libpmi libraries we export (they are nothing more than symlinks to libpmix), and (b) specify --mpi=pmix on the srun cmd line.



> On Dec 21, 2017, at 11:44 AM, Philip Kovacs <pkdevel at yahoo.com> wrote:
> 
> OK, so slurm's libpmi2 is a functional superset of the libpmi2 provided by pmix 2.0+.  That's good to know.
> 
> My point here is that, if you use slurm's mpi/pmi2 plugin, regardless of which libpmi2 is installed, 
> slurm or pmix, you will always run the slurm pmi2 code since it is compiled directly into the plugin.
> 
> 
> On Wednesday, December 20, 2017 10:47 PM, "rhc at open-mpi.org" <rhc at open-mpi.org> wrote:
> 
> 
> On Dec 20, 2017, at 6:21 PM, Philip Kovacs <pkdevel at yahoo.com <mailto:pkdevel at yahoo.com>> wrote:
>> 
>> >  -- slurm.spec: move libpmi to a separate package to solve a conflict with the
>> >    version provided by PMIx. This will require a separate change to PMIx as
>> >    well.
>> 
>> I see the intention behind this change since the pmix 2.0+ package provides libpmi/libpmi2
>> and there is a possible (installation) conflict with the Slurm implementation of those libraries.  
>> We've discussed  that issue earlier.
>> 
>> Now, suppose a user installs the pmix versions of libpmi/pmi2 with the expectation that pmi
>> calls will be forwarded to libpmix for greater speed, the so-called "backward compatibility" feature.
>> 
>> Shouldn't the Slurm mpi_pmi2 plugin attempt to link with libpmi2 instead of its internal 
>> implementation of pmi2?  As it stands now, there won't be any forwarding of pmi2 code 
>> to libpmix which I imagine users would expect in that scenario.
> 
> Sadly, it isn’t quite that simple. Most of the standard PMI2 calls are covered by the backward compatibility libraries, so things like MPICH should work out-of-the-box.
> 
> However, MVAPICH2 added a PMI2 extension call to the SLURM PMI2 library that they use and PMIx doesn’t cover (as there really isn’t an easy equivalent, and they called it PMIX_foo which causes a naming conflict), and so they would not work.
> 
> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20171221/a449d3a0/attachment-0001.html>