<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Very true - one of the risks with installing from packages. However, be aware that slurm 17.02 doesn’t support PMIx v2.0, and so this combination isn’t going to work anyway.<div class=""><br class=""></div><div class="">If you want PMIx v2.x, then you need to pair it with SLURM 17.11.</div><div class=""><br class=""></div><div class="">Ralph</div><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On Nov 28, 2017, at 2:32 PM, Philip Kovacs <<a href="mailto:pkdevel@yahoo.com" class="">pkdevel@yahoo.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div class=""><div style="background-color: rgb(255, 255, 255); font-family: 'Helvetica Neue', Helvetica, Arial, 'Lucida Grande', sans-serif; font-size: 13px;" class=""><div id="yiv3332494612" class=""><div id="yui_3_16_0_ym19_1_1511904995913_19392" class=""><div style="background-color: rgb(255, 255, 255); font-size: 13px;" id="yui_3_16_0_ym19_1_1511904995913_19391" class=""><div id="yiv3332494612yui_3_16_0_ym19_1_1511904995913_13882" style="font-family: "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif;" class=""><span id="yiv3332494612yui_3_16_0_ym19_1_1511904995913_14678" class="">This issue is that pmi 2.0+ provides a "backward compatibility" feature, enabled by default, which installs</span></div><div dir="ltr" id="yiv3332494612yui_3_16_0_ym19_1_1511904995913_13882" style="font-family: "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif;" class=""><span id="yui_3_16_0_ym19_1_1511904995913_19393" class="">both libpmi.so and libpmi2.so in addition to libpmix.so. The route with the least friction for you would probably</span></div><div dir="ltr" id="yiv3332494612yui_3_16_0_ym19_1_1511904995913_13882" style="font-family: "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif;" class=""><span id="yui_3_16_0_ym19_1_1511904995913_19394" class="">be to uninstall pmix, then install slurm normally, letting it install its libpmi and libpmi2. Next configure and compile</span></div><div dir="ltr" id="yiv3332494612yui_3_16_0_ym19_1_1511904995913_13882" style="font-family: "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif;" class=""><span id="yui_3_16_0_ym19_1_1511904995913_20421" class="">a custom pmix with that backward </span>feature _disabled_, so it only installs libpmix.so. Slurm will "see" the pmix library</div><div dir="ltr" id="yiv3332494612yui_3_16_0_ym19_1_1511904995913_13882" style="font-family: "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif;" class="">after you install it and load it via its plugin when you use --mpi=pmix. Again, just use the Slurm pmi and pmi2 and </div><div dir="ltr" id="yiv3332494612yui_3_16_0_ym19_1_1511904995913_13882" style="font-family: "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif;" class="">install pmix separately with the backward compatible option disabled.</div><div dir="ltr" id="yiv3332494612yui_3_16_0_ym19_1_1511904995913_13882" style="font-family: "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif;" class=""><span class=""><br class=""></span></div><div dir="ltr" id="yiv3332494612yui_3_16_0_ym19_1_1511904995913_13882" style="font-family: "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif;" class=""><span id="yui_3_16_0_ym19_1_1511904995913_20590" class="">There is a packaging issue there in which two packages are trying to install their own versions of the same files. </span></div><div dir="ltr" id="yiv3332494612yui_3_16_0_ym19_1_1511904995913_13882" style="font-family: "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif;" class=""><span class="">That s</span>hould be brought to attention of the packages. Meantime you can work around it.</div><div dir="ltr" id="yiv3332494612yui_3_16_0_ym19_1_1511904995913_13882" style="font-family: "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif;" class=""><span class=""><br class=""></span></div><div dir="ltr" id="yiv3332494612yui_3_16_0_ym19_1_1511904995913_13882" style="font-family: "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif;" class=""><span class="">For PMIX:</span></div><div dir="ltr" id="yiv3332494612yui_3_16_0_ym19_1_1511904995913_13882" style="font-family: "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif;" class=""><span class=""><br class=""></span></div><div dir="ltr" id="yiv3332494612yui_3_16_0_ym19_1_1511904995913_13882" style="" class=""><font face="Courier New, courier, monaco, monospace, sans-serif" id="yui_3_16_0_ym19_1_1511904995913_20598" class="">./configure --disable-pmi-backward-compatibility // ... etc ...</font></div><div class="yiv3332494612qtdSeparateBR" id="yui_3_16_0_ym19_1_1511904995913_19395" style="font-family: "Helvetica Neue", Helvetica, Arial, "Lucida Grande", sans-serif;"><br clear="none" class=""></div></div></div></div><div class=".yiv3332494612yahoo_quoted"> <div style="font-family:Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:13px;" class=""> <div style="font-family:HelveticaNeue, Helvetica Neue, Helvetica, Arial, Lucida Grande, sans-serif;font-size:16px;" class=""> <div class="qtdSeparateBR"><br class=""><br class=""></div><div class="yiv3332494612yqt7102444885" id="yiv3332494612yqtfd45512"><div dir="ltr" class=""><font size="2" face="Arial" class=""> On Tuesday, November 28, 2017 4:44 PM, Artem Polyakov <<a href="mailto:artpol84@gmail.com" class="">artpol84@gmail.com</a>> wrote:<br clear="none" class=""></font></div> <br clear="none" class=""><br clear="none" class=""> <div class="yiv3332494612y_msg_container"><div id="yiv3332494612" class=""><div class=""><div dir="ltr" class="">Hello, Paul<div class=""><br clear="none" class=""></div><div class="">Please see below.<br clear="none" class=""><div class="yiv3332494612gmail_extra"><br clear="none" class=""><div class="yiv3332494612gmail_quote">2017-11-28 13:13 GMT-08:00 Paul Edmon <span dir="ltr" class=""><<a rel="nofollow" shape="rect" ymailto="mailto:pedmon@cfa.harvard.edu" target="_blank" href="mailto:pedmon@cfa.harvard.edu" class="">pedmon@cfa.harvard.edu</a>></span>:<br clear="none" class=""><blockquote class="yiv3332494612gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;">So in an effort to future proof ourselves we are trying to build Slurm against PMIx, but when I tried to do so I got the following:<br clear="none" class="">
<br clear="none" class="">
Transaction check error:<br clear="none" class="">
file /usr/lib64/libpmi.so from install of slurm-17.02.9-1fasrc02.el7.cen tos.x86_64 conflicts with file from package pmix-2.0.2-1.el7.centos.x86_64<br clear="none" class="">
file /usr/lib64/libpmi2.so from install of slurm-17.02.9-1fasrc02.el7.cen tos.x86_64 conflicts with file from package pmix-2.0.2-1.el7.centos.x86_64<br clear="none" class="">
<br clear="none" class="">
This is with compiling Slurm with the --with-pmix=/usr option. A few things:<br clear="none" class="">
<br clear="none" class="">
1. I'm surprised when I tell it to use PMIx it still builds its own versions of libpmi and pmi2 given that PMIx handles that now.<br clear="none" class=""></blockquote><div class=""><br clear="none" class=""></div><div class="">PMIx is a plugin and from multiple perspectives it makes sense to keep the other versions available (i.e. backward compat or perf comparison) </div><div class=""> </div><blockquote class="yiv3332494612gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;">
<br clear="none" class="">
2. Does this mean I have to install PMIx in a nondefault location? If so how does that work with user build codes? I'd rather not have multiple versions of PMI around for people to build against.<br clear="none" class=""></blockquote><div class="">When we introduced PMIx it was in the beta stage and we didn't want to build against it by default. Now it probably makes sense to assume --with-pmix by default.</div><div class="">I'm also thinking that we might need to solve it at the packagers level by distributing "slurm-pmix" package that is builded and depends on the pmix package that is currently shipped with particular Linux distro.</div><div class=""> </div><blockquote class="yiv3332494612gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;">
<br clear="none" class="">
3. What is the right way of building PMIx and Slurm such that they interoperate properly?<br clear="none" class=""></blockquote><div class="">As for now it is better to have a PMIx installed in the well-known location. And then build your MPIs or other apps against this PMIx installation.</div><div class="">Starting (I think) from PMIx v2.1 we will have a cross-version support that will give some flexibility about what installation to use with application,</div><div class=""> </div><blockquote class="yiv3332494612gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;">
<br clear="none" class="">
Suffice it to say little to no documentation exists on how to properly this, so any guidance would be much appreciated.</blockquote><div class="">Indeed we have some problems with the documentation as PMIx technology is relatively new. Hopefully we can fix this in near future.</div><div class="">Being the original developer of the PMIx plugin I'll be happy to answer any questions and help to resolve the issues.</div><div class="yiv3332494612yqt4550724288" id="yiv3332494612yqtfd41798"><div class=""><br clear="none" class=""></div><div class=""><br clear="none" class=""></div><div class=""> </div></div><blockquote class="yiv3332494612gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex;"><span class="yiv3332494612gmail-HOEnZb"><font color="#888888" class=""></font></span><div class="yiv3332494612yqt4550724288" id="yiv3332494612yqtfd43154"><br clear="none" class="">
<br clear="none" class="">
-Paul Edmon-</div><br clear="none" class="">
<br clear="none" class="">
<br clear="none" class="">
</blockquote></div><br clear="none" class=""><br clear="all" class=""><div class=""><br clear="none" class=""></div>-- <br clear="none" class=""><div class="yiv3332494612gmail_signature">С Уважением, Поляков Артем Юрьевич<br clear="none" class="">Best regards, Artem Y. Polyakov</div><div class="yiv3332494612yqt4550724288" id="yiv3332494612yqtfd01302">
</div></div></div></div></div></div><br clear="none" class=""><br clear="none" class=""></div> </div></div><div class="yiv3332494612yqt7102444885" id="yiv3332494612yqtfd28013"> </div></div><div class="yiv3332494612yqt7102444885" id="yiv3332494612yqtfd78525"> </div></div></div></div></div></blockquote></div><br class=""></div></body></html>