[slurm-users] Generating OPA topology.conf

[Jeffrey Frey]

I'll preface this by saying that I'm no expert in CMake, so there may be better ways to construct the CMakeLists.txt in this instance.  Anyone who is, in fact, better-versed in CMake authoring should feel free to fork and eventually send a pull request for a CMake redo.


That being said, I went back and redid the CMakeLists.txt to act (I think!) a bit more logically and included checks for all necessary opamgt functions (to avoid the problem Ole noted in his build).



> On Jun 14, 2018, at 4:07 AM, Marcus Wagner <wagner at itc.rwth-aachen.de> wrote:
> 
> Hi Ole, hi Jeffrey.
> 
> Thanks Jeffrey for that tool, for me it is working. I changed a little bit the CMakeLists.txt such that slurm can be found also in non standard install paths ;)
> 
> replaced
> SET (SLURM_PREFIX "/usr/local" CACHE PATH "Directory in which SLURM is installed.")
> with
> FIND_PATH(SLURM_PREFIX NAMES sbatch)
> 
> and changed
> FIND_PATH(SLURM_INCLUDE_DIR NAMES slurm/slurm.h)
> FIND_LIBRARY(SLURM_LIBRARY NAMES libslurm.so)
> to
> FIND_PATH(SLURM_INCLUDE_DIR NAMES slurm/slurm.h HINTS ${SLURM_PREFIX}/../include)
> FIND_LIBRARY(SLURM_LIBRARY NAMES libslurm.so HINTS ${SLURM_PREFIX}/../lib64)
> 
> @Ole,
> 
> I also got first compile errors:
> 
> CMakeFiles/opa2slurm.dir/opa2slurm.c.o: In function `main':
> opa2slurm.c:(.text+0x15ee): undefined reference to `omgt_status_totext'
> opa2slurm.c:(.text+0x16df): undefined reference to `omgt_status_totext'
> 
> I used the libopamgt coming with CentOS 7.5 (10.5.0.0.140-2.el7), after changing this to version 10.6.1.0-2.el7 everything went through:
> 
> $> cmake . && make
> -- The C compiler identification is GNU 4.8.5
> -- Check for working C compiler: /usr/bin/gcc
> -- Check for working C compiler: /usr/bin/gcc -- works
> -- Detecting C compiler ABI info
> -- Detecting C compiler ABI info - done
> -- Found SLURM: /opt/slurm/lib64/libslurm.so
> -- Found OPA: /usr/lib/libopamgt.so
> -- Configuring done
> -- Generating done
> -- Build files have been written to: /home/mw445520/git/opa2slurm
> Scanning dependencies of target opa2slurm
> [ 25%] Building C object CMakeFiles/opa2slurm.dir/ptr_list.c.o
> [ 50%] Building C object CMakeFiles/opa2slurm.dir/opa_host.c.o
> [ 75%] Building C object CMakeFiles/opa2slurm.dir/opa_switch.c.o
> [100%] Building C object CMakeFiles/opa2slurm.dir/opa2slurm.c.o
> Linking C executable opa2slurm
> [100%] Built target opa2slurm
> 
> the only additional prerequisites for me have been the libopamgt and libopamgt-devel packages.
> 
> So, since you seem to use the same version as me, I'm not sure why you have these linking problems :/
> 
> 
> Best
> Marcus
> 
> On 06/14/2018 09:17 AM, Ole Holm Nielsen wrote:
>> Hi Jeffrey,
>> 
>> On 06/13/2018 10:35 PM, Jeffrey Frey wrote:
>>> Intel's OPA doesn't include the old IB net discovery library/API; instead, they have their own library to enumerate nodes, links, etc.   I've started a rewrite of ye olde "ib2slurm" utility to make use of Intel's new enumeration library.
>>> 
>>> 
>>> https://gitlab.com/jtfrey/opa2slurm
>> 
>> Thanks for sharing this tool!  We have an OPA fabric, and I'd like to try out your tool to generate topology.conf.  Unfortunately I'm unable to build opa2slurm because of missing headers and libraries:
>> 
>> 1. The RPM package opa-libopamgt-devel is required for header files. This RPM may or may not be installed from the IntelOPA-Basic.RHEL74-x86_64.10.6.1.0.2.tgz tar-ball.  I had a node where the OPA FM software was installed, including the required header files.
>> 
>> 2. After the CMake configuration the make fails:
>> 
>> $ make
>> Scanning dependencies of target opa2slurm
>> [ 20%] Building C object CMakeFiles/opa2slurm.dir/ptr_list.c.o
>> [ 40%] Building C object CMakeFiles/opa2slurm.dir/opa_host.c.o
>> [ 60%] Building C object CMakeFiles/opa2slurm.dir/opa_switch.c.o
>> [ 80%] Building C object CMakeFiles/opa2slurm.dir/opa2slurm.c.o
>> [100%] Linking C executable opa2slurm
>> CMakeFiles/opa2slurm.dir/opa2slurm.c.o: In function `opa_link_speed':
>> opa2slurm.c:(.text+0xe7): undefined reference to `omgt_sa_get_portinfo_records'
>> opa2slurm.c:(.text+0x1ae): undefined reference to `omgt_sa_free_records'
>> CMakeFiles/opa2slurm.dir/opa2slurm.c.o: In function `main':
>> opa2slurm.c:(.text+0x734): undefined reference to `omgt_open_port'
>> opa2slurm.c:(.text+0x791): undefined reference to `omgt_open_port_by_num'
>> opa2slurm.c:(.text+0x7e1): undefined reference to `omgt_open_port_by_guid'
>> opa2slurm.c:(.text+0x88a): undefined reference to `omgt_sa_get_node_records'
>> opa2slurm.c:(.text+0xa12): undefined reference to `omgt_sa_free_records'
>> opa2slurm.c:(.text+0xa5d): undefined reference to `omgt_sa_get_node_records'
>> opa2slurm.c:(.text+0xbd8): undefined reference to `omgt_sa_free_records'
>> opa2slurm.c:(.text+0xc19): undefined reference to `omgt_sa_get_link_records'
>> opa2slurm.c:(.text+0xee5): undefined reference to `omgt_sa_free_records'
>> opa2slurm.c:(.text+0x15e9): undefined reference to `omgt_status_totext'
>> opa2slurm.c:(.text+0x1623): undefined reference to `omgt_sa_free_records'
>> opa2slurm.c:(.text+0x163e): undefined reference to `omgt_sa_free_records'
>> opa2slurm.c:(.text+0x1659): undefined reference to `omgt_sa_free_records'
>> opa2slurm.c:(.text+0x169a): undefined reference to `omgt_close_port'
>> opa2slurm.c:(.text+0x16d6): undefined reference to `omgt_status_totext'
>> opa2slurm.c:(.text+0x1717): undefined reference to `omgt_sa_free_records'
>> opa2slurm.c:(.text+0x1732): undefined reference to `omgt_sa_free_records'
>> opa2slurm.c:(.text+0x174d): undefined reference to `omgt_sa_free_records'
>> opa2slurm.c:(.text+0x178e): undefined reference to `omgt_close_port'
>> collect2: error: ld returned 1 exit status
>> make[2]: *** [opa2slurm] Error 1
>> make[1]: *** [CMakeFiles/opa2slurm.dir/all] Error 2
>> make: *** [all] Error 2
>> 
>> Could you kindly add detailed building instructions including software prerequisites to your Gitlab page?
>> 
>> Thanks a lot,
>> Ole
>> 
> 
> -- 
> Marcus Wagner, Dipl.-Inf.
> 
> IT Center
> Abteilung: Systeme und Betrieb
> RWTH Aachen University
> Seffenter Weg 23
> 52074 Aachen
> Tel: +49 241 80-24383
> Fax: +49 241 80-624383
> wagner at itc.rwth-aachen.de
> www.itc.rwth-aachen.de
> 
> 


::::::::::::::::::::::::::::::::::::::::::::::::::::::
Jeffrey T. Frey, Ph.D.
Systems Programmer V / HPC Management
Network & Systems Services / College of Engineering
University of Delaware, Newark DE  19716
Office: (302) 831-6034  Mobile: (302) 419-4976
::::::::::::::::::::::::::::::::::::::::::::::::::::::