[slurm-users] [EXT] Building Slurm with UCX support

[Ozeryan, Vladimir]

I am not sure about the rest of the Slurm world, but since I will most likely update OpenMPI more often than Slurm, I've configured and built OpenMPI with UCX and Slurm support and I think they are both default unless you specify "--without" option. Works great so far!

-----Original Message-----
From: slurm-users <slurm-users-bounces at lists.schedmd.com> On Behalf Of Matthias Leopold
Sent: Wednesday, January 12, 2022 11:54 AM
To: Slurm User Community List <slurm-users at lists.schedmd.com>
Subject: [EXT] [slurm-users] Building Slurm with UCX support

APL external email warning: Verify sender slurm-users-bounces at lists.schedmd.com before clicking links or attachments 

Hi,

I'm compiling Slurm with ansible playbooks from NVIDIA deepops framework (https://github.com/NVIDIA/deepops). I'm trying to add UCX support. How can I tell if UCX is actually included in the resulting binaries (without actually using Slurm)? I was looking at executables and *so files with ldd, but found no reference to the UCX installation in /usr/lib/ucx.

Background:
- I'm struggling with the build system using a non-existent path (PMIXP_UCX_LIBPATH=\"/usr/lib64\"). The last ugly hack was to create a symlink from /usr/lib/ucx to /usr/lib64/ucx
- I can't easily test actual operation of MPI with UCX because I'm on a limited test/dev system and (frankly) because I'm not yet a MPI expert

The configure string used is:
./configure --prefix=/usr/local --disable-dependency-tracking --disable-debug --disable-x11 --enable-really-no-cray --enable-salloc-kill-cmd --with-hdf5=no --sysconfdir=/etc/slurm --enable-pam --with-pam_dir=/lib/x86_64-linux-gnu/security
--with-shared-libslurm --without-rpath --with-pmix=/opt/deepops/pmix --with-hwloc=/opt/deepops/hwloc --with-ucx=/usr

thx
Matthias