How was your binary compiled? If it is dynamically linked, please reply with the ldd listing of the binary ( ldd binary ) Jenny From: S L via slurm-users <slurm-users@lists.schedmd.com> Sent: Tuesday, February 20, 2024 10:55 AM To: slurm-users@lists.schedmd.com Subject: [slurm-users] RHEL 8.9+SLURM-23.11.3+MLNX_OFED_LINUX-23.10-1.1.9.0+ OpenMPI-5.0.2 You don't often get email from slurm-users@lists.schedmd.com<mailto:slurm-users@lists.schedmd.com>. Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> Hi All, We're currently in the process of setting up SLURM on a RHEL 8.9 based cluster. Here's a summary of the steps we've taken so far: Installed MLNX OFED ConnectX-5.2. Compiled and installed PMiX and UCX. Compiled and installed Slurm with PMiX_v4 and UCX support. Compiled OpenMPI with SLURM, PMIx, libevent, and hwloc support. All compute nodes are reachable via the IB network. Problem: While hello world MPI jobs are working fine on multiple nodes, the jobs are not utilizing Infiniband. srun --mpi=pmix -N2 -n2 --ntasks-per-node=2 ./hello > log.out 2>&1 Output from srun --mpi=list: MPI plugin types are... none cray_shasta pmi2 pmix specific pmix plugin versions available: pmix_v4 Could someone please point me in the right direction on how to troubleshoot this issue? Thank you for your assistance. Sudhakar