[slurm-users] NIC gres types and lack of device files?

[Geert Geurts]

Hi Chris,
I have no experience with gres nic config, but can't you use /sys/class/net instead of /dev?
I think there is also a link to the pci device..

Regards,
Geert


________________________________
From: Chris Samuel <samuel at unimelb.edu.au>
Sent: Nov 13, 2017 8:40 AM
To: slurm-users at schedmd.com
Subject: [slurm-users] NIC gres types and lack of device files?

Hi folks,

Just curious if anyone out there is currently using the "nic" gres type and
how do you deal with the fact that the Files= entry in gres.conf has to refer
to an existing device when network devices generally don't appear under /dev?

Do you just hack with with "touch" or "mkdev" ?

The bug that introduced the code was:

https://bugs.schedmd.com/show_bug.cgi?id=1718#c3

and the example given was:

Name=nic File=/dev/mlx4_0 CPUs=0-1
Name=nic File=/dev/mlx4_1 CPUs=2-3

but nothing like those names will exist with Mellanox OFED.

[root at spartan-gpgpu009 ~]# ibdev2netdev
mlx5_0 port 1 ==> p1p1 (Up)
mlx5_1 port 1 ==> p1p2 (Up)
mlx5_2 port 1 ==> p2p1 (Down)
mlx5_3 port 1 ==> p2p2 (Down)
[root at spartan-gpgpu009 ~]# find /dev/ -name mlx*
[root at spartan-gpgpu009 ~]# find /dev/ -name p1*
[root at spartan-gpgpu009 ~]#


cheers,
Chris
--
 Christopher Samuel        Senior Systems Administrator
 Melbourne Bioinformatics - The University of Melbourne
 Email: samuel at unimelb.edu.au Phone: +61 (0)3 903 55545