[slurm-users] Ideal NFS exported StateSaveLocation size.
Ole Holm Nielsen
Ole.H.Nielsen at fysik.dtu.dk
Mon Oct 24 07:32:03 UTC 2022
On 10/24/22 06:12, Richard Chang wrote:
> Is there a thumb rule for the size of the directory that is NFS exported,
> and to be used as StateSaveLocation.
> I have a two node Slurmctld setup and both will mount an NFS exported
> directory as the state save location.
It is definitely a BAD idea to store Slurm StateSaveLocation on a slow NFS
directory! SchedMD recommends to use local NVME or SSD disks because
there will be many IOPS to this file system!
I recommend you to read "Field Notes 6: From The Frontlines of Slurm
Support", Jason Booth, SchedMD available from
https://slurm.schedmd.com/publications.html. Read the Hardware pages
18-20 which recommend:
Fast path to the StateSaveLocation
■ IOPS this filesystem can sustain is a major bottleneck to job throughput
● At least 2 directories and two files created per job
● The corresponding unlink() calls will add to the load
More information about the slurm-users