Hello,
I am in the process of setting up SLURM to be used in a profiling cluster. The purpose of SLURM is to allow users to submit jobs to be profiled. The latency is a very important aspect of profiling the applications correctly. I was able to leverage cgroupsv2.0 to isolate user.slice from the cores that would be used by SLURM jobs. The issue is that slurmstepd shares the resources with system.slice; I was digging through the code, and I saw that the creation of the scope is here:
https://github.com/SchedMD/slurm/blob/master/src/plugins/cgroup/v2/cgroup_v2...
And I noticed that the slice is hardcoded in the following line:
https://github.com/SchedMD/slurm/blob/master/src/plugins/cgroup/v2/cgroup_v2...
So, my question, now, is about why is the slice hardcoded? What was the reason behind such a decision? I would have thought that the slice chosen would be set through cgroups.conf, instead.
I would like to switch the slice for slurmstepd to a slice other than system.slice; by doing so, I would be able to isolate cores better by making sure that services' processes are isolated from the cores used for SLURM jobs. I can definitely change the defined value in the code and recompile. Are there anything to consider before doing so?
Thanks, Khalid