[slurm-users] Running job using our serial queue

Marcus Wagner wagner at itc.rwth-aachen.de
Tue Nov 5 07:47:57 UTC 2019


Hi David,

doing it the way you do it, is the same way, we do it.

When the Matlab job asks for one CPU, it only gets on CPU this way. That 
means, that all the processes are bound to this one CPU. So 
(theoretically) the user is just disturbing himself, if he uses more.

But especially Matlab, there are more things to do. I t does not suffice 
to add '-singleCompThread' to the commandline. Matlab is not the only 
tool, that tries to use all cores, it finds on the node.
The same is valid for CPLEX and Gurobi, both often used from Matlab. So 
even, if the user sets '-singleCompThread' for Matlab, that does not 
mean at all, the job is only using one CPU.


Best
Marcus

On 11/4/19 4:14 PM, David Baker wrote:
> Hello,
>
> We decided to route all jobs requesting from 1 to 20 cores to our 
> serial queue. Furthermore, the nodes controlled by the serial queue 
> are shared by multiple users. We did this to try to reduce the level 
> of fragmentation across the cluster -- our default "batch" queue 
> provides exclusive access to compute nodes.
>
> It looks like the downside of the serial queue is that jobs from 
> different users can interact quite badly. To some extent this is an 
> education issue -- for example matlab users need to be told to add the 
> "-singleCompThread" option to their command line. On the other hand I 
> wonder if our cgroups setup is optimal for the serial queue. Our 
> *cgroup.conf* contains...
>
> *CgroupAutomount=yes
> *
> *CgroupReleaseAgentDir="/etc/slurm/cgroup"
> *
> *
> *
> *ConstrainCores=yes
> *
> *ConstrainRAMSpace=yes
> *
> *ConstrainDevices=yes
> *
> *TaskAffinity=no
> *
> *
> *
> *CgroupMountpoint=/sys/fs/cgroup*
>
> The relevant cgroup configuration in the *slurm.conf *is...
> *ProctrackType=proctrack/cgroup
> TaskPlugin=affinity,cgroup*
>
> Could someone please advise us on the required/recommended cgroup 
> setup for the above scenario? For example, should we really set 
> "TaskAffinity=yes"? I assume the interaction between jobs (sometimes 
> jobs can get stalled) is due to context switching at the kernel level, 
> however (apart from educating users) how can we minimise that 
> switching on the serial nodes?
>
> Best regards,
> David
>

-- 
Marcus Wagner, Dipl.-Inf.

IT Center
Abteilung: Systeme und Betrieb
RWTH Aachen University
Seffenter Weg 23
52074 Aachen
Tel: +49 241 80-24383
Fax: +49 241 80-624383
wagner at itc.rwth-aachen.de
www.itc.rwth-aachen.de

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20191105/3e31e0a5/attachment-0001.htm>


More information about the slurm-users mailing list