[slurm-users] [EXTERNAL]: Array Job Node Allocation

[Hwa, George]

This is exactly what we want do, spreading array jobs across nodes. The primary motivation for us is to achieve load-balancing.


-----Original Message-----
From: slurm-users [mailto:slurm-users-bounces at lists.schedmd.com] On Behalf Of Emyr James
Sent: Tuesday, March 20, 2018 11:54 PM
To: slurm-users at schedmd.com
Subject: [EXTERNAL]: [slurm-users] Array Job Node Allocation

Dear all,

I would like to be able to have an array job load nodes with a round-robin allocation instead of what seems to be the default method of loading the first node till full before moving on to the next node. Our cluster is used for bioinformatics and jobs tend to be serial high throughput with one or a few threads on a node as opposed to jobs being distributed across nodes. The default whereby nodes are filled sequentially doesn't work well for us given that jobs tend to be i/o bound.

I've seen the thread starting at
https://urldefense.proofpoint.com/v2/url?u=https-3A__groups.google.com_d_msg_slurm-2Dusers_uiKuFF8C-2DkU_mnJ1VcESBwAJ&d=DwICaQ&c=YlQJHsR4e2z6sq5iNUb1-8UzPbEw9OnHGjBjyVyvpyA&r=gGwUuBfaFXnwG8KkT-_AZZgo3stXIzSp-vkxcPEYsSM&m=oZnxTErc-Y8zqbtRy3TYqBA8v-lnjX8U4qKOaV7ugzs&s=XRWiskacNSoo6NL4Xi0R7LrhAnZPeJ-9Vf_vlCJ8KbQ&e= but I can't see the solution mentioned there (periodically setting node weights according to load) working for array jobs as it submits jobs in clumps.

The LLN strategy seems to be what I'm after but as in the thread above I can't get it to work. Has anyone managed to get this working ?

Regards,

Emyr



--
 The Wellcome Sanger Institute is operated by Genome Research  Limited, a charity registered in England with number 1021457 and a  company registered in England with number 2742969, whose registered  office is 215 Euston Road, London, NW1 2BE.