[slurm-users] Array jobs vs. many jobs

[Christopher Samuel]

Hi Ryan,

On 11/22/19 12:18 PM, Ryan Novosielski wrote:

> Quick question that I'm not sure how to find the answer to otherwise: do array jobs have less impact on the scheduler in any way than a whole long list of jobs run the more traditional way? Less startup overhead, anything like that?

Slurm will represent the whole job array as a single entity until it 
needs to create elements for scheduling purposes (ageing if you limit 
the number of jobs that can accrue time, or just starting them up).

So if you have a 10,000 element job array it uses the same amount of 
memory as 1 job until things start to happen to it.

It's a big win if you've got a workload that can take advantage of it.

All the best,
Chris
-- 
   Chris Samuel  :  http://www.csamuel.org/  :  Berkeley, CA, USA