[slurm-users] Array jobs vs. many jobs

[Mark Hahn]

> Quick question that I'm not sure how to find the answer to otherwise: do
>array jobs have less impact on the scheduler in any way than a whole long
>list of jobs run the more traditional way? Less startup overhead, anything
>like that?

kinda sorta.
I think of array jobs as a bit like Python generators: when you ask
it for more items, they get generated dynamically.

when you submit an array job, it is a single entity in the pending queue.
then the scheduler evaluates it, it says "why yes, here's something to run!" -
effectively generating a job.  the trick is: the array job remains
pending util it has generated (forked/buded?) all the array elements.

each array element is a perfectly normal job - has its own jobid,
consumes exactly as many resources and overheads as any other job.

an array job prevents congestion of the pending queue with lots of jobs 
that stand no chance of starting because resources won't be
available for a long time.

so it depends very much on how bursty your workload is: whether you have 
a huge backlog sometimes.  (of course, a huge backlog can't occur unless
the jobs are short.  that is, a huge backlog implies short jobs - and high
throughput, or else your system would get stuck with ridiculous wait times.)

I do not think array jobs are a good excuse to run short jobs,
simply because all short jobs have high (relative) running overhead.
array jobs do nothing more than minimize the scheduler's effort 
in sorting the pending queue.  to some extent, they let you view 
a set of jobs as a unit, but you can also organize sets of jobs 
via jobname.

regards, mark hahn
-- 
operator may differ from spokesperson.	            hahn at mcmaster.ca