[slurm-users] giving smaller jobs higher priority

[Nicholas McCollum]

You can use the job_submit.lua plugin to modify the nice value of jobs, but I
caution against modifying with a negative value.  I was previously modifying
the values for jobs to give multi-node jobs a boost, but ran into a bug
that would assign comically priority values to jobs.

function slurm_job_submit(job_desc, part_list, submit_uid)
  local job_requested_cpus = job_desc.min_cpus
  local job_min_nodes = job_desc.min_nodes
  
  if job_requested_cpus == 1 then
    job_desc.nice = "8000"
  end
  if job_requested_cpus > 1 and job_min_nodes > 1 then
    job_desc.nice = "9000"
  end
  
  return slurm.SUCCESS
end

---

Nicholas McCollum
HPC Systems Administrator
Alabama Supercomputer Authority


On Wed, Nov 22, 2017 at 01:14:16PM -0500, Satrajit Ghosh wrote:
> hi sam,
> 
> thanks for that pointer. we already have:
> 
> PriorityFavorSmall=YES
> 
> PriorityFlags=SMALL_RELATIVE_TO_TIME
> 
> but small jobs still seem to hold up. that's because cores more than nodes
> are important in our usage scenarios. 99% of jobs request one node
> 
> so ideally we wanted to assign negative weights to TRES but that didn't
> work for priority.
> 
> cheers,
> 
> satra
> 
> On Wed, Nov 22, 2017 at 12:13 PM, Sam Gallop (NBI) <sam.gallop at nbi.ac.uk>
> wrote:
> 
> > Hi Satra,
> >
> >
> >
> > Have a look at PriorityFavorSmall (in slurm.conf). It may fit your needs.
> > Not used it myself, so I'm not able to say if it'll do exactly what you're
> > after.
> >
> >
> >
> > *---*
> >
> > *Samuel Gallop*
> >
> >
> >
> > *From:* slurm-users [mailto:slurm-users-bounces at lists.schedmd.com] *On
> > Behalf Of *Satrajit Ghosh
> > *Sent:* 22 November 2017 16:54
> > *To:* slurm-users at lists.schedmd.com
> > *Subject:* [slurm-users] giving smaller jobs higher priority
> >
> >
> >
> > slurm has a way of giving larger jobs more priority. is it possible to do
> > the reverse?
> >
> >
> >
> > i.e., is there a way to configure priority to give smaller jobs (use less
> > resources) higher priority than bigger ones?
> >
> >
> > cheers,
> >
> > satra
> >
> > resources: can be a weighted combination depending on system resources
> > available:
> >
> >
> >
> > w1*core + w2*memory + w3*time + w4*gpu
> >
> >
> >
> > where core, memory, time, gpu are those requested by the job, and w1-4 are
> > determined by system resources/group allocations.
> >