[slurm-users] Transparently assign different walltime limit to a group of nodes ?

[Jens Dreger]

On Mon, Aug 13, 2018 at 03:28:49PM +0200, Loris Bennett wrote:
> Hi Jens,
> 
> Jens Dreger <jens.dreger at physik.fu-berlin.de> writes:
> 
> > Hi everyone!
> >
> > Is it possible to transparently assign different walltime limits
> > to nodes without forcing users to specify partitions when submitting
> > jobs?
> >
> > Example: let's say I have 10 nodes. Nodes n01-n05 should be available
> > for jobs with a walltime up to 14 days, while n06-n10 should only
> > be used for jobs with a walltime limit less then 1 day. Then as long
> > as nodes n06-n10 have free resources, jobs with walltime <1day should
> > be scheduled to these nodes. If n06-n10 are full, jobs with walltime
> > <1day should start on n01-n05. Users should not have to specify
> > partitions.
> >
> > Would this even be possible to do with just one partition much
> > like nodes with different memory size using weights to fill nodes
> > with less memoery first?
> >
> > Background of this question is that it would be helpfull to be able
> > to lower the walltime for a rack of nodes, e.g. when adding this rack
> > to an existing cluster in order to be able to easily shut down just
> > this rack after one day in case of instabilities. Much like adding
> > N nodes to a cluster without changing anything else and have only
> > jobs with walltime <1day on thiese nodes in the beginning.
> 
> If you just want to reduce the allowed wall-time for a given rack, can't
> you just use a maintenance reservation for the appropriate set of nodes?

I could if I knew that I really want to shut it down at a given time,
but if everything works it should just keep going. What I want is a
solution for "Do not schedule jobs with walltime larger than X on
these nodes" untill I release this limit. And I want to be able to
apply and release this limit without forcing users to change anything
in their scripts. Basically user should never notice any difference.

Regards,

Jens.

-- 
Jens Dreger                      Freie Universitaet Berlin
dreger at physik.fu-berlin.de       Fachbereich Physik - ZEDV
Tel: +49 30 83854774             Arnimallee 14
Fax: +49 30 838454774	         14195 Berlin