[slurm-users] one job at a time - how to set?
Carlos Fenoy
minibit at gmail.com
Sat May 2 17:38:42 UTC 2020
Hi,
If you only want 1 job at a time, the best option is to configure
the SelectType plugin to select/linear (
https://slurm.schedmd.com/slurm.conf.html#OPT_select/cons_res). That will
force slurm to allocate full nodes.
Regards,
Carlos
On Thu, Apr 30, 2020 at 7:01 PM Allan Streib <astreib at indiana.edu> wrote:
> Well looking at the current slurm.conf it appears that the name was
> changed, and "Shared" is now called "OverSubscribe" in more modern slurm
> versions. So you might look deeper at what config options are in
> conflict, since with the EXCLUSIVE mode I get one node per job here.
>
> Allan
>
> Allan Streib <astreib at indiana.edu> writes:
>
> > I'm running an even older slurm than you (it does what I need, I am a
> > team of one and I have many things to take care of other than chasing
> > the latest version of every piece of software).
> >
> > Anyway, did you try Shared=EXCLUSIVE in the partition configuration?
> >
> > From the (v14.11.7) slurm.conf man page:
> >
> > Shared Controls the ability of the partition to execute more than
> > one job at a time on each resource [...]
> >
> > EXCLUSIVE Allocates entire nodes to jobs even with select/cons_res
> > configured. Jobs that run in partitions with "Shared=EXCLUSIVE" will
> > have exclusive access to all allocated nodes. (node, socket or core
> > depending upon the value of SelectTypeParameters).
> >
> > Allan
> >
> >
> > Rutger Vos <rutger.vos at naturalis.nl> writes:
> >
> >> I did some tests and this doesn't work with MPI, in the following
> >> sense: the way we were running parallel jobs was by invoking `sbatch
> >> -n4 script.sh` where the script.sh would contain an `mpirun` command
> >> without specifying the number of processors. It would then simply claim
> >> the ones assigned to it by sbatch. With one CPU as per the slurm.conf,
> >> mpirun would only ever get one processor, hence no parallelization.
> >>
> >> The other test I did was to try OverSubscribe=Exclusive on the
> >> partition, but the slurmctld failed to start so there is some
> >> combination of config options that it is unhappy about.
> >>
> >> The workable option I can think of right now is to have everyone run
> >> their jobs with --exclusive as an extra argument.
> >>
> >> On Thu, Apr 30, 2020 at 12:22 PM Rutger Vos <rutger.vos at naturalis.nl>
> >> wrote:
> >>
> >> This seems like the easiest solution, but how would that interact
> >> with MPI jobs? Apologies if these are really basic questions, and
> >> thank you very much for thinking along.
> >>
> >> Rutger
> >>
> >> On Wed, Apr 29, 2020 at 10:25 PM Doug Meyer <dameyer at raytheon.com>
> >> wrote:
> >>
> >> Change node definition in slurm.conf for that one node to 1
> >> CPU.
> >>
> >> Doug Meyer
> >>
> >> From: slurm-users <slurm-users-bounces at lists.schedmd.com> On
> >> Behalf Of Rutger Vos
> >> Sent: Wednesday, April 29, 2020 1:20 PM
> >> To: Slurm User Community List <slurm-users at lists.schedmd.com>
> >> Subject: [External] Re: [slurm-users] one job at a time - how
> >> to set?
> >>
> >> Hi Michael,
> >>
> >> thanks very much for your swift reply. So here we would have to
> >> convince the users they'd have to specify this when submitting,
> >> right? I.e. 'sbatch --exclusive myjob.sh', if I
> >> understand correctly. Would there be a way to simply enforce
> >> this, i.e. at the slurm.conf level or something?
> >>
> >> Thanks again!
> >>
> >> Rutger
> >>
> >> On Wed, Apr 29, 2020 at 10:06 PM Renfro, Michael <
> >> Renfro at tntech.edu> wrote:
> >>
> >> That’s a *really* old version, but
> >>
> https://slurm.schedmd.com/archive/slurm-15.08.13/sbatch.html
> >> indicates there’s an exclusive flag you can set.
> >>
> >> On Apr 29, 2020, at 1:54 PM, Rutger Vos <
> >> rutger.vos at naturalis.nl> wrote:
> >>
> >> .
> >>
> >> Hi,
> >>
> >> for a smallish machine that has been having degraded
> >> performance we want to implement a policy where
> >> only one job (submitted with sbatch) is allowed to run
> >> and any others submitted after it are supposed to wait
> >> in line.
> >>
> >> I assumed this was straightforward but I can't seem to
> >> figure it out. Can I set that up in slurm.conf or in
> >> some other way? Thank you very much for your help. BTW
> >> we are running slurm 15.08.7 if that is at all
> >> relevant.
> >>
> >> Best wishes,
> >>
> >> Dr. Rutger A. Vos
> >>
> >> Researcher / Bioinformatician
> >>
> >> [logo-new]
> >>
> >> [+31717519600 - +31627085806 ]
> >>
> >> [rutger.vos at naturalis.nl - www.naturalis.nl]
> >>
> >> [Darwinweg 2, 2333 CR Leiden]
> >>
> >> [Postbus 9517, 2300 RA Leiden]
> >>
> >> [schildpad]
> >>
> >> --
> >>
> >> Met vriendelijke groet,
> >>
> >> Dr. Rutger A. Vos
> >>
> >> Researcher / Bioinformatician
> >>
> >> [logo-new]
> >>
> >> [+31717519600 - +31627085806 ]
> >>
> >> [rutger.vos at naturalis.nl - www.naturalis.nl]
> >>
> >> [Darwinweg 2, 2333 CR Leiden]
> >>
> >> [Postbus 9517, 2300 RA Leiden]
> >>
> >> [schildpad]
> >>
> >> --
> >>
> >> Met vriendelijke groet,
> >>
> >> Dr. Rutger A. Vos
> >> Researcher / Bioinformatician
> >> [logo-new]
> >>
> >> [+31717519600 - +31627085806 ]
> >> [rutger.vos at naturalis.nl - www.naturalis.nl]
> >> [Darwinweg 2, 2333 CR Leiden]
> >> [Postbus 9517, 2300 RA Leiden]
> >>
> >> [schildpad]
> >>
> >> --
> >>
> >> Met vriendelijke groet,
> >>
> >> Dr. Rutger A. Vos
> >> Researcher / Bioinformatician
> >> [logo-new]
> >>
> >> [+31717519600 - +31627085806 ]
> >> [rutger.vos at naturalis.nl - www.naturalis.nl]
> >> [Darwinweg 2, 2333 CR Leiden]
> >> [Postbus 9517, 2300 RA Leiden]
> >>
> >> [schildpad]
> >>
>
>
--
--
Carles Fenoy
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20200502/cb14ef91/attachment.htm>
More information about the slurm-users
mailing list