[slurm-users] Proposal for new TRES - "Processor Performance Units"....
hearnsj at googlemail.com
Thu Jun 20 06:58:50 UTC 2019
Janne, thankyou. That FGCI benchmark in a container is pretty smart.
I always say that real application benchmarks beat synthetic benchmarks.
Taking a small mix of applications like that and taking a geometric mean is
Note: *"a reference result run on a Dell PowerEdge C4130"*
In the old days CERN had a standard unit of compute, which was equivalent
to a VAX.
I am sure that unit has long been retired.
Though I must say that having participated in CERN tenders a few years ago
SpecFP measurements to compare systems.
On Thu, 20 Jun 2019 at 07:41, Janne Blomqvist <janne.blomqvist at aalto.fi>
> On 19/06/2019 22.30, Fulcomer, Samuel wrote:
> > (...and yes, the name is inspired by a certain OEM's software licensing
> > schemes...)
> > At Brown we run a ~400 node cluster containing nodes of multiple
> > architectures (Sandy/Ivy, Haswell/Broadwell, and Sky/Cascade) purchased
> > in some cases by University funds and in others by investigator funding
> > (~50:50). They all appear in the default SLURM partition. We have 3
> > classes of SLURM users:
> > 1. Exploratory - no-charge access to up to 16 cores
> > 2. Priority - $750/quarter for access to up to 192 cores (and with a
> > GrpTRESRunMins=cpu limit). Each user has their own QoS
> > 3. Condo - an investigator group who paid for nodes added to the
> > cluster. The group has its own QoS and SLURM Account. The QoS allows
> > use of the number of cores purchased and has a much higher priority
> > than the QoS' of the "priority" users.
> > The first problem with this scheme is that condo users who have
> > purchased the older hardware now have access to the newest without
> > penalty. In addition, we're encountering resistance to the idea of
> > turning off their hardware and terminating their condos (despite MOUs
> > stating a 5yr life). The pushback is the stated belief that the hardware
> > should run until it dies.
> > What I propose is a new TRES called a Processor Performance Unit (PPU)
> > that would be specified on the Node line in slurm.conf, and used such
> > that GrpTRES=ppu=N was calculated as the number of allocated cores
> > multiplied by their associated PPU numbers.
> > We could then assign a base PPU to the oldest hardware, say, "1" for
> > Sandy/Ivy and increase for later architectures based on performance
> > improvement. We'd set the condo QoS to GrpTRES=ppu=N*X+M*Y,..., where N
> > is the number of cores of the oldest architecture multiplied by the
> > configured PPU/core, X, and repeat for any newer nodes/cores the
> > investigator has purchased since.
> > The result is that the investigator group gets to run on an
> > approximation of the performance that they've purchased, rather on the
> > raw purchased core count.
> > Thoughts?
> What we do is that our nodes are grouped into separate partitions based
> on the CPU model. E.g. the partition "batch-skl" is where our Skylake
> (6148) nodes are. The we have a job_submit.lua script which sends jobs
> without an explicit partition spec to all batch-xxx partitions (checking
> constraints etc. along the way). Then for each partition we set
> TRESBillingWeights= to "normalize" the fairshare consumption based on
> the geometric mean of a set of hopefully not too unrepresentative
> single-node benchmarks .
> We also set a memory billing weight, and have MAX_TRES among our
> PriorityFlags, approximating dominant resource fairness (DRF) 
>  https://github.com/AaltoScienceIT/docker-fgci-benchmark
>  https://people.eecs.berkeley.edu/~alig/papers/drf.pdf
> Janne Blomqvist, D.Sc. (Tech.), Scientific Computing Specialist
> Aalto University School of Science, PHYS & NBE
> +358503841576 || janne.blomqvist at aalto.fi
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the slurm-users