[slurm-users] Preemption Priority

[Oytun Peksel]

Hi

Thanks for your reply. I enabled both of them but still the oldest job gets suspended. Here is my slurm.conf if you can see something I can't.

  0 MpiDefault=none
  1 ProctrackType=proctrack/linuxproc
  2 ReturnToService=1
  3 SlurmctldPidFile=/var/run/slurmctld.pid
  4 SlurmdPidFile=/var/run/slurmd.pid
  5 SlurmdSpoolDir=/var/spool/slurmd
  6 SlurmUser=slurm
  7 StateSaveLocation=/var/spool/slurmSave
  8 SwitchType=switch/none
  9 TaskPlugin=task/affinity
10 # # SCHEDULING
11 FastSchedule=1
12 SchedulerType=sched/builtin
13 SchedulerParameters=preempt_youngest_first,preempt_strict_order
14 PriorityType=priority/multifactor
15
16 SelectType=select/cons_res
17 SelectTypeParameters=CR_CPU_Memory
18 DefMemPerCPU=1000
19 PreemptType=preempt/partition_prio
20 PreemptMode=suspend,gang
21 SlurmctldParameters=preempt_send_user_signal
22
23 JobSubmitPlugins=lua
24 # # LOGGING AND ACCOUNTING
25 AccountingStorageHost=localhost
26 AccountingStorageType=accounting_storage/slurmdbd
27 ClusterName=simulation
28 # #JobAcctGatherFrequency=30
29 JobAcctGatherType=jobacct_gather/linux
30 SlurmctldLogFile=/var/log/slurmLog/slurmctld.log
31 SlurmdLogFile=/var/log/slurmLog/slurmd.log
32 JobCompType=jobcomp/filetxt
# # COMPUTE NODES
33 NodeName=localhost CPUs=2 CoresPerSocket=1 ThreadsPerCore=1 State=UNKNOWN RealMemory=7500
34  PartitionName=DEFAULT Nodes=localhost  PreemptMode=suspend OverSubscribe=FORCE:1 Shared=FORCE:1 MaxTime=INF    INITE State=UP
36 PartitionName=LowPrio Default=YES   PriorityTier=1  PreemptMode=requeue
37 PartitionName=HighPrio MaxTime=240  PriorityTier=2  PreemptMode=suspend Default=NO
38 PartitionName=Express  MaxTime=30 PriorityTier=3 PreemptMode=off Default=NO

Oytun Peksel

oytun.peksel at semcon.com <mailto:oytun.peksel at semcon.com>

+46739205917


From: slurm-users <slurm-users-bounces at lists.schedmd.com> On Behalf Of Paul Edmon
Sent: den 25 oktober 2019 16:16
To: slurm-users at lists.schedmd.com
Subject: Re: [slurm-users] Preemption Priority


You might want to look at these options for SchedulerParamtesr:
preempt_strict_order
If set, then execute extra logic in an attempt to preempt only the lowest priority jobs. It may be desirable to set this configuration parameter when there are multiple priorities of preemptable jobs. The logic to support this option is only available in the select/cons_res and select/cons_tres plugins.
preempt_youngest_first
If set, then the preemption sorting algorithm will be changed to sort by the job start times to favor preempting younger jobs over older. (Requires preempt/partition_prio or preempt/qos plugins.)

-Paul Edmon-
On 10/25/19 7:21 AM, Oytun Peksel wrote:
Hi,

Let's say I have two partitions assigned to the same single load in the cluster.
LowPrio with PreemptMode=suspend Priority=1
HighPrio with PreemtMode=off Priority=5

I have 4 identical jobs requiring 1/2 of resources in the cluster. So only 2 jobs can run at the same time.
I queue up two of these jobs to LowPrio partition one after the other. Let's say one have jobid=10 and the next one has jobd=11

Then I queue up a third job to HighPrio partition. One of the LowPrio jobs immediately get suspended.

Q1. How can I make sure the one submitted latest(jobid=11 in this case) gets suspended and not jobid=10?

Then before any job is finished I submit the final job to LowPrio partition.

So the queue looks like this
[cid:image001.png at 01D58E25.DD245E60]

Q2. When HighPrio job (12) finishes how can I make sure suspended job(10) would start instead of the pending job(13)?



Thanks


Oytun Peksel




When you communicate with us or otherwise interact with Semcon, we will process personal data that you provide to us or we collect about you, please read more in our Privacy Policy<https://semcon.com/data-privacy-policy/>.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20191030/beeca6bb/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 10973 bytes
Desc: image001.png
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20191030/beeca6bb/attachment-0001.png>