<div dir="ltr">Hello everyone,<div><br></div><div>We wish to deploy "fair share" scheduling configuration and would like to inquire if we should be aware of effects this might have on jobs already running or already queued when the config is changed. </div><div><br></div><div>The proposed changes are from the example at  <a href="https://slurm.schedmd.com/archive/slurm-18.08.9/priority_multifactor.html#config">https://slurm.schedmd.com/archive/slurm-18.08.9/priority_multifactor.html#config</a> :</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"># Activate the Multi-factor Job Priority Plugin with decay<br>PriorityType=priority/multifactor<br># 2 week half-life<br>PriorityDecayHalfLife=14-0<br># The larger the job, the greater its job size priority.<br>PriorityFavorSmall=NO<br># The job's age factor reaches 1.0 after waiting in the<br># queue for 2 weeks.<br>PriorityMaxAge=14-0<br># This next group determines the weighting of each of the<br># components of the Multi-factor Job Priority Plugin.<br># The default value for each of the following is 1.<br>PriorityWeightAge=1000<br>PriorityWeightFairshare=10000<br>PriorityWeightJobSize=1000<br>PriorityWeightPartition=1000<br>PriorityWeightQOS=0 # don't use the qos factor</blockquote><div>We're running SLURM 18.08.8 on CentOS Linux 7.8.2003. The current slurm.conf is defaults as far as fair share is concerned:</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">EnforcePartLimits=ALL<br>GresTypes=gpu<br>MpiDefault=pmix<br>ProctrackType=proctrack/cgroup<br>PrologFlags=x11,contain<br>PropagateResourceLimitsExcept=MEMLOCK,STACK<br>RebootProgram=/sbin/reboot<br>ReturnToService=1<br>SlurmctldPidFile=/var/run/slurmctld.pid<br>SlurmctldPort=6817<br>SlurmdPidFile=/var/run/slurmd.pid<br>SlurmdPort=6818<br>SlurmdSpoolDir=/var/spool/slurmd<br>SlurmUser=slurm<br>SlurmdSyslogDebug=verbose<br>StateSaveLocation=/var/spool/slurm/ctld<br>SwitchType=switch/none<br>TaskPlugin=task/cgroup,task/affinity<br>TaskPluginParam=Sched<br>HealthCheckInterval=300<br>HealthCheckProgram=/usr/sbin/nhc<br>InactiveLimit=0<br>KillWait=30<br>MinJobAge=300<br>SlurmctldTimeout=120<br>SlurmdTimeout=300<br>Waittime=0<br>DefMemPerCPU=1024<br>FastSchedule=1<br>SchedulerType=sched/backfill<br>SelectType=select/cons_res<br>SelectTypeParameters=CR_Core_Memory<br>AccountingStorageHost=sched-db.lan<br>AccountingStorageLoc=slurm_acct_db<br>AccountingStoragePass=/var/run/munge/munge.socket.2<br>AccountingStoragePort=6819<br>AccountingStorageType=accounting_storage/slurmdbd<br>AccountingStorageUser=slurm<br>AccountingStoreJobComment=YES<br>AccountingStorageTRES=gres/gpu<br>JobAcctGatherFrequency=30<br>JobAcctGatherType=jobacct_gather/linux<br>SlurmctldDebug=info<br>SlurmdDebug=info<br>SlurmSchedLogFile=/var/log/slurm/slurmsched.log<br>SlurmSchedLogLevel=1</blockquote><div>Node and partition configs are omitted above.</div><div><br></div><div>Any and all advice will be greatly appreciated.</div><div><br></div><div>Best wishes,<br><br>~Kevin<br><br>Kevin Walsh</div><div>Senior Systems Administration Specialist<br>New Jersey Institute of Technology</div><div>Academic & Research Computing Systems<br></div><div><br></div><div><br></div></div>