[slurm-users] Servers in pending state

Zohar Roe MLM RZohar8 at iai.co.il
Wed Mar 11 12:03:35 UTC 2020


I have a queue with 6 servers.
When 4 of the servers are with heavy load, If I send new jobs to the other 2 servers which are free and under different partition and features, The jobs are still in pending mode (can take them 20 minutes to start running)

If I change their priority with "scontrol update" they start to run immediately.

I am guessing it take Slurm a lot of time to reschedule all jobs when there is a heavy load so the new jobs are not check until I change their priority.

Is there a way to tell  Slurm to check all pending jobs every 2 minutes so if there are pending jobs on a free servers they will start running?

More info:
SchedulerType = sched/backfill
SchedulerParameters = bf_continue,bf_max_job_test=300


*********************************************************************************************** Please consider the environment before printing this email ! The information contained in this communication is proprietary to Israel Aerospace Industries Ltd. and/or third parties, may contain confidential or privileged information, and is intended only for the use of the intended addressee thereof. If you are not the intended addressee, please be aware that any use, disclosure, distribution and/or copying of this communication is strictly prohibited. If you receive this communication in error, please notify the sender immediately and delete it from your computer. Thank you. Visit us at: www.iai.co.il
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20200311/c7623040/attachment.htm>

More information about the slurm-users mailing list