<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<div style="margin:0px;font-size:12pt;color:black;background-color:rgb(255, 255, 255)">
Hello,</div>
<div style="margin:0px;font-size:12pt;color:black;background-color:rgb(255, 255, 255)">
<br>
</div>
<div style="margin:0px;font-size:12pt;color:black;background-color:rgb(255, 255, 255)">
Last year I posted on this forum looking for some help on backfill in Slurm. We are currently using Slurm 19.05.8 and we find that backfilled (smaller) jobs tend to push back large jobs in our cluster. Chris Samuel replied to our post with the following response...</div>
<div style="margin:0px;font-size:12pt;color:black;background-color:rgb(255, 255, 255)">
<br>
</div>
<div style="margin:0px;font-size:12pt;color:black;background-color:rgb(255, 255, 255)">
<i><span style="font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;font-size:14.6667px;background-color:rgb(255, 255, 255);display:inline !important">This sounds like a
problem that we had at NERSC (small jobs pushing back </span><span style="font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;font-size:14.6667px;background-color:rgb(255, 255, 255);display:inline !important">multi-thousand
node jobs), and we carried a local patch for which Doug</span><br style="font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;font-size:14.6667px;background-color:rgb(255, 255, 255)">
<span style="font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;font-size:14.6667px;background-color:rgb(255, 255, 255);display:inline !important">managed to get upstreamed
in 20.02.x (I think it landed in 20.02.3, but </span><span style="font-family:"Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif;font-size:14.6667px;background-color:rgb(255, 255, 255);display:inline !important">20.02.6
is the current version).</span></i><br>
</div>
<div style="margin:0px;font-size:12pt;color:black;background-color:rgb(255, 255, 255)">
<br>
</div>
<div style="margin:0px;font-size:12pt;color:black;background-color:rgb(255, 255, 255)">
We looked through the release notes and sure enough there is a reference to a job starvation patch, however I'm not sure that it is the relevant patch... (in 20.02.2)</div>
<div style="margin:0px;font-size:12pt;color:black;background-color:rgb(255, 255, 255)">
<span style="color: black; font-size: 12pt;">></span><i style="font-family: inherit; font-size: 12pt; font-variant-ligatures: inherit; font-variant-caps: inherit; font-weight: inherit;"> -- Fix scheduling issue when there are not enough nodes available to
run a job</i></div>
<div style="margin:0px;font-size:12pt;color:black;background-color:rgb(255, 255, 255)">
<span style="color: black; font-size: 12pt;">></span><i style="font-family: inherit; font-size: 12pt; font-variant-ligatures: inherit; font-variant-caps: inherit; font-weight: inherit;"> resulting in possible job starvation.</i></div>
<div style="margin:0px;font-size:12pt;color:black;background-color:rgb(255, 255, 255)">
<br>
</div>
<div style="margin:0px;font-size:12pt;color:black;background-color:rgb(255, 255, 255)">
We decided to download and install the latest production version, 20.11.2, of Slurm. One of my team members managed the installation and ran his backfill tests only to find that the above backfill issue was still present. Should we wind back to version 20.02.6
and insall/test that instead? Could someone please advise use? It would seem odd that a recent version of slurm would still have a backfill issue that starves larger job out. We're wondering if you have forgotten to configure something very fundamental, for
example.</div>
<div style="margin:0px;font-size:12pt;color:black;background-color:rgb(255, 255, 255)">
<br>
</div>
<div style="margin:0px;font-size:12pt;color:black;background-color:rgb(255, 255, 255)">
Best regards,</div>
<div style="margin:0px;font-size:12pt;color:black;background-color:rgb(255, 255, 255)">
David</div>
</div>
</body>
</html>