<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
</div>
<p style="text-decoration: none; border: 0px solid rgb(217, 217, 227); box-sizing: border-box; margin: 0px 0px 1.25em; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; color: rgb(55, 65, 81);" class="ContentPasted3">
Hello,</p>
<p style="text-decoration: none; border: 0px solid rgb(217, 217, 227); box-sizing: border-box; margin: 1.25em 0px; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; color: rgb(55, 65, 81);" class="ContentPasted3 elementToProof">
I'm quite a newbie regarding Slurm. I recently created a small Slurm instance to manage our GPU resources. I have this situation:</p>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof ContentPasted0">
<span style="font-family: Consolas, Courier, monospace;"> JOBID STATE TIME ACCOUNT PARTITION PRIORITY REASON CPU MIN_MEM TRES_PER_NODE</span>
<div class="ContentPasted0"><span style="font-family: Consolas, Courier, monospace;"> 1739 PENDING 0:00 standard gpu-low 5 Priority 1 80G gres:gpu:a100_1g.10gb:1</span></div>
<div class="ContentPasted0"><span style="font-family: Consolas, Courier, monospace;"> 1738 PENDING 0:00 standard gpu-low 5 Priority 1 80G gres:gpu:a100-sxm4-80gb:1</span></div>
<div class="ContentPasted0"><span style="font-family: Consolas, Courier, monospace;"> 1737 PENDING 0:00 standard gpu-low 5 Priority 1 80G gres:gpu:a100-sxm4-80gb:1</span></div>
<div class="ContentPasted0"><span style="font-family: Consolas, Courier, monospace;"> 1736 PENDING 0:00 standard gpu-low 5 Resources 1 80G gres:gpu:a100-sxm4-80gb:1</span></div>
<div class="ContentPasted0"><span style="font-family: Consolas, Courier, monospace;"> 1740 PENDING 0:00 standard gpu-low 1 Priority 1 8G gres:gpu:a100_3g.39gb</span></div>
<div class="ContentPasted0"><span style="font-family: Consolas, Courier, monospace;"> 1735 PENDING 0:00 standard gpu-low 1 Priority 8 64G gres:gpu:a100-sxm4-80gb:1</span></div>
<div class="ContentPasted0"><span style="font-family: Consolas, Courier, monospace;"> 1596 RUNNING 1-13:26:45 standard gpu-low 3 None 2 64G gres:gpu:a100_1g.10gb:1</span></div>
<div class="ContentPasted0"><span style="font-family: Consolas, Courier, monospace;"> 1653 RUNNING 21:09:52 standard gpu-low 2 None 1 16G gres:gpu:1</span></div>
<div class="ContentPasted0"><span style="font-family: Consolas, Courier, monospace;"> 1734 RUNNING 59:52 standard gpu-low 1 None 8 64G gres:gpu:a100-sxm4-80gb:1</span></div>
<div class="ContentPasted0"><span style="font-family: Consolas, Courier, monospace;"> 1733 RUNNING 1:01:54 standard gpu-low 1 None 8 64G gres:gpu:a100-sxm4-80gb:1</span></div>
<div class="ContentPasted0"><span style="font-family: Consolas, Courier, monospace;"> 1732 RUNNING 1:02:39 standard gpu-low 1 None 8 40G gres:gpu:a100-sxm4-80gb:1</span></div>
<div class="ContentPasted0"><span style="font-family: Consolas, Courier, monospace;"> 1731 RUNNING 1:08:28 standard gpu-low 1 None 8 40G gres:gpu:a100-sxm4-80gb:1</span></div>
<div class="ContentPasted0"><span style="font-family: Consolas, Courier, monospace;"> 1718 RUNNING 10:16:40 standard gpu-low 1 None 2 8G gres:gpu:v100</span></div>
<div class="ContentPasted0"><span style="font-family: Consolas, Courier, monospace;"> 1630 RUNNING 1-00:21:21 standard gpu-low 1 None 1 30G gres:gpu:a100_3g.39gb</span></div>
<span style="font-family: Consolas, Courier, monospace;"> 1610 RUNNING 1-09:53:23 standard gpu-low 1 None 2 8G gres:gpu:v100</span></div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
<br>
</div>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
</div>
<p style="text-decoration: none; border: 0px solid rgb(217, 217, 227); box-sizing: border-box; margin: 1.25em 0px; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; color: rgb(55, 65, 81);" class="ContentPasted4">
Job 1736 is in the PENDING state since there are no more available a100-sxm4-80gb GPUs. The job priority starts to rise with time (priority 5) as expected. Now another user submits job 1739 on a gres:gpu:a100_1g.10gb:1 that is available, but the job is not
starting since its priority is 1. This is obviously not the desired outcome, and I believe I must change the scheduling strategy. Could someone with more experience than me give me some hints?</p>
<p style="text-decoration: none; border: 0px solid rgb(217, 217, 227); box-sizing: border-box; margin: 1.25em 0px 0px; font-family: Söhne, ui-sans-serif, system-ui, -apple-system, "Segoe UI", Roboto, Ubuntu, Cantarell, "Noto Sans", sans-serif, "Helvetica Neue", Arial, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol", "Noto Color Emoji"; color: rgb(55, 65, 81);" class="ContentPasted4">
Thanks, Cristiano</p>
<div style="font-family: Aptos, Aptos_EmbeddedFont, Aptos_MSFontService, Calibri, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);" class="elementToProof">
</div>
</body>
</html>