<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="color: rgb(0, 0, 0); font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt;">
<span>Have you defined the TmpDisk value for each node?</span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt;">
<span><br>
</span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt;">
<span>As far as I know, local disk space is not a valid type for GRES.</span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt;">
<span><br>
</span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt;">
<span><a href="https://slurm.schedmd.com/gres.html">https://slurm.schedmd.com/gres.html</a><br>
</span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt;">
<span><br>
</span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt;">
<span><span style="color: rgb(70, 84, 92); font-family: "Source Sans Pro", Helvetica, Arial, sans-serif; font-size: 20px; text-align: start; background-color: rgb(255, 255, 255); display: inline !important">"Generic resource (GRES) scheduling is supported through
a flexible plugin mechanism. Support is currently provided for Graphics Processing Units (GPUs), CUDA Multi-Process Service (MPS), and Intel® Many Integrated Core (MIC) processors.</span>"</span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt;">
<span><br>
</span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt;">
<span>The only valid solution I've found for scratch is to:</span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt;">
<span><br>
</span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt;">
In slurm.conf, d<span>efine the location of local scratch globally via TmpFS.</span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt;">
<span><br>
</span></div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt;">
<span>
<div style="border-bottom-color: currentColor; border-left-color: currentColor; border-right-color: currentColor; border-top-color: currentColor; color: black; font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt; font-size-adjust: none; margin-bottom: 0px; margin-top: 0px">
And then the amount per host is defined via TmpDisk=xxx.</div>
<span style="display: inline !important; background-color: rgb(255, 255, 255); font-size-adjust: none"></span>
<div style="border-bottom-color: currentColor; border-left-color: currentColor; border-right-color: currentColor; border-top-color: currentColor; color: black; font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt; font-size-adjust: none; margin-bottom: 0px; margin-top: 0px">
<br>
</div>
<span style="display: inline !important; background-color: rgb(255, 255, 255); font-size-adjust: none"></span>
<div style="border-bottom-color: currentColor; border-left-color: currentColor; border-right-color: currentColor; border-top-color: currentColor; color: black; font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt; font-size-adjust: none; margin-bottom: 0px; margin-top: 0px">
Then the request for srun/sbatch via --tmp=X</div>
<span style="display: inline !important; background-color: rgb(255, 255, 255); font-size-adjust: none"></span>
<div style="border-bottom-color: currentColor; border-left-color: currentColor; border-right-color: currentColor; border-top-color: currentColor; color: black; font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt; font-size-adjust: none; margin-bottom: 0px; margin-top: 0px">
<br>
</div>
<span style="display: inline !important; background-color: rgb(255, 255, 255); font-size-adjust: none"></span>
<div style="border-bottom-color: currentColor; border-left-color: currentColor; border-right-color: currentColor; border-top-color: currentColor; color: black; font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt; font-size-adjust: none; margin-bottom: 0px; margin-top: 0px">
<br>
</div>
</span></div>
<div>
<div style="color: rgb(0, 0, 0); font-family: Calibri,Arial,Helvetica,sans-serif; font-size: 12pt;">
<br>
</div>
<div id="Signature">
<div></div>
<div></div>
<div id="divtagdefaultwrapper" style="font-size:12pt; color:#000000; font-family:Calibri,Helvetica,sans-serif" dir="ltr">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-size:13px">
<div class="BodyFragment"><font size="2"><span style="font-size:10pt"><font face="Courier New"></font>
<div class="PlainText"><font face="Courier New">---</font></div>
<div class="PlainText"><font face="Courier New"><span style="font-family:"Courier New",monospace">Erik Ellestad</span></font><br style="">
</div>
<span style="font-family:"Courier New",monospace; display:inline!important; font-size-adjust:none; background-color:rgb(255,255,255)">Wynton Cluster SysAdmin</span><br style="">
<span style="font-family:"Courier New",monospace; display:inline!important; font-size-adjust:none; background-color:rgb(255,255,255)">UCSF</span>
<div class="PlainText"><font face="Courier New"></font></div>
</span></font></div>
</div>
</div>
</div>
</div>
</div>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> slurm-users <slurm-users-bounces@lists.schedmd.com> on behalf of navin srivastava <navin.altair@gmail.com><br>
<b>Sent:</b> Tuesday, April 14, 2020 7:32 AM<br>
<b>To:</b> Slurm User Community List <slurm-users@lists.schedmd.com><br>
<b>Subject:</b> Re: [slurm-users] How to request for the allocation of scratch .</font>
<div> </div>
</div>
<div>
<div dir="ltr"><br>
<div>Any suggestion on the above query.need help to understand it. </div>
<div>Does TmpFS=/scratch and the request is #SBATCH --tmp=500GB then it will reserve the 500GB from scratch.</div>
<div>let me know if my assumption is correct?</div>
<div><br>
</div>
<div>Regards<br>
</div>
<div>Navin.</div>
<div><br>
</div>
</div>
<br>
<div class="x_gmail_quote">
<div dir="ltr" class="x_gmail_attr">On Mon, Apr 13, 2020 at 11:10 AM navin srivastava <<a href="mailto:navin.altair@gmail.com">navin.altair@gmail.com</a>> wrote:<br>
</div>
<blockquote class="x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<div dir="ltr">Hi Team,
<div><br>
</div>
<div>i wanted to define a mechanism to request the local disk space while submitting the job.<br>
</div>
<div><br>
</div>
<div>we have dedicated /scratch of 1.2 TB file system for the execution of the job on each of the compute nodes other than / and other file system.</div>
<div>i have defined in slurm.conf as TmpFS=/scratch and then wanted to use #SBATCH --scratch =10GB in the request.<br>
</div>
<div>but it seems it is not accepting this variable except /tmp. </div>
<div><br>
</div>
<div>Then i have opted the mechanism of gres.conf </div>
<div><br>
</div>
<div>GresTypes=gpu,scratch<br>
</div>
<div><br>
</div>
<div>and defined each node the scratch value and then requested using --gres=lscratch:10GB</div>
<div>but in this scenario if requesting both gres resources gpu as well as scratch it show me only scratch in my Gres resource not gpu.</div>
<div>does it using the gpu also as a gres resource?</div>
<div><br>
</div>
<div>could anybody please advice which is the correct method to achieve the same?<br>
</div>
<div>Also, is scratch will be able to calculate the actual usage value on the node. </div>
<div><br>
</div>
<div>REgards</div>
<div>Navin.</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div></div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
</blockquote>
</div>
</div>
</body>
</html>