<div dir="ltr">I attempted again and it gets succeed. <br><div>Thanks for your help.</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Apr 16, 2020 at 9:45 PM Ellestad, Erik <<a href="mailto:Erik.Ellestad@ucsf.edu" target="_blank">Erik.Ellestad@ucsf.edu</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
That all seems fine to me.</div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
I would check into your slurm logs to try and determine why slurm put your nodes into drain state.</div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
Erik</div>
<div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div id="gmail-m_-7448771380267513126gmail-m_-1620983108968355891Signature">
<div></div>
<div></div>
<div id="gmail-m_-7448771380267513126gmail-m_-1620983108968355891divtagdefaultwrapper" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif" dir="ltr">
<div style="font-family:Tahoma;font-size:13px">
<div style="font-family:Tahoma;font-size:13px">
<div style="font-size:13px">
<div><font size="2"><span style="font-size:10pt"><font face="Courier New"></font>
<div><font face="Courier New">---</font></div>
<div><font face="Courier New"><span style="font-family:"Courier New",monospace">Erik Ellestad</span></font><br>
</div>
<span style="font-family:"Courier New",monospace;background-color:rgb(255,255,255);display:inline">Wynton Cluster SysAdmin</span><br>
<span style="font-family:"Courier New",monospace;background-color:rgb(255,255,255);display:inline">UCSF</span>
<div><font face="Courier New"></font></div>
</span></font></div>
</div>
</div>
</div>
</div>
</div>
</div>
<div id="gmail-m_-7448771380267513126gmail-m_-1620983108968355891appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-7448771380267513126gmail-m_-1620983108968355891divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> slurm-users <<a href="mailto:slurm-users-bounces@lists.schedmd.com" target="_blank">slurm-users-bounces@lists.schedmd.com</a>> on behalf of navin srivastava <<a href="mailto:navin.altair@gmail.com" target="_blank">navin.altair@gmail.com</a>><br>
<b>Sent:</b> Wednesday, April 15, 2020 10:37 PM<br>
<b>To:</b> Slurm User Community List <<a href="mailto:slurm-users@lists.schedmd.com" target="_blank">slurm-users@lists.schedmd.com</a>><br>
<b>Subject:</b> Re: [slurm-users] How to request for the allocation of scratch .</font>
<div> </div>
</div>
<div>
<div dir="ltr">Thanks Erik.<br>
<div><br>
</div>
<div>Last night i made the changes.</div>
<div><br>
</div>
<div>i defined in slurm.conf on all the nodes as well as on the slurm server.</div>
<div><br>
</div>
<div>TmpFS=/lscratch</div>
<div><br>
</div>
<div> NodeName=node[01-10] CPUs=44 RealMemory=257380 Sockets=2 CoresPerSocket=22 ThreadsPerCore=1 TmpDisk=1600000 State=UNKNOWN Feature=P4000 Gres=gpu:2</div>
<div><br>
</div>
<div>These nodes having 1.6TB local scratch. i did a scontrol reconfig on all the nodes but after sometime we saw all nodes went into drain state.then i revert back the changes with old one.</div>
<div><br>
</div>
<div>on all nodes jobs were running and the localsctratch is 20-25% in use.<br>
</div>
<div>we have already cleanup script in crontab which used to clean the scratch space regularly. <br>
</div>
<div><br>
</div>
<div>is anything wrong here? <br>
</div>
<div><br>
</div>
<div></div>
<div><br>
</div>
<div>Regards</div>
<div>Navin.</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div>
<div dir="ltr">On Thu, Apr 16, 2020 at 12:26 AM Ellestad, Erik <<a href="mailto:Erik.Ellestad@ucsf.edu" target="_blank">Erik.Ellestad@ucsf.edu</a>> wrote:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
The default value for TmpDisk is 0, so if <span>you want local scratch available on a node, the amount of TmpDisk space must be defined in the node configuration in slurm.conf.</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span><br>
</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span>
<div style="border-color:currentcolor;font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;margin-bottom:0px;margin-top:0px">
<span style="border-color:currentcolor;margin-bottom:0px;margin-top:0px">example:</span></div>
<div style="border-color:currentcolor;font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;margin-bottom:0px;margin-top:0px">
<span style="border-color:currentcolor;margin-bottom:0px;margin-top:0px"><br>
</span></div>
<div style="border-color:currentcolor;font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;margin-bottom:0px;margin-top:0px">
<span style="border-color:currentcolor;margin-bottom:0px;margin-top:0px">NodeName=TestNode01 CPUs=8 Boards=1 SocketsPerBoard=2 CoresPerSocket=4 ThreadsPerCore=1 RealMemory=24099 TmpDisk=150000</span></div>
<br>
</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span>T</span><font style="background-color:rgb(255,255,255)"><span style="display:inline">he configuration value for the node definition is in MB.</span></font></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span><font style="background-color:rgb(255,255,255)"></font><br>
</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span>
<div style="border-color:currentcolor;font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;margin-bottom:0px;margin-top:0px">
<span style="border-color:currentcolor;margin-bottom:0px;margin-top:0px"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__slurm.schedmd.com_slurm.conf.html&d=DwMFaQ&c=iORugZls2LlYyCAZRB3XLg&r=Ct3zEADMmPgyUYfpHDJQWaWsE9mNbEHEhGxpsYoThbE&m=S4Dlg4-bVtpN7A5uGS3U053sTbGLQhWWqF7czJWRhO8&s=-q_lB4fzIj8i2PkR5NfWJomcaoDnRlBuyvyxkf4V0hQ&e=" style="border-color:currentcolor;margin-bottom:0px;margin-top:0px" target="_blank">https://slurm.schedmd.com/slurm.conf.html</a><br>
</span></div>
<div style="border-color:currentcolor;margin-bottom:0px;margin-top:0px">
<div style="border-color:currentcolor;font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;margin-bottom:0px;margin-top:0px">
<br>
</div>
<div style="border-color:currentcolor;font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;margin-bottom:0px;margin-top:0px">
<dl><dt><b style="box-sizing:border-box;font-weight:bold;margin-bottom:0px;margin-top:0px">TmpDisk</b></dt><dd>Total size of temporary disk storage in<span style="border-color:currentcolor;margin-bottom:0px;margin-top:0px"> </span><b style="box-sizing:border-box;font-weight:bold;margin-bottom:0px;margin-top:0px">TmpFS</b><span style="border-color:currentcolor;margin-bottom:0px;margin-top:0px"> </span>in
megabytes (e.g. "16384").<span style="border-color:currentcolor;margin-bottom:0px;margin-top:0px"> </span><b style="box-sizing:border-box;font-weight:bold;margin-bottom:0px;margin-top:0px">TmpFS</b><span style="border-color:currentcolor;margin-bottom:0px;margin-top:0px"> </span>(for
"Temporary File System") identifies the location which jobs should use for temporary storage. Note this does not indicate the amount of free space available to the user on the node, only the total file system size. The system administration should ensure this
file system is purged as needed so that user jobs have access to most of this space. The Prolog and/or Epilog programs (specified in the configuration file) might be used to ensure the file system is kept clean. The default value is 0.<br>
</dd></dl>
</div>
</div>
</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span><br>
</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span>When requesting --tmp with srun or sbatch, it can be done in various size formats:</span><span><br>
</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span><br>
</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span><span style="color:rgb(70,84,92);font-family:"Source Sans Pro",Helvetica,Arial,sans-serif;font-size:20px;text-align:start;background-color:rgb(255,255,255);display:inline">
<dl><dt style="box-sizing:border-box;font-family:"Source Sans Pro",Helvetica,Arial,sans-serif;text-align:start;background-color:rgb(255,255,255)">
<b style="box-sizing:border-box;margin:0px;font-weight:bold">--tmp</b>=<<i style="box-sizing:border-box;margin:0px;font-style:italic">size[units]</i>></dt><dd style="box-sizing:border-box;font-family:"Source Sans Pro",Helvetica,Arial,sans-serif;text-align:start;background-color:rgb(255,255,255)">
Specify a minimum amount of temporary disk space per node. Default units are megabytes unless the SchedulerParameters configuration parameter includes the "default_gbytes" option for gigabytes. Different units can be specified using the suffix [K|M|G|T].</dd></dl>
</span><br>
</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span><a id="gmail-m_-7448771380267513126gmail-m_-1620983108968355891x_gmail-m_4734527141577504288LPNoLP229416" href="https://urldefense.proofpoint.com/v2/url?u=https-3A__slurm.schedmd.com_sbatch.html&d=DwMFaQ&c=iORugZls2LlYyCAZRB3XLg&r=Ct3zEADMmPgyUYfpHDJQWaWsE9mNbEHEhGxpsYoThbE&m=S4Dlg4-bVtpN7A5uGS3U053sTbGLQhWWqF7czJWRhO8&s=yF9k9ysKRaVnu_ZnNdHsxj8Yc6X7PXTlId7i3XgZ5V4&e=" target="_blank">https://slurm.schedmd.com/sbatch.html</a><br>
</span></div>
<br>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<u><font color="#000120"></font></u><br>
</div>
<div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div id="gmail-m_-7448771380267513126gmail-m_-1620983108968355891x_gmail-m_4734527141577504288Signature">
<div></div>
<div></div>
<div id="gmail-m_-7448771380267513126gmail-m_-1620983108968355891x_gmail-m_4734527141577504288divtagdefaultwrapper" dir="ltr" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif">
<div style="font-family:Tahoma;font-size:13px">
<div style="font-family:Tahoma;font-size:13px">
<div style="font-size:13px">
<div><font size="2"><span style="font-size:10pt"><font face="Courier New"></font>
<div><font face="Courier New">---</font></div>
<div><font face="Courier New"><span style="font-family:"Courier New",monospace">Erik Ellestad</span></font><br>
</div>
<span style="font-family:"Courier New",monospace;background-color:rgb(255,255,255);display:inline">Wynton Cluster SysAdmin</span><br>
<span style="font-family:"Courier New",monospace;background-color:rgb(255,255,255);display:inline">UCSF</span>
<div><font face="Courier New"></font></div>
</span></font></div>
</div>
</div>
</div>
</div>
</div>
</div>
<div id="gmail-m_-7448771380267513126gmail-m_-1620983108968355891x_gmail-m_4734527141577504288appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-7448771380267513126gmail-m_-1620983108968355891x_gmail-m_4734527141577504288divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> slurm-users <<a href="mailto:slurm-users-bounces@lists.schedmd.com" target="_blank">slurm-users-bounces@lists.schedmd.com</a>>
on behalf of navin srivastava <<a href="mailto:navin.altair@gmail.com" target="_blank">navin.altair@gmail.com</a>><br>
<b>Sent:</b> Tuesday, April 14, 2020 11:19 PM<br>
<b>To:</b> Slurm User Community List <<a href="mailto:slurm-users@lists.schedmd.com" target="_blank">slurm-users@lists.schedmd.com</a>><br>
<b>Subject:</b> Re: [slurm-users] How to request for the allocation of scratch .</font>
<div> </div>
</div>
<div>
<div dir="ltr">Thank you Erik.
<div><br>
</div>
<div>To define the local scratch on all the compute node is not mandatory? only on slurm server is enough right?</div>
<div>Also the TMPdisk should be defined in MB or can be defined in GB as well<br>
</div>
<div><br>
</div>
<div>while requesting --tmp , we can use the value in GB right?</div>
<div><br>
</div>
<div>Regards</div>
<div>Navin.</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div>
<div dir="ltr">On Tue, Apr 14, 2020 at 11:04 PM Ellestad, Erik <<a href="mailto:Erik.Ellestad@ucsf.edu" target="_blank">Erik.Ellestad@ucsf.edu</a>> wrote:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span>Have you defined the TmpDisk value for each node?</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span><br>
</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span>As far as I know, local disk space is not a valid type for GRES.</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span><br>
</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__slurm.schedmd.com_gres.html&d=DwMFaQ&c=iORugZls2LlYyCAZRB3XLg&r=Ct3zEADMmPgyUYfpHDJQWaWsE9mNbEHEhGxpsYoThbE&m=qpy9RbpYHEmd6jqfs9j8b4IiRJf3GkO-X_v05nL-8Bo&s=B5wXNgnl6EdXIaS0QQpITnBTSxcjnAHJENyJGLyWltI&e=" target="_blank">https://slurm.schedmd.com/gres.html</a><br>
</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span><br>
</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span><span style="color:rgb(70,84,92);font-family:"Source Sans Pro",Helvetica,Arial,sans-serif;font-size:20px;text-align:start;background-color:rgb(255,255,255);display:inline">"Generic resource (GRES) scheduling is supported through a flexible plugin
mechanism. Support is currently provided for Graphics Processing Units (GPUs), CUDA Multi-Process Service (MPS), and Intel® Many Integrated Core (MIC) processors.</span>"</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span><br>
</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span>The only valid solution I've found for scratch is to:</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span><br>
</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
In slurm.conf, d<span>efine the location of local scratch globally via TmpFS.</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span><br>
</span></div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<span>
<div style="border-color:currentcolor;color:black;font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;margin-bottom:0px;margin-top:0px">
And then the amount per host is defined via TmpDisk=xxx.</div>
<span style="background-color:rgb(255,255,255);display:inline"></span>
<div style="border-color:currentcolor;color:black;font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;margin-bottom:0px;margin-top:0px">
<br>
</div>
<span style="background-color:rgb(255,255,255);display:inline"></span>
<div style="border-color:currentcolor;color:black;font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;margin-bottom:0px;margin-top:0px">
Then the request for srun/sbatch via --tmp=X</div>
<span style="background-color:rgb(255,255,255);display:inline"></span>
<div style="border-color:currentcolor;color:black;font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;margin-bottom:0px;margin-top:0px">
<br>
</div>
<span style="background-color:rgb(255,255,255);display:inline"></span>
<div style="border-color:currentcolor;color:black;font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt;margin-bottom:0px;margin-top:0px">
<br>
</div>
</span></div>
<div>
<div style="color:rgb(0,0,0);font-family:Calibri,Arial,Helvetica,sans-serif;font-size:12pt">
<br>
</div>
<div id="gmail-m_-7448771380267513126gmail-m_-1620983108968355891x_gmail-m_4734527141577504288x_gmail-m_6961401004416808594Signature">
<div></div>
<div></div>
<div id="gmail-m_-7448771380267513126gmail-m_-1620983108968355891x_gmail-m_4734527141577504288x_gmail-m_6961401004416808594divtagdefaultwrapper" dir="ltr" style="font-size:12pt;color:rgb(0,0,0);font-family:Calibri,Helvetica,sans-serif">
<div style="font-family:Tahoma;font-size:13px">
<div style="font-family:Tahoma;font-size:13px">
<div style="font-size:13px">
<div><font size="2"><span style="font-size:10pt"><font face="Courier New"></font>
<div><font face="Courier New">---</font></div>
<div><font face="Courier New"><span style="font-family:"Courier New",monospace">Erik Ellestad</span></font><br>
</div>
<span style="font-family:"Courier New",monospace;background-color:rgb(255,255,255);display:inline">Wynton Cluster SysAdmin</span><br>
<span style="font-family:"Courier New",monospace;background-color:rgb(255,255,255);display:inline">UCSF</span>
<div><font face="Courier New"></font></div>
</span></font></div>
</div>
</div>
</div>
</div>
</div>
</div>
<div id="gmail-m_-7448771380267513126gmail-m_-1620983108968355891x_gmail-m_4734527141577504288x_gmail-m_6961401004416808594appendonsend">
</div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-7448771380267513126gmail-m_-1620983108968355891x_gmail-m_4734527141577504288x_gmail-m_6961401004416808594divRplyFwdMsg" dir="ltr">
<font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> slurm-users <<a href="mailto:slurm-users-bounces@lists.schedmd.com" target="_blank">slurm-users-bounces@lists.schedmd.com</a>> on behalf of navin srivastava <<a href="mailto:navin.altair@gmail.com" target="_blank">navin.altair@gmail.com</a>><br>
<b>Sent:</b> Tuesday, April 14, 2020 7:32 AM<br>
<b>To:</b> Slurm User Community List <<a href="mailto:slurm-users@lists.schedmd.com" target="_blank">slurm-users@lists.schedmd.com</a>><br>
<b>Subject:</b> Re: [slurm-users] How to request for the allocation of scratch .</font>
<div> </div>
</div>
<div>
<div dir="ltr"><br>
<div>Any suggestion on the above query.need help to understand it. </div>
<div>Does TmpFS=/scratch and the request is #SBATCH --tmp=500GB then it will reserve the 500GB from scratch.</div>
<div>let me know if my assumption is correct?</div>
<div><br>
</div>
<div>Regards<br>
</div>
<div>Navin.</div>
<div><br>
</div>
</div>
<br>
<div>
<div dir="ltr">On Mon, Apr 13, 2020 at 11:10 AM navin srivastava <<a href="mailto:navin.altair@gmail.com" target="_blank">navin.altair@gmail.com</a>> wrote:<br>
</div>
<blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">Hi Team,
<div><br>
</div>
<div>i wanted to define a mechanism to request the local disk space while submitting the job.<br>
</div>
<div><br>
</div>
<div>we have dedicated /scratch of 1.2 TB file system for the execution of the job on each of the compute nodes other than / and other file system.</div>
<div>i have defined in slurm.conf as TmpFS=/scratch and then wanted to use #SBATCH --scratch =10GB in the request.<br>
</div>
<div>but it seems it is not accepting this variable except /tmp. </div>
<div><br>
</div>
<div>Then i have opted the mechanism of gres.conf </div>
<div><br>
</div>
<div>GresTypes=gpu,scratch<br>
</div>
<div><br>
</div>
<div>and defined each node the scratch value and then requested using --gres=lscratch:10GB</div>
<div>but in this scenario if requesting both gres resources gpu as well as scratch it show me only scratch in my Gres resource not gpu.</div>
<div>does it using the gpu also as a gres resource?</div>
<div><br>
</div>
<div>could anybody please advice which is the correct method to achieve the same?<br>
</div>
<div>Also, is scratch will be able to calculate the actual usage value on the node. </div>
<div><br>
</div>
<div>REgards</div>
<div>Navin.</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div></div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
<div><br>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote></div>