<div dir="auto">Thanks either I can use which slurmd -C gives because I see same set of node giving different value.or I can also choose the available memory I mean 251*1024 <div dir="auto"><br></div><div dir="auto">Regards </div><div dir="auto">Navin </div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Jul 10, 2020, 20:34 Stephan Roth <<a href="mailto:stephan.roth@ee.ethz.ch">stephan.roth@ee.ethz.ch</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">It's recommended to round RealMemory down to the next lower gigabyte <br>
value to prevent nodes from entering a drain state after rebooting with <br>
a bios- or kernel-update.<br>
<br>
Source: <a href="https://slurm.schedmd.com/SLUG17/FieldNotes.pdf" rel="noreferrer noreferrer" target="_blank">https://slurm.schedmd.com/SLUG17/FieldNotes.pdf</a>, "Node <br>
configuration"<br>
<br>
Stephan<br>
<br>
On 10.07.20 13:46, Sarlo, Jeffrey S wrote:<br>
> If you run slurmd -C on the compute node, it should tell you what <br>
> slurm thinks the RealMemory number is.<br>
> <br>
> Jeff<br>
> <br>
> ------------------------------------------------------------------------<br>
> *From:* slurm-users <<a href="mailto:slurm-users-bounces@lists.schedmd.com" target="_blank" rel="noreferrer">slurm-users-bounces@lists.schedmd.com</a>> on behalf of <br>
> navin srivastava <<a href="mailto:navin.altair@gmail.com" target="_blank" rel="noreferrer">navin.altair@gmail.com</a>><br>
> *Sent:* Friday, July 10, 2020 6:24 AM<br>
> *To:* Slurm User Community List <<a href="mailto:slurm-users@lists.schedmd.com" target="_blank" rel="noreferrer">slurm-users@lists.schedmd.com</a>><br>
> *Subject:* Re: [slurm-users] changes in slurm.<br>
> Thank you for the answers.<br>
> <br>
> is the RealMemory will be decided on the Total Memory value or total <br>
> usable memory value.<br>
> <br>
> i mean if a node having 256GB RAM but free -g will tell about only 251 GB.<br>
> deda1x1591:~ # free -g<br>
> total used free shared buffers cached<br>
> Mem: 251 67 184 6 0 47<br>
> <br>
> so we can add the value is 251*1024 MB or 256*1024MB. or is there any <br>
> slurm command which will provide me the value to add.<br>
> <br>
> Regards<br>
> Navin.<br>
> <br>
> <br>
> <br>
> On Thu, Jul 9, 2020 at 8:01 PM Brian Andrus <<a href="mailto:toomuchit@gmail.com" target="_blank" rel="noreferrer">toomuchit@gmail.com</a> <br>
> <mailto:<a href="mailto:toomuchit@gmail.com" target="_blank" rel="noreferrer">toomuchit@gmail.com</a>>> wrote:<br>
> <br>
> Navin,<br>
> <br>
> 1. you will need to restart slurmctld when you make changes to the<br>
> physical definition of a node. This can be done without affecting<br>
> running jobs.<br>
> <br>
> 2. You can have a node in more than one partition. That will not hurt<br>
> anything. Jobs are allocated to nodes, not partitions, the partition is<br>
> used to determine which node(s) and filter/order jobs. You should add<br>
> the node to the new partition, but also leave it in the 'test'<br>
> partition. If you are looking to remove the 'test' partition, set it to<br>
> down and once all the running jobs that are in it finish, then<br>
> remove it.<br>
> <br>
> Brian Andrus<br>
> <br>
> On 7/8/2020 10:57 PM, navin srivastava wrote:<br>
> > Hi Team,<br>
> ><br>
> > i have 2 small query.because of the lack of testing environment i am<br>
> > unable to test the scenario. working on to set up a test environment.<br>
> ><br>
> > 1. In my environment i am unable to pass #SBATCH --mem-2GB option.<br>
> > i found the reason is because there is no RealMemory entry in the<br>
> node<br>
> > definition of the slurm.<br>
> ><br>
> > NodeName=Node[1-12] NodeHostname=deda1x[1450-1461]<br>
> NodeAddr=Node[1-12]<br>
> > Sockets=2 CoresPerSocket=10 State=UNKNOWN<br>
> ><br>
> > if i add the RealMemory it should be able to pick. So my query here<br>
> > is, is it possible to add RealMemory in the definition anytime while<br>
> > the jobs are in progres and execute the scontrol reconfigure and<br>
> > reload the daemon on client node? or do we need to take a<br>
> > downtime?(which i don't think so)<br>
> ><br>
> > 2. Also I would like to know what will happen if some jobs are<br>
> running<br>
> > in a partition(say test) and I will move the associated node to some<br>
> > other partition(say normal) without draining the node.or if i<br>
> suspend<br>
> > the job and then change the node partition and will resume the<br>
> job. I<br>
> > am not deleting the partition here.<br>
> ><br>
> > Regards<br>
> > Navin.<br>
> ><br>
> ><br>
> ><br>
> ><br>
> ><br>
> ><br>
> ><br>
> <br>
<br>
<br>
-------------------------------------------------------------------<br>
Stephan Roth | <a href="http://ISG.EE" rel="noreferrer noreferrer" target="_blank">ISG.EE</a> D-ITET ETH Zurich | <a href="http://www.isg.ee.ethz.ch" rel="noreferrer noreferrer" target="_blank">http://www.isg.ee.ethz.ch</a><br>
+4144 632 30 59 | ETF D 104 | Sternwartstrasse 7 | 8092 Zurich<br>
-------------------------------------------------------------------<br>
<br>
</blockquote></div>