<div dir="ltr"><div dir="ltr"><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, 1 Mar 2023 at 07:51, Doug Meyer <<a href="mailto:dameyer99@gmail.com">dameyer99@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>I forgot one thing you didn't mention. When you change the node descriptors and partitions you have to also restart slurmctld. scontrol reconfigure works for the nodes but the main daemon has to be told to reread the config. Until you restart the daemon it will be referencing the config from the last time it started.</div><div><br></div></div></blockquote><div><br></div><div><br></div><div>Yeah, I restart all three daemons and run scontrol reconfigure after the changes are done.</div><div><br></div><div>I think I've solved the problem. SLURM was double-counting the CPUs, <a href="https://github.com/hariseldon99/buparamshavak/commit/59fadd33e485ea8ab3b8d9a6c57381bfa9b89d72">so I set FORCE:2 to FORCE:1 to oversubscribe</a> and now it's working. I launched 4 infinite loop MPI test jobs with 32 cores and it started running two of them and queued the other two, as required.</div><div><br></div><div><br></div><div>Thanks for all the help. It was awesome.</div><div><br></div><div>AR</div><div><br></div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div></div><div>Doug</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Feb 26, 2023 at 10:25 PM Analabha Roy <<a href="mailto:hariseldon99@gmail.com" target="_blank">hariseldon99@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">Hey,</div><div><br></div><div><br></div>Thanks for sticking with this.<div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, 26 Feb 2023 at 23:43, Doug Meyer <<a href="mailto:dameyer99@gmail.com" target="_blank">dameyer99@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr">Hi,<div><br></div><div>Suggest removing "boards=1", The docs say to include it but in previous discussions with schedmd we were advised to remove it.</div><div><br></div></div></div></blockquote><div><br></div><div>I just did. Then ran scontrol reconfigure.</div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div></div><div>When you are running execute "scontrol show node <nodename>" and look at the lines ConfigTres and AllocTres. The former is what the maitre d believes is available, the latter what has been allocated.</div><div><br></div><div>Then "scontrol show job <jobid>" looking down at the "NumNodes" like which will show you what the job requested.</div><div><br></div><div>I suspect there is a syntax error in the submit.</div><div><br></div></div></div></blockquote><div><br></div><div>Okay. Now this is strange.</div><div><br></div><div>First, I launched <a href="https://pastebin.com/s21yXFH2" target="_blank">this job twice</a></div><div>This should take up 20 + 20 = 40 cores, because of the</div><div> </div><div><ol style="box-sizing:border-box;padding:0px 10px;font-size:12px;vertical-align:baseline;color:rgb(51,51,51);background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial;font-family:Consolas,Menlo,Monaco,"Lucida Console","Liberation Mono","DejaVu Sans Mono","Bitstream Vera Sans Mono",monospace,serif;margin:0px"><li style="box-sizing:border-box;padding:0px;vertical-align:baseline;list-style-type:decimal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial;color:rgb(172,172,172)"><div style="box-sizing:border-box;padding:0px 8px;vertical-align:top;color:rgb(0,0,0);border-left:1px solid rgb(221,221,221);background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial;white-space:pre-wrap"><span style="box-sizing:border-box;margin:0px;padding:0px;vertical-align:baseline;color:rgb(102,102,102);font-style:italic">#SBATCH -n 20 # Number of tasks</span></div></li><li style="box-sizing:border-box;padding:0px;vertical-align:baseline;list-style-type:decimal;background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial;color:rgb(172,172,172)"><div style="box-sizing:border-box;padding:0px 8px;vertical-align:top;color:rgb(0,0,0);border-left:1px solid rgb(221,221,221);background-image:initial;background-position:initial;background-size:initial;background-repeat:initial;background-origin:initial;background-clip:initial;white-space:pre-wrap"><span style="box-sizing:border-box;margin:0px;padding:0px;vertical-align:baseline;color:rgb(102,102,102);font-style:italic">#SBATCH --cpus-per-task=1</span></div></li></ol><div><i style="font-size:12px;white-space:pre-wrap;color:rgb(102,102,102);font-family:Consolas,Menlo,Monaco,"Lucida Console","Liberation Mono","DejaVu Sans Mono","Bitstream Vera Sans Mono",monospace,serif"><br></i></div><div><br></div>running scontrol show job on both jobids yields</div><div><ul><li> NumNodes=1 NumCPUs=20 NumTasks=20 CPUs/Task=1 ReqB:S:C:T=0:0:*:*<br></li><li> NumNodes=1 NumCPUs=20 NumTasks=20 CPUs/Task=1 ReqB:S:C:T=0:0:*:*<br></li></ul></div><div><div>Then, running scontrol on the node yields:</div><div><br></div><div><ul><li>scontrol show node $HOSTNAME</li><li>CfgTRES=cpu=64,mem=95311M,billing=64,gres/gpu=1</li><li>AllocTRES=cpu=40</li></ul></div><div><br></div>So far so good. Both show 40 cores allocated.</div><div><br></div><div><br></div><div><br></div><div>However, if I now add <a href="https://pastebin.com/C0uW0Aut" target="_blank">another job with 60 cores</a>,this happens:</div><div><br></div><div>scontrol on the node:<br></div><div><br></div><div>CfgTRES=cpu=64,mem=95311M,billing=64,gres/gpu=1<br> AllocTRES=cpu=60<br></div><div><br></div><div><br></div><div>squeue</div><div> JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)<br> 413 CPU normal admin R 21:22 1 shavak-DIT400TR-55L<br> 414 CPU normal admin R 19:53 1 shavak-DIT400TR-55L<br> 417 CPU elevated admin R 1:31 1 shavak-DIT400TR-55L<br></div><div><br></div><div>scontrol on the jobids:</div><div><br></div><div>admin@shavak-DIT400TR-55L:~/mpi_runs_inf$ scontrol show job 413|grep NumCPUs<br> NumNodes=1 NumCPUs=20 NumTasks=20 CPUs/Task=1 ReqB:S:C:T=0:0:*:*<br>admin@shavak-DIT400TR-55L:~/mpi_runs_inf$ scontrol show job 414|grep NumCPUs<br> NumNodes=1 NumCPUs=20 NumTasks=20 CPUs/Task=1 ReqB:S:C:T=0:0:*:*<br>admin@shavak-DIT400TR-55L:~/mpi_runs_inf$ scontrol show job 417|grep NumCPUs<br></div> NumNodes=1 NumCPUs=60 NumTasks=60 CPUs/Task=1 ReqB:S:C:T=0:0:*:*<div><br></div><div>So there are 100 CPUs running, according to this, but 60 according to scontrol on the node??????</div><div><br></div><div>The submission scripts are on pastebin:</div><div><br></div><div><a href="https://pastebin.com/s21yXFH2" target="_blank">https://pastebin.com/s21yXFH2<br></a></div><div><a href="https://pastebin.com/C0uW0Aut" target="_blank">https://pastebin.com/C0uW0Aut<br></a></div><div><br></div><div><br></div><div>AR</div><div><br></div><div><br></div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div dir="ltr"><div></div><div>Doug</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, Feb 26, 2023 at 2:43 AM Analabha Roy <<a href="mailto:hariseldon99@gmail.com" target="_blank">hariseldon99@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi Doug,</div><div><br></div><div>Again, many thanks for your detailed response.</div><div>Based on my understanding of your previous note, I did the following:</div><div><br></div><div>I set the nodename with CPUs=64 Boards=1 SocketsPerBoard=2 CoresPerSocket=16 ThreadsPerCore=2</div><div><br></div><div>and the partitions with oversubscribe=force:2</div><div><br></div><div>then I put further restrictions with the default qos to MaxTRESPerNode:cpu=32, MaxJobsPU=MaxSubmit=2</div><div><br></div><div>That way, no single user can request more than 2 X 32 cores legally.</div><div><br></div><div>I launched two jobs, sbatch -n 32 each as one user. They started running immediately, taking up all 64 cores.</div><div><br></div><div>Then I logged in as another user and launched the same job with sbatch -n 2. To my dismay, it started to run!</div><div><br></div><div>Shouldn't slurm have figured out that all 64 cores were occupied and queued the -n 2 job to pending?</div><div><br></div><div>AR</div><div><br></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sun, 26 Feb 2023 at 02:18, Doug Meyer <<a href="mailto:dameyer99@gmail.com" target="_blank">dameyer99@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>You got me, I didn't know that "
oversubscribe=FORCE:2" is an option. I'll need to explore that.</div><div><br></div><div>I missed the question about srun. srun is the preferred I believe. I am not associated with drafting the submit scripts but can ask my peer. You do need to stipulate the number of cores you want. Your "sbatch -n 1" should be changed to the number of MPI ranks you desire.</div><div><br></div><div>As good as slurm is, many come to assume it does far more than it does. I explain slurm as a maître d' in a very exclusive restaurant, aware of every table and the resources they afford. When a reservation is placed, a job submitted, a review of the request versus the resources matches the pending guest/job against the resources and when the other diners/jobs are expected to finish. If a guest requests resources that are not available in the restaurant, the reservation is denied. If a guest arrives and does not need all the resources, the place settings requested but unused are left in reservation until the job finishes. Slurm manages requests against an inventory. Without enforcement, a job that requests 1 core but uses 12 will run. If your 64 core system accepts 64 single core reservations, slurm believing 64 cores are needed, 64 jobs wll start. and then the wait staff (the OS) is left to deal with 768 tasks running on 64 cores. It becomes a sad comedy as the system will probably run out of RAM triggering OOM killer or just run horribly slow. Never assume slurm is going to prevent bad actors once they begin running unless you have configured it to do so.</div><div><br></div><div>We run a very lax environment. We set a standard of 6 GB per job unless the sbatch declares otherwise and a max runtime default. Without an estimated runtime to work with the backfill scheduler is crippled. In an environment mixing single thread and MPI jobs of various sizes it is critical the jobs are honest in their requirements providing slurm the information needed to correctly assign resources.<br></div><div><br></div><div>Doug<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, Feb 25, 2023 at 12:04 PM Analabha Roy <<a href="mailto:hariseldon99@gmail.com" target="_blank">hariseldon99@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>Thanks for your considered response. Couple of questions linger...</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Sat, 25 Feb 2023 at 21:46, Doug Meyer <<a href="mailto:dameyer99@gmail.com" target="_blank">dameyer99@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>Declaring cores=64 will absolutely work but if you start running MPI you'll want a more detailed config description. The easy way to read it is "128=2 sockets * 32 corespersocket * 2 threads per core". <br>
</div><div><br></div><div>NodeName=hpc[306-308] CPUs=128 Sockets=2 CoresPerSocket=32 ThreadsPerCore=2 RealMemory=512000 TmpDisk=100 </div><div><br></div><div>
But if you just want to work with logical cores the "cpus=128" will work.<br></div><div><br></div><div>If you go with the more detailed description then you need to declare oversubscription (hyperthreading) in the partition declaration. </div></div></blockquote><div><br></div><div><br></div><div>Yeah, I'll try that.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>By default slurm will not let two different jobs share the logical cores comprising a physical core. For example if Sue has an Array of 1-1000 her array tasks could each take a logical core on a physical core. But if Jamal is also running they would not be able to share the physical core. (as I understand it).<br></div><div><br></div><div>
<p class="MsoNormal">PartitionName=a Nodes= [301-308] Default=No OverSubscribe=YES:2 MaxTime=Infinite State=Up AllowAccounts=cowboys</p><p class="MsoNormal"><br></p><p class="MsoNormal">In the sbatch/srun the user needs to add a declaration "oversubscribe=yes" telling slurm the job can run on both logical cores available. </p></div></div></blockquote><div><br></div><div>How about setting oversubscribe=FORCE:2? That way, users need not add a setting in their scripts.</div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><p class="MsoNormal">In the days on Knight's Landing each core could handle four logical cores but I don't believe there are any current AMD or Intel processors supporting more then two logical cores (hyperthreads per core). The conversation about hyperthreads is difficult as the Intel terminology is logical cores for hyperthreading and cores for physical cores but the tendency is to call the logical cores threads or hyperthreaded cores. This can be very confusing for consumers of the resources.</p><p class="MsoNormal"><br></p><p class="MsoNormal">In any case, if you create an array job of 1-100 sleep jobs, my simplest logical test job, then you can use scontrol show node <nodename> to see the nodes resource configuration as well as consumption. squeue -w <nodename> -i 10 will iteratate every ten seconds to show you the node chomping through the job.</p><p class="MsoNormal"><br></p><p class="MsoNormal">Hope this helps. Once you are comfortable I would urge you to use the NodeName/Partition descriptor format above and encourage your users to declare oversubscription in their jobs. It is a little more work up front but far easier than correcting scripts later.</p><p class="MsoNormal"><br></p><p class="MsoNormal">Doug<br></p>
<p class="MsoNormal"><u></u> <u></u></p>
<p class="MsoNormal"><br></p>
</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Feb 23, 2023 at 9:41 PM Analabha Roy <<a href="mailto:hariseldon99@gmail.com" target="_blank">hariseldon99@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Howdy, and thanks for the warm welcome,</div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, 24 Feb 2023 at 07:31, Doug Meyer <<a href="mailto:dameyer99@gmail.com" target="_blank">dameyer99@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>Hi,</div><div><br></div><div>Did you configure your node definition with the outputs of slurmd -C? Ignore boards. Don't know if it is still true but several years ago declaring boards made things difficult. <br></div><div><br></div></div></blockquote><div><br></div><div>$ slurmd -C<br>NodeName=shavak-DIT400TR-55L CPUs=64 Boards=1 SocketsPerBoard=2 CoresPerSocket=16 ThreadsPerCore=2 RealMemory=95311<br>UpTime=0-00:47:51</div><div>$ grep NodeName /etc/slurm-llnl/slurm.conf<br>NodeName=shavak-DIT400TR-55L CPUs=64 RealMemory=95311 Gres=gpu:1<br></div><div><br></div><div>There is a difference. I, too, discarded the Boards and sockets in slurmd.conf . Is that the problem?</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div></div><div>Also, if you have hyperthreaded AMD or Intel processors your partition declaration should be overscribe:2</div><div><br></div></div></blockquote><div><br></div><div>Yes I do, It's actually 16 X 2 cores with hyperthreading, but the BIOS is set to show them as 64 cores.</div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div></div><div>Start with a very simple job with a script containing sleep 100 or something else without any runtime issues.</div><div><br></div></div></blockquote><div><br></div><div>I ran t<a href="https://github.com/hariseldon99/buparamshavak/blob/main/shavak_root/usr/local/share/examples/mpi_runs_inf/mpi_count.c" target="_blank">his MPI hello world thing </a>with <a href="https://github.com/hariseldon99/buparamshavak/blob/main/shavak_root/usr/local/share/examples/mpi_runs_inf/mpi_count_normal.sbatch" target="_blank">this sbatch script.</a> Should be the same thing as your suggestion, basically.</div><div>Should I switch to 'srun' in the batch file?</div><div><br></div><div>AR</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div></div><div>When I started with slurm I built the sbatch one small step at a time. Nodes, cores. memory, partition, mail, etc <br></div><div><br></div><div>It sounds like your config is very close but your problem may be in the submit script.</div><div><br></div><div>Best of luck and welcome to slurm. It is very powerful with a huge community.</div><div><br></div><div>Doug<br></div><div><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, Feb 23, 2023 at 6:58 AM Analabha Roy <<a href="mailto:hariseldon99@gmail.com" target="_blank">hariseldon99@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Hi folks,<div><br></div><div>I have a single-node "cluster" running Ubuntu 20.04 LTS with the distribution packages for slurm (slurm-wlm 19.05.5)</div><div>Slurm only ran one job in the node at a time with the default configuration, leaving all other jobs pending.</div><div><div>This happened even if that one job only requested like a few cores (the node has 64 cores, and slurm.conf is configged accordingly).</div><div></div></div><div><br></div><div>in slurm conf, SelectType is set to select/cons_res, and SelectTypeParameters to CR_Core. NodeName is set with CPUs=64. Path to file is referenced below.<br></div><div><br></div><div>So I set OverSubscribe=FORCE in the partition config and restarted the daemons. </div><div><br></div><div>Multiple jobs are now run concurrently, but when Slurm is oversubscribed, it is <b>truly</b> <b>oversubscribed</b>. That is to say, it runs so many jobs that there are more processes running than cores/threads.</div><div>How should I config slurm so that it runs multiple jobs at once per node, but ensures that it doesn't run more processes than there are cores? Is there some TRES magic for this that I can't seem to figure out?</div><div><br></div><div>My slurm.conf is here on github: <a href="https://github.com/hariseldon99/buparamshavak/blob/main/shavak_root/etc/slurm-llnl/slurm.conf" target="_blank">https://github.com/hariseldon99/buparamshavak/blob/main/shavak_root/etc/slurm-llnl/slurm.conf</a></div><div>The only gres I've set is for the GPU: <a href="https://github.com/hariseldon99/buparamshavak/blob/main/shavak_root/etc/slurm-llnl/gres.conf" target="_blank">https://github.com/hariseldon99/buparamshavak/blob/main/shavak_root/etc/slurm-llnl/gres.conf</a></div><div><div><br></div><div>Thanks for your attention,</div><div>Regards,</div><div>AR</div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div>Analabha Roy<br></div><div>Assistant Professor</div><div><a href="http://www.buruniv.ac.in/academics/department/physics" target="_blank">Department of Physics</a></div><div><a href="http://www.buruniv.ac.in/" target="_blank">The University of Burdwan</a></div><div>Golapbag Campus, Barddhaman 713104</div><div>West Bengal, India</div><div>Emails: <a href="mailto:daneel@utexas.edu" target="_blank">daneel@utexas.edu</a>, <a href="mailto:aroy@phys.buruniv.ac.in" target="_blank">aroy@phys.buruniv.ac.in</a>, <a href="mailto:hariseldon99@gmail.com" target="_blank">hariseldon99@gmail.com</a><br><div><font face="tahoma, sans-serif">Webpage: <a href="http://www.ph.utexas.edu/~daneel/" target="_blank">http://www.ph.utexas.edu/~daneel/</a></font></div></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div>Analabha Roy<br></div><div>Assistant Professor</div><div><a href="http://www.buruniv.ac.in/academics/department/physics" target="_blank">Department of Physics</a></div><div><a href="http://www.buruniv.ac.in/" target="_blank">The University of Burdwan</a></div><div>Golapbag Campus, Barddhaman 713104</div><div>West Bengal, India</div><div>Emails: <a href="mailto:daneel@utexas.edu" target="_blank">daneel@utexas.edu</a>, <a href="mailto:aroy@phys.buruniv.ac.in" target="_blank">aroy@phys.buruniv.ac.in</a>, <a href="mailto:hariseldon99@gmail.com" target="_blank">hariseldon99@gmail.com</a><br><div><font face="tahoma, sans-serif">Webpage: <a href="http://www.ph.utexas.edu/~daneel/" target="_blank">http://www.ph.utexas.edu/~daneel/</a></font></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div>Analabha Roy<br></div><div>Assistant Professor</div><div><a href="http://www.buruniv.ac.in/academics/department/physics" target="_blank">Department of Physics</a></div><div><a href="http://www.buruniv.ac.in/" target="_blank">The University of Burdwan</a></div><div>Golapbag Campus, Barddhaman 713104</div><div>West Bengal, India</div><div>Emails: <a href="mailto:daneel@utexas.edu" target="_blank">daneel@utexas.edu</a>, <a href="mailto:aroy@phys.buruniv.ac.in" target="_blank">aroy@phys.buruniv.ac.in</a>, <a href="mailto:hariseldon99@gmail.com" target="_blank">hariseldon99@gmail.com</a><br><div><font face="tahoma, sans-serif">Webpage: <a href="http://www.ph.utexas.edu/~daneel/" target="_blank">http://www.ph.utexas.edu/~daneel/</a></font></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div>Analabha Roy<br></div><div>Assistant Professor</div><div><a href="http://www.buruniv.ac.in/academics/department/physics" target="_blank">Department of Physics</a></div><div><a href="http://www.buruniv.ac.in/" target="_blank">The University of Burdwan</a></div><div>Golapbag Campus, Barddhaman 713104</div><div>West Bengal, India</div><div>Emails: <a href="mailto:daneel@utexas.edu" target="_blank">daneel@utexas.edu</a>, <a href="mailto:aroy@phys.buruniv.ac.in" target="_blank">aroy@phys.buruniv.ac.in</a>, <a href="mailto:hariseldon99@gmail.com" target="_blank">hariseldon99@gmail.com</a><br><div><font face="tahoma, sans-serif">Webpage: <a href="http://www.ph.utexas.edu/~daneel/" target="_blank">http://www.ph.utexas.edu/~daneel/</a></font></div></div></div></div></div></div></div>
</blockquote></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr"><div dir="ltr"><div><div dir="ltr"><div>Analabha Roy<br></div><div>Assistant Professor</div><div><a href="http://www.buruniv.ac.in/academics/department/physics" target="_blank">Department of Physics</a></div><div><a href="http://www.buruniv.ac.in/" target="_blank">The University of Burdwan</a></div><div>Golapbag Campus, Barddhaman 713104</div><div>West Bengal, India</div><div>Emails: <a href="mailto:daneel@utexas.edu" target="_blank">daneel@utexas.edu</a>, <a href="mailto:aroy@phys.buruniv.ac.in" target="_blank">aroy@phys.buruniv.ac.in</a>, <a href="mailto:hariseldon99@gmail.com" target="_blank">hariseldon99@gmail.com</a><br><div><font face="tahoma, sans-serif">Webpage: <a href="http://www.ph.utexas.edu/~daneel/" target="_blank">http://www.ph.utexas.edu/~daneel/</a></font></div></div></div></div></div></div></div></div>
</blockquote></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div>Analabha Roy<br></div><div>Assistant Professor</div><div><a href="http://www.buruniv.ac.in/academics/department/physics" target="_blank">Department of Physics</a></div><div><a href="http://www.buruniv.ac.in/" target="_blank">The University of Burdwan</a></div><div>Golapbag Campus, Barddhaman 713104</div><div>West Bengal, India</div><div>Emails: <a href="mailto:daneel@utexas.edu" target="_blank">daneel@utexas.edu</a>, <a href="mailto:aroy@phys.buruniv.ac.in" target="_blank">aroy@phys.buruniv.ac.in</a>, <a href="mailto:hariseldon99@gmail.com" target="_blank">hariseldon99@gmail.com</a><br><div><font face="tahoma, sans-serif">Webpage: <a href="http://www.ph.utexas.edu/~daneel/" target="_blank">http://www.ph.utexas.edu/~daneel/</a></font></div></div></div></div></div></div></div>