<div dir="ltr"><div>Hi Florian,</div><div>Thanks for the valuable reply and help.</div><div><br></div><div>My answers to you are in green.</div><div><br></div><div dir="ltr"><li>Do you have an active support contract with SchedMD? AFAIK they only offer paid support.</li></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><b style="color:rgb(56,118,29)">I don't have an active support contact. I just started learning slurm by installing it on my fedora machine. This is the first time I am installing and experimenting with slurm kind of software.</b><br></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><br></div></blockquote><div dir="ltr"><li><span style="font-family:Calibri,Helvetica,sans-serif;font-size:11pt">The error message is pretty straight forward, slurmctld is not running. Did you start it (systemctl start slurmctld)?</span></li></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><font color="#38761d"><b>I did: systemctl start slurmctld and got this message: Failed to start slurmctld.service: Unit slurmctld.service not found.</b></font></div><div><br></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"></blockquote><div dir="ltr"><li>slurmd needs to run on the node(s) you want to run on as well, and as I'm guessing you are using localhost for the controller and want to run jobs on localhost, so slurmctld and slurmd need to be running on localhost.</li></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><font color="#38761d"><b>systemctl start slurmd</b></font></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><font color="#38761d"><b>Failed to start slurmd.service: Unit slurmd.service not found.</b></font></div></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><font color="#38761d"><b>Similar to slurmctrld</b></font></div><div><font color="#38761d"><b><br></b></font></div></blockquote><div dir="ltr"><li>Is munge running?</li></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><font color="#38761d"><b>Yes. Here is the status:</b></font></div><div><font color="#38761d"><b>[johnsy@homepc ~]$ systemctl status munge</b></font></div><div><font color="#38761d"><b>munge.service - MUNGE authentication service</b></font></div><div><font color="#38761d"><b>     Loaded: loaded (/usr/lib/systemd/system/munge.service; enabled; vendor preset: disabled)</b></font></div><div><font color="#38761d"><b>     Active: active (running) since Mon 2021-04-19 07:49:13 EDT; 13min ago <-- it is always enabled after restart. This log is just after a restart.</b></font></div><div><font color="#38761d"><b>       Docs: man:munged(8)</b></font></div><div><font color="#38761d"><b>    Process: 1070 ExecStart=/usr/sbin/munged (code=exited, status=0/SUCCESS)</b></font></div><div><font color="#38761d"><b>   Main PID: 1072 (munged)</b></font></div><div><font color="#38761d"><b>      Tasks: 4 (limit: 76969)</b></font></div><div><font color="#38761d"><b>     Memory: 1.4M</b></font></div><div><font color="#38761d"><b>        CPU: 8ms</b></font></div><div><font color="#38761d"><b>     CGroup: /system.slice/munge.service</b></font></div><b style="color:rgb(56,118,29)">             └─1072 /usr/sbin/munged</b><br></blockquote><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><br></div></blockquote></blockquote><div dir="ltr"><li>May I ask why you're chown-ing pid and logfiles? The slurm user (typically "slurm") needs to have access to those files. Munge for instance checks for ownership and complains if something is not correct.</li></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div><b><font color="#38761d">I tried to follow some instructions mentioned in: <a href="https://wiki.fysik.dtu.dk/niflheim/Slurm_configuration#copy-slurm-conf-to-all-nodes">https://wiki.fysik.dtu.dk/niflheim/Slurm_configuration#copy-slurm-conf-to-all-nodes</a></font></b></div><div><b><font color="#38761d">I thought, as I am installing the slurm as root, the user "johnsy" has to have ownership permissions.</font></b></div><div><br></div></blockquote><div dir="ltr"><li>"srun /proc/cpuinfo" will fail, even if slurmctld and slurmd are running, because /proc/cpuinfo is not an executable file. You may want to insert "cat" after srun. Another simple test would be "srun hostname"</li></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div class="gmail_quote"><div class="gmail_attr"><font color="#38761d"><b>I tried : srun hostname and got the following error message: </b></font></div></div><div class="gmail_quote"><div class="gmail_attr"><font color="#38761d"><b>srun: error: Unable to allocate resources: Unable to contact slurm controller (connect failure)</b></font></div></div></blockquote><div class="gmail_quote"><div dir="ltr" class="gmail_attr"><br></div></div><blockquote style="margin:0 0 0 40px;border:none;padding:0px"><div class="gmail_quote"><div class="gmail_attr"><b><font color="#38761d">Also tried:</font></b></div></div><div class="gmail_quote"><div class="gmail_attr"><b><font color="#38761d">systemctl status slurmctld</font></b></div></div><div class="gmail_quote"><div class="gmail_attr"><b><font color="#38761d">Unit slurmctld.service could not be found.</font></b></div></div></blockquote><div><br></div><div>Also I tried installing the packaged version: <a href="https://src.fedoraproject.org/rpms/slurm">https://src.fedoraproject.org/rpms/slurm</a> using dnf.</div><div>The same problem exists.</div><div><br></div>Any help in this regard will be appreciated.<div><br></div><div>Thanks a lot.</div><div>Johnsy</div><div><br><div><div class="gmail_quote"><div dir="ltr" class="gmail_attr"><br></div><div dir="ltr" class="gmail_attr">On Mon, Apr 19, 2021 at 5:04 AM Florian Zillner <<a href="mailto:fzillner@lenovo.com">fzillner@lenovo.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">




<div dir="ltr">
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:11pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
Hi Johnsy,</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:11pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
<ol>
<li><span>Do you have an active support contract with SchedMD? AFAIK they only offer paid support.</span></li><li><span style="font-family:Calibri,Helvetica,sans-serif;font-size:11pt">The error message is pretty straight forward, slurmctld is not running. Did you start it (systemctl start slurmctld)?</span><br>
</li><li>slurmd needs to run on the node(s) you want to run on as well, and as I'm guessing you are using localhost for the controller and want to run jobs on localhost, so slurmctld and slurmd need to be running on localhost.</li><li>Is munge running?</li><li>May I ask why you're chown-ing pid and logfiles? The slurm user (typically "slurm") needs to have access to those files. Munge for instance checks for ownership and complains if something is not correct.</li><li>"srun /proc/cpuinfo" will fail, even if slurmctld and slurmd are running, because /proc/cpuinfo is not an executable file. You may want to insert "cat" after srun. Another simple test would be "srun hostname"</li></ol>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:11pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
And, just my personal opinion, if this is your first experiment with Slurm, I wouldn't change too much right from the beginning but instead get it working first and then change things to your needs. Slurm is also available in the EPEL repos, so you could install
 it using dnf and experiment with the packaged version.</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:11pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:11pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
Hope this helps,</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:11pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
Florian</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:11pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:11pt;color:rgb(0,0,0);background-color:rgb(255,255,255)">
<br>
</div>
<div>
<div id="gmail-m_-7468091536853339426Signature">
<div>
<div>
<div></div>
</div>
</div>
</div>
</div>
<div id="gmail-m_-7468091536853339426appendonsend"></div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-7468091536853339426divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> slurm-users <<a href="mailto:slurm-users-bounces@lists.schedmd.com" target="_blank">slurm-users-bounces@lists.schedmd.com</a>> on behalf of Johnsy K. John <<a href="mailto:johnsyjohnk@gmail.com" target="_blank">johnsyjohnk@gmail.com</a>><br>
<b>Sent:</b> Monday, 19 April 2021 01:43<br>
<b>To:</b> <a href="mailto:sales@schedmd.com" target="_blank">sales@schedmd.com</a> <<a href="mailto:sales@schedmd.com" target="_blank">sales@schedmd.com</a>>; johnsy john <<a href="mailto:johnsyjohnk@gmail.com" target="_blank">johnsyjohnk@gmail.com</a>>; <a href="mailto:slurm-users@schedmd.com" target="_blank">slurm-users@schedmd.com</a> <<a href="mailto:slurm-users@schedmd.com" target="_blank">slurm-users@schedmd.com</a>><br>
<b>Subject:</b> [External] [slurm-users] Slurm Configuration assistance: Unable to use srun after installation (slurm on fedora 33)</font>
<div> </div>
</div>
<div>
<div dir="ltr"><font color="#000000">Hello SchedMD team,</font>
<div><font color="#000000"><br>
</font>
<div><font color="#000000">I would like to use your slurm workload manager for learning purposes.</font></div>
<div><font color="#000000">And I tried installing the the software (downloaded from: <a href="https://www.schedmd.com/downloads.php" target="_blank">https://www.schedmd.com/downloads.php</a> ) and followed the steps as mentioned in:</font></div>
<div><font color="#0b5394"><br>
</font></div>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font color="#0b5394"><a href="https://slurm.schedmd.com/download.html" target="_blank">https://slurm.schedmd.com/download.html</a></font></div>
<div><font color="#0b5394"><a href="https://slurm.schedmd.com/quickstart_admin.html" target="_blank">https://slurm.schedmd.com/quickstart_admin.html</a></font></div>
</blockquote>
<div><font color="#0b5394"><br>
</font></div>
<div><font color="#000000">My Linux OS is fedora 33, and i tried installing it as root login.</font></div>
<div><font color="#000000">After installation and configuration as mentioned in page: </font><font color="#0b5394"><a href="https://slurm.schedmd.com/quickstart_admin.html" target="_blank">https://slurm.schedmd.com/quickstart_admin.html</a></font></div>
<div>I got some errors when I tried to do <font color="#0000ff">srun.</font></div>
<div><font color="#000000">Details about the installation and use are as follows:</font></div>
<div><font color="#0b5394"><br>
</font></div>
<div><font color="#000000">Using root permissions, copied to: /root/installations/</font><br>
</div>
<div><font color="#0b5394"><br>
</font></div>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace"><font color="#0b5394">cd /root/installations/</font></font></div>
<div><font face="monospace"><br>
</font></div>
<div><font face="monospace"><font color="#0b5394">tar --bzip -x -f slurm-20.11.5.tar.bz2</font></font></div>
<div><font face="monospace"><br>
</font></div>
<div><font face="monospace"><font color="#0b5394">cd slurm-20.11.5/</font></font></div>
<div><font face="monospace"><br>
</font></div>
<div><font face="monospace"><font color="#0b5394">./configure --enable-debug --prefix=/usr/local --sysconfdir=/usr/local/etc</font></font></div>
<div><font face="monospace"><br>
</font></div>
<div><font face="monospace"><font color="#0b5394">make</font></font></div>
<div><font face="monospace"><font color="#0b5394">make install</font></font></div>
</blockquote>
<div><br>
<font color="#000000">Following steps are based on <a href="https://wiki.fysik.dtu.dk/niflheim/Slurm_configuration" target="_blank">
https://wiki.fysik.dtu.dk/niflheim/Slurm_configuration</a></font><br>
<br>
</div>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font color="#0b5394"><font face="monospace">mkdir /var/spool/slurmctld /var/log/slurm</font></font></div>
<div><font color="#0b5394"><font face="monospace">chown johnsy /var/spool/slurmctld</font></font></div>
<div><font color="#0b5394"><font face="monospace">chown johnsy /var/log/slurm</font></font></div>
<div><font color="#0b5394"><font face="monospace">chmod 755 /var/spool/slurmctld /var/log/slurm</font></font></div>
<div><font color="#0b5394"><font face="monospace"><br>
</font></font></div>
<div><font color="#0b5394"><font face="monospace"> cp /var/run/slurmctld.pid /var/run/slurmd.pid</font></font></div>
<div><font color="#0b5394"><font face="monospace"><br>
</font></font></div>
<div><font color="#0b5394"><font face="monospace">touch /var/log/slurm/slurmctld.log</font></font></div>
<div><font color="#0b5394"><font face="monospace">chown johnsy /var/log/slurm/slurmctld.log</font></font></div>
<div><font color="#0b5394"><font face="monospace"><br>
</font></font></div>
<div><font color="#0b5394"><font face="monospace">touch /var/log/slurm/slurm_jobacct.log /var/log/slurm/slurm_jobcomp.log</font></font></div>
<div><font color="#0b5394"><font face="monospace">chown johnsy /var/log/slurm/slurm_jobacct.log /var/log/slurm/slurm_jobcomp.log</font></font></div>
<div><font color="#0b5394"><font face="monospace"><br>
</font></font></div>
<div><font color="#0b5394"><font face="monospace">ldconfig -n /usr/lib64</font></font></div>
</blockquote>
<div><font color="#0b5394"></font></div>
<div><font color="#0b5394"><br>
</font></div>
<div><font color="#000000">Now when I tried an example command for trial:</font></div>
<div><font color="#0b5394"><br>
</font></div>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font color="#0b5394" face="monospace">srun /proc/cpuinfo</font></div>
</blockquote>
<div><font face="monospace" style="color:rgb(11,83,148)"><br>
</font><font color="#000000">I get the following error:</font></div>
<div><br>
</div>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font color="#ff0000" face="monospace">srun: error: Unable to allocate resources: Unable to contact slurm controller (connect failure)</font></div>
</blockquote>
<div><br>
</div>
<div><br>
</div>
<div>My configuration file slurm.conf f that i created is: </div>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">######################################################################################################</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">######################################################################################################</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394"># slurm.conf file generated by configurator.html.</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394"># Put this file on all nodes of your cluster.</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394"># See the slurm.conf man page for more information.</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">SlurmctldHost=homepc</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#SlurmctldHost=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#DisableRootJobs=NO</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#EnforcePartLimits=NO</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#Epilog=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#EpilogSlurmctld=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#FirstJobId=1</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#MaxJobId=999999</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#GresTypes=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#GroupUpdateForce=0</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#GroupUpdateTime=600</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#JobFileAppend=0</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#JobRequeue=1</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#JobSubmitPlugins=1</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#KillOnBadExit=0</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#LaunchType=launch/slurm</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#Licenses=foo*4,bar</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#MailProg=/bin/mail</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#MaxJobCount=5000</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#MaxStepCount=40000</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#MaxTasksPerNode=128</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">MpiDefault=none</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#MpiParams=ports=#-#</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PluginDir=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PlugStackConfig=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PrivateData=jobs</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">ProctrackType=proctrack/cgroup</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#Prolog=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PrologFlags=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PrologSlurmctld=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PropagatePrioProcess=0</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PropagateResourceLimits=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PropagateResourceLimitsExcept=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#RebootProgram=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">ReturnToService=1</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">SlurmctldPidFile=/var/run/slurmctld.pid</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">SlurmctldPort=6817</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">SlurmdPidFile=/var/run/slurmd.pid</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">SlurmdPort=6818</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">SlurmdSpoolDir=/var/spool/slurmd</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">SlurmUser=johnsy</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#SlurmdUser=root</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#SrunEpilog=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#SrunProlog=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">StateSaveLocation=/var/spool</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">SwitchType=switch/none</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#TaskEpilog=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">TaskPlugin=task/affinity</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#TaskProlog=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#TopologyPlugin=topology/tree</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#TmpFS=/tmp</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#TrackWCKey=no</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#TreeWidth=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#UnkillableStepProgram=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#UsePAM=0</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394"># TIMERS</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#BatchStartTimeout=10</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#CompleteWait=0</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#EpilogMsgTime=2000</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#GetEnvTimeout=2</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#HealthCheckInterval=0</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#HealthCheckProgram=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">InactiveLimit=0</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">KillWait=30</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#MessageTimeout=10</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#ResvOverRun=0</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">MinJobAge=300</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#OverTimeLimit=0</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">SlurmctldTimeout=120</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">SlurmdTimeout=300</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#UnkillableStepTimeout=60</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#VSizeFactor=0</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">Waittime=0</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394"># SCHEDULING</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#DefMemPerCPU=0</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#MaxMemPerCPU=0</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#SchedulerTimeSlice=30</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">SchedulerType=sched/backfill</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">SelectType=select/cons_tres</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">SelectTypeParameters=CR_Core</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394"># JOB PRIORITY</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PriorityFlags=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PriorityType=priority/basic</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PriorityDecayHalfLife=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PriorityCalcPeriod=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PriorityFavorSmall=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PriorityMaxAge=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PriorityUsageResetPeriod=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PriorityWeightAge=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PriorityWeightFairshare=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PriorityWeightJobSize=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PriorityWeightPartition=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#PriorityWeightQOS=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394"># LOGGING AND ACCOUNTING</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#AccountingStorageEnforce=0</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#AccountingStorageHost=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#AccountingStoragePass=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#AccountingStoragePort=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">AccountingStorageType=accounting_storage/none</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#AccountingStorageUser=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">AccountingStoreJobComment=YES</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">ClusterName=cluster</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#DebugFlags=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#JobCompHost=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#JobCompLoc=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#JobCompPass=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#JobCompPort=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">JobCompType=jobcomp/none</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#JobCompUser=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#JobContainerType=job_container/none</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">JobAcctGatherFrequency=30</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">JobAcctGatherType=jobacct_gather/none</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">SlurmctldDebug=info</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#SlurmctldLogFile=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">SlurmdDebug=info</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#SlurmdLogFile=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#SlurmSchedLogFile=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#SlurmSchedLogLevel=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394"># POWER SAVE SUPPORT FOR IDLE NODES (optional)</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#SuspendProgram=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#ResumeProgram=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#SuspendTimeout=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#ResumeTimeout=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#ResumeRate=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#SuspendExcNodes=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#SuspendExcParts=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#SuspendRate=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#SuspendTime=</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">#</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394"># COMPUTE NODES</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">NodeName=localhost CPUs=12 Sockets=1 CoresPerSocket=6 ThreadsPerCore=2 State=UNKNOWN</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">PartitionName=debug Nodes=localhost Default=YES MaxTime=INFINITE State=UP</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">######################################################################################################</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">######################################################################################################</font></div>
</blockquote>
<blockquote style="margin:0px 0px 0px 40px;border:none;padding:0px">
<div><font face="monospace" color="#0b5394">######################################################################################################</font></div>
</blockquote>
</blockquote>
</div>
</div>
</div>
</div>

</blockquote></div></div></div></div>