<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div dir="ltr">
<div>Hi Mahmood,</div>
<div><br>
</div>
<div>Your running job is requesting 6 CPUs per node (4 nodes, 6 CPUs per node). That means 6 CPUs are being used on node hpc.</div>
<div><br>
</div>
<div>Your queued job is requesting 5 CPUs per node (4 nodes, 5 CPUs per node). In total, if it was running, that would require 11 CPUs on node hpc. But hpc only has 10 cores, so it can't run.</div>
<div><br>
</div>
<div>Sean<br>
</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, 17 Dec 2019 at 20:03, Mahmood Naderan <<a href="mailto:mahmood.nt@gmail.com">mahmood.nt@gmail.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_default" style="font-family:tahoma,sans-serif">Please see the latest update</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"># for i in {0..2}; do scontrol show node compute-0-$i | grep RealMemory; done && scontrol show node hpc | grep RealMemory<br>
RealMemory=64259 AllocMem=1024 FreeMem=57163 Sockets=32 Boards=1<br>
RealMemory=120705 AllocMem=1024 FreeMem=97287 Sockets=32 Boards=1<br>
RealMemory=64259 AllocMem=1024 FreeMem=40045 Sockets=32 Boards=1<br>
RealMemory=64259 AllocMem=1024 FreeMem=24154 Sockets=10 Boards=1</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif">$ sbatch slurm_qe.sh<br>
Submitted batch job 125<br>
$ squeue<br>
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)<br>
125 SEA qe-fb mahmood PD 0:00 4 (Resources)<br>
124 SEA U1phi1 abspou R 3:52 4 compute-0-[0-2],hpc<br>
$ scontrol show -d job 125<br>
JobId=125 JobName=qe-fb<br>
UserId=mahmood(1000) GroupId=mahmood(1000) MCS_label=N/A<br>
Priority=1751 Nice=0 Account=fish QOS=normal WCKey=*default<br>
JobState=PENDING Reason=Resources Dependency=(null)<br>
Requeue=1 Restarts=0 BatchFlag=1 Reboot=0 ExitCode=0:0<br>
DerivedExitCode=0:0<br>
RunTime=00:00:00 TimeLimit=30-00:00:00 TimeMin=N/A<br>
SubmitTime=2019-12-17T12:29:08 EligibleTime=2019-12-17T12:29:08<br>
AccrueTime=2019-12-17T12:29:08<br>
StartTime=Unknown EndTime=Unknown Deadline=N/A<br>
SuspendTime=None SecsPreSuspend=0 LastSchedEval=2019-12-17T12:29:09<br>
Partition=SEA AllocNode:Sid=<a href="http://hpc.scu.ac.ir:22742" target="_blank">hpc.scu.ac.ir:22742</a><br>
ReqNodeList=(null) ExcNodeList=(null)<br>
NodeList=(null)<br>
NumNodes=4-4 NumCPUs=20 NumTasks=20 CPUs/Task=1 ReqB:S:C:T=0:0:*:*<br>
TRES=cpu=20,mem=40G,node=4,billing=20<br>
Socks/Node=* NtasksPerN:B:S:C=5:0:*:* CoreSpec=*<br>
MinCPUsNode=5 MinMemoryNode=10G MinTmpDiskNode=0<br>
Features=(null) DelayBoot=00:00:00<br>
OverSubscribe=OK Contiguous=0 Licenses=(null) Network=(null)<br>
Command=/home/mahmood/qe/f_borophene/slurm_qe.sh<br>
WorkDir=/home/mahmood/qe/f_borophene<br>
StdErr=/home/mahmood/qe/f_borophene/my_fb.log<br>
StdIn=/dev/null<br>
StdOut=/home/mahmood/qe/f_borophene/my_fb.log<br>
Power=<br>
<br>
$ cat slurm_qe.sh<br>
#!/bin/bash<br>
#SBATCH --job-name=qe-fb<br>
#SBATCH --output=my_fb.log<br>
#SBATCH --partition=SEA<br>
#SBATCH --account=fish<br>
#SBATCH --mem=10GB<br>
#SBATCH --nodes=4<br>
#SBATCH --ntasks-per-node=5<br>
mpirun -np $SLURM_NTASKS /share/apps/q-e-qe-6.5/bin/pw.x -in <a href="http://f_borophene_scf.in" target="_blank">
f_borophene_scf.in</a><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif">You can also see the job detail of 124</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif">$ scontrol show -d job 124<br>
JobId=124 JobName=U1phi1<br>
UserId= abspou(1002) GroupId= abspou(1002) MCS_label=N/A<br>
Priority=958 Nice=0 Account=fish QOS=normal WCKey=*default<br>
JobState=RUNNING Reason=None Dependency=(null)<br>
Requeue=1 Restarts=0 BatchFlag=1 Reboot=0 ExitCode=0:0<br>
DerivedExitCode=0:0<br>
RunTime=00:06:17 TimeLimit=30-00:00:00 TimeMin=N/A<br>
SubmitTime=2019-12-17T12:25:17 EligibleTime=2019-12-17T12:25:17<br>
AccrueTime=2019-12-17T12:25:17<br>
StartTime=2019-12-17T12:25:17 EndTime=2020-01-16T12:25:17 Deadline=N/A<br>
SuspendTime=None SecsPreSuspend=0 LastSchedEval=2019-12-17T12:25:17<br>
Partition=SEA AllocNode:Sid=<a href="http://hpc.scu.ac.ir:20085" target="_blank">hpc.scu.ac.ir:20085</a><br>
ReqNodeList=(null) ExcNodeList=(null)<br>
NodeList=compute-0-[0-2],hpc<br>
BatchHost=compute-0-0<br>
NumNodes=4 NumCPUs=24 NumTasks=24 CPUs/Task=1 ReqB:S:C:T=0:0:*:*<br>
TRES=cpu=24,mem=4G,node=4,billing=24<br>
Socks/Node=* NtasksPerN:B:S:C=6:0:*:* CoreSpec=*<br>
Nodes=compute-0-[0-2],hpc CPU_IDs=0-5 Mem=1024 GRES=<br>
MinCPUsNode=6 MinMemoryNode=1G MinTmpDiskNode=0<br>
Features=(null) DelayBoot=00:00:00<br>
OverSubscribe=OK Contiguous=0 Licenses=(null) Network=(null)<br>
Command=/home/abspou/OpenFOAM/abbaspour-6/run/laminarSMOKEPhi1U1/slurm_script.sh<br>
WorkDir=/home/abspou/OpenFOAM/abbaspour-6/run/laminarSMOKEPhi1U1<br>
StdErr=/home/abspou/OpenFOAM/abbaspour-6/run/laminarSMOKEPhi1U1/alpha3.45U1phi1lamSmoke.log<br>
StdIn=/dev/null<br>
StdOut=/home/abspou/OpenFOAM/abbaspour-6/run/laminarSMOKEPhi1U1/alpha3.45U1phi1lamSmoke.log<br>
Power=</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif">I can not figure out what is the root of the problem.</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div>
<div dir="ltr">
<div dir="ltr"><font face="tahoma,sans-serif">Regards,<br>
Mahmood</font><br>
<br>
<br>
</div>
</div>
</div>
<br>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Tue, Dec 17, 2019 at 11:18 AM Marcus Wagner <<a href="mailto:wagner@itc.rwth-aachen.de" target="_blank">wagner@itc.rwth-aachen.de</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">Dear Mahmood,<br>
<br>
could you please show the output of<br>
<br>
scontrol show -d job 119<br>
<br>
Best<br>
Marcus </div>
</blockquote>
</div>
</div>
</blockquote>
</div>
</body>
</html>