<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Gary,</p>
<p>Well your first issue is using Cyclecloud, but that is mostly
opinion :)<br>
</p>
<p>Your error states there aren't enough CPUs in the partition,
which means we should take a look at the partition settings.</p>
<p>Take a look at 'scontrol show partition hpc' and see how many
nodes are assigned to it. Also check the state of the nodes with
'sinfo'<br>
</p>
<p>It would also be good to ensure the node settings are right. Run
'slurmd -C' on a node and see if the output matches what is in the
config.</p>
<p>Brian Andrus<br>
</p>
<div class="moz-cite-prefix">On 12/13/2022 1:38 AM, Gary Mansell
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CAAsSk-xEejRWrxrKgPmZnVZ0f0=U8HtYXjArgATKTjJneaD9aQ@mail.gmail.com">
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<div dir="ltr">
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)">Dear Slurm
Users, perhaps you can
help me with a problem that I am having using the Scheduler
(I am new to this,
so please forgive me for any stupid
mistakes/misunderstandings).<span></span></span></p>
<div><br>
</div>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)">I am not able to
submit a Multi-Threaded
MPI job on a small demo cluster that I have setup using
Azure CycleCloud that
uses all the 10x CPUs on my cluster, and I don’t understand
why – perhaps you
can explain why and how I can fix this to use all available
CPUs?<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)">The hpc
partition that I have setup
consists of 5 nodes (Azure VM type = Standard_F2s_v2), each
with 2 cpu’s (I
presume that these are Hyperthreaded cores, rather than 2
cpus – but I am not
certain of this)?<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">[azccadmin@ricslurm-hpc-pg0-1
~]$ <span style="background:yellow">cat /proc/cpuinfo</span><span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">processor
: 0<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">vendor_id
:
GenuineIntel<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">cpu
family : 6<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">model
: 106<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">model
name : Intel(R)
Xeon(R) Platinum 8370C CPU @ 2.80GHz<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">stepping
:
6<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">microcode
:
0xffffffff<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">cpu
MHz
: 2793.436<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">cache
size : 49152 KB<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">physical
id : 0<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">siblings
:
2<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">core
id : 0<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">cpu
cores : 1<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">apicid
: 0<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">initial
apicid : 0<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">fpu
: yes<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">fpu_exception
: yes<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">cpuid
level : 21<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">wp
: yes<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">flags
: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov pat pse36
clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm
constant_tsc
rep_good nopl xtopology eagerfpu pni pclmulqdq vmx ssse3 fma
cx16 pcid sse4_1
sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor
lahf_lm abm
3dnowprefetch invpcid_single tpr_shadow vnmi ept vpid
fsgsbase bmi1 hle avx2
smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap
clflushopt avx512cd
avx512bw avx512vl xsaveopt xsavec md_clear<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">bogomips
:
5586.87<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">clflush
size : 64<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">cache_alignment
: 64<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">address
sizes : 46 bits physical, 48
bits virtual<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">power
management:<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New""><span> </span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">processor
: 1<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">vendor_id
:
GenuineIntel<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">cpu
family : 6<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">model
: 106<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">model
name : Intel(R)
Xeon(R) Platinum 8370C CPU @ 2.80GHz<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">stepping
:
6<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">microcode
:
0xffffffff<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">cpu
MHz : 2793.436<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">cache
size : 49152 KB<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">physical
id : 0<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">siblings
:
2<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">core
id : 0<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">cpu
cores : 1<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">apicid
: 1<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">initial
apicid : 1<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">fpu
: yes<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">fpu_exception
: yes<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">cpuid
level : 21<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">wp
: yes<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">flags
: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov pat pse36
clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm
constant_tsc
rep_good nopl xtopology eagerfpu pni pclmulqdq vmx ssse3 fma
cx16 pcid sse4_1
sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor
lahf_lm abm
3dnowprefetch invpcid_single tpr_shadow vnmi ept vpid
fsgsbase bmi1 hle avx2
smep bmi2 erms invpcid rtm avx512f avx512dq rdseed adx smap
clflushopt avx512cd
avx512bw avx512vl xsaveopt xsavec md_clear<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">bogomips
:
5586.87<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">clflush
size : 64<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">cache_alignment
: 64<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">address
sizes : 46 bits physical, 48
bits virtual<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">power
management:</span><span
style="font-size:8pt;font-family:"Courier New""><span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)">This is how
Slurm sees one of the
nodes:<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">[azccadmin@ricslurm-scheduler
LID_CAVITY]$ <span style="background:yellow">scontrol show
nodes</span><span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">NodeName=ricslurm-hpc-pg0-1
Arch=x86_64
CoresPerSocket=1<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
CPUAlloc=0 CPUEfctv=1 CPUTot=1
CPULoad=0.88<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
AvailableFeatures=cloud<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
ActiveFeatures=cloud<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
Gres=(null)<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
NodeAddr=ricslurm-hpc-pg0-1
NodeHostName=ricslurm-hpc-pg0-1 Version=22.05.3<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
OS=Linux 3.10.0-1127.19.1.el7.x86_64 #1
SMP Tue Aug 25 17:23:54 UTC 2020<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
RealMemory=3072 AllocMem=0 FreeMem=1854
Sockets=1 Boards=1<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
State=IDLE+CLOUD ThreadsPerCore=2
TmpDisk=0 Weight=1 Owner=N/A MCS_label=N/A<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
Partitions=hpc<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
BootTime=2022-12-12T17:42:27
SlurmdStartTime=2022-12-12T17:42:28<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
LastBusyTime=2022-12-12T17:52:29<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
CfgTRES=cpu=1,mem=3G,billing=1<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
AllocTRES=<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
CapWatts=n/a<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
CurrentWatts=0 AveWatts=0<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
ExtSensorsJoules=n/s ExtSensorsWatts=0
ExtSensorsTemp=n/s<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)">This is the
Slurm Job Control
Script I have come up with to run the Vectis Job (I have set
5x Node, 1x CPU,
and 2x Threads – is this right?):<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#!/bin/bash<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New""><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">##
Job name<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#SBATCH
--job-name=run-grma<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">##
File to write standard output and error<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#SBATCH
--output=run-grma.out<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#SBATCH
--error=run-grma.err<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">##
Partition for the cluster (you might not need that)<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#SBATCH
--partition=hpc<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">##
Number of nodes<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#SBATCH
--nodes=5<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">##
Number of CPUs per nodes<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#SBATCH
--ntasks-per-node=1<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">##
Number
of CPUs per task<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#SBATCH
--cpus-per-task=2<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New""><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">##
General<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">module
purge<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New""><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">##
Initialise VECTIS 2022.3b4<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">if
[
-d /shared/apps/RealisSimulation/2022.3/bin ]<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">then<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
export PATH=$PATH:/shared/apps/RealisSimulation/2022.3/bin<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">else<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
echo "Failed to Initialise VECTIS"<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">fi<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New""><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">##
Run<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New""><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">vpre
-V 2022.3 -np $SLURM_NTASKS
/shared/data/LID_CAVITY/files/lid.GRD<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">vsolve
-V 2022.3 -np $SLURM_NTASKS -mpi intel_2018.4 -rdmu
/shared/data/LID_CAVITY/files/lid_no_write.inp<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)">But, the
submitted job will not run as it says that there is not
enough CPUs.<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)">Here is the
debug log from slurmctld – where you can see that it
is saying the job has requested 10 CPUs (which is what I
want), but the hpc
partition only has 5 (which I think is wrong?):<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.177]
debug2: Processing RPC:
REQUEST_NODE_INFO from UID=0<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.370]
debug2: Processing RPC:
REQUEST_SUBMIT_BATCH_JOB from UID=20001<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.371]
debug3: _set_hostname:
Using auth hostname for alloc_node: ricslurm-scheduler<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.371]
debug3: JobDesc:
user_id=20001 JobId=N/A partition=hpc name=run-grma<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.371]
debug3:
cpus=10-4294967294 pn_min_cpus=2 core_spec=-1<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.371]
debug3:
Nodes=5-[5] Sock/Node=65534 Core/Sock=65534
Thread/Core=65534<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.371]
debug3:
pn_min_memory_job=18446744073709551615 pn_min_tmp_disk=-1<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.371]
debug3:
immediate=0 reservation=(null)<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.371]
debug3:
features=(null) batch_features=(null)
cluster_features=(null) prefer=(null)<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.371]
debug3:
req_nodes=(null) exc_nodes=(null)<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.371]
debug3:
time_limit=15-15 priority=-1 contiguous=0 shared=-1<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.371]
debug3:
kill_on_node_fail=-1 script=#!/bin/bash<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New""><span> </span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">##
Job name<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">#SBATCH
--job-n...<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.371]
debug3:
argv="/shared/data/LID_CAVITY/slurm-runit.sh"<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.371]
debug3:
environment=XDG_SESSION_ID=12,HOSTNAME=ricslurm-scheduler,SELINUX_ROLE_REQUESTED=,...<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.371]
debug3:
stdin=/dev/null stdout=/shared/data/LID_CAVITY/run-grma.out
stderr=/shared/data/LID_CAVITY/run-grma.err<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
debug3:
work_dir=/shared/data/LID_CAVITY
alloc_node:sid=ricslurm-scheduler:13464<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
debug3:
power_flags=<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
debug3:
resp_host=(null) alloc_resp_port=0 other_port=0<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
debug3:
dependency=(null) account=(null) qos=(null) comment=(null)<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
debug3:
mail_type=0 mail_user=(null) nice=0 num_tasks=5 open_mode=0
overcommit=-1
acctg_freq=(null)<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
debug3:
network=(null) begin=Unknown cpus_per_task=2 requeue=-1
licenses=(null)<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
debug3:
end_time= signal=0@0 wait_all_nodes=-1 cpu_freq=<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
debug3:
ntasks_per_node=1 ntasks_per_socket=-1 ntasks_per_core=-1
ntasks_per_tres=-1<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
debug3:
mem_bind=0:(null) plane_size:65534<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
debug3:
array_inx=(null)<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
debug3:
burst_buffer=(null)<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
debug3:
mcs_label=(null)<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
debug3:
deadline=Unknown<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
debug3:
bitflags=0x1a00c000 delay_boot=4294967294<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
debug3: job_submit/lua:
slurm_lua_loadscript: skipping loading Lua script:
/etc/slurm/job_submit.lua<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
lua: Setting reqswitch to
1.<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
lua: returning.<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.372]
debug2:
_part_access_check: <span style="background:yellow">Job
requested too many CPUs (10) of partition hpc(5)</span><span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.373]
debug2: _part_access_check:
Job requested too many CPUs (10) of partition hpc(5)<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Courier New"">[2022-12-13T09:05:01.373]
debug2: JobId=1 can't run
in partition hpc: More processors requested than permitted<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)">The job will run
fine if I use the
below settings (across 5 nodes, but only using one of the
two CPUs on each
node):<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">##
Number of nodes<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#SBATCH
--nodes=5<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">##
Number of CPUs per nodes<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#SBATCH
--ntasks-per-node=1<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">##
Number
of CPUs per task<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">#SBATCH
--cpus-per-task=1<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span>
</span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)">Here is the
successfully submitted Job details showing it using 5
CPU’s (only one CPU per node) across 5x Nodes:<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">[azccadmin@ricslurm-scheduler
LID_CAVITY]$ <span style="background:yellow">scontrol show
job 3</span><span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">JobId=3
JobName=run-grma<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
UserId=azccadmin(20001)
GroupId=azccadmin(20001) MCS_label=N/A<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
Priority=4294901757 Nice=0
Account=(null) QOS=(null)<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
JobState=RUNNING Reason=None
Dependency=(null)<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
Requeue=1 Restarts=0 BatchFlag=1
Reboot=0 ExitCode=0:0<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
RunTime=00:07:35 TimeLimit=00:15:00
TimeMin=N/A<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
SubmitTime=2022-12-12T17:32:01
EligibleTime=2022-12-12T17:32:01<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
AccrueTime=2022-12-12T17:32:01<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
StartTime=2022-12-12T17:42:46
EndTime=2022-12-12T17:57:46 Deadline=N/A<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
SuspendTime=None SecsPreSuspend=0
LastSchedEval=2022-12-12T17:32:01 Scheduler=Main<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
Partition=hpc
AllocNode:Sid=ricslurm-scheduler:11723<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
ReqNodeList=(null) ExcNodeList=(null)<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
NodeList=ricslurm-hpc-pg0-[1-5]<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
BatchHost=ricslurm-hpc-pg0-1<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
<span style="background:yellow">NumNodes=5 NumCPUs=5
NumTasks=5 CPUs/Task=1</span>
ReqB:S:C:T=0:0:*:*<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
TRES=cpu=5,mem=15G,node=5,billing=5<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
Socks/Node=* NtasksPerN:B:S:C=1:0:*:*
CoreSpec=*<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
MinCPUsNode=1 MinMemoryCPU=3G
MinTmpDiskNode=0<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
Features=(null) DelayBoot=00:00:00<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
OverSubscribe=OK Contiguous=0
Licenses=(null) Network=(null)<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
Command=/shared/data/LID_CAVITY/slurm-runit.sh<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
WorkDir=/shared/data/LID_CAVITY<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
StdErr=/shared/data/LID_CAVITY/run-grma.err<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
StdIn=/dev/null<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
StdOut=/shared/data/LID_CAVITY/run-grma.out<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
Switches=1@00:00:24<span></span></span></p>
<p class="MsoNormal" style="margin:0cm 0cm 0cm
36pt;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:8pt;font-family:"Courier New"">
Power=</span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span><br>
</span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)">What am I doing
wrong here - how do
I get it to run the job on both CPU’s on all 5 nodes (i.e.
fully utilising the
available cluster resources of 10x CPUs)?<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)">Regards<span></span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)"><span> </span></span></p>
<p class="MsoNormal"
style="margin:0cm;font-size:11pt;font-family:"Calibri",sans-serif"><span
style="font-size:10pt;font-family:"Century
Gothic",sans-serif;color:rgb(0,0,153)">Gary<span></span></span></p>
</div>
</blockquote>
</body>
</html>