<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body bgcolor="#FFFFFF" text="#000000">
I just made a little bit debugging, setting the debug level to
debug5 during submission.<br>
<br>
I submitted (or at least tried to) two jobs:<br>
<br>
sbatch -n 48 --wrap hostname<br>
got submitted, got jobid 199801<br>
<br>
sbatch -N 1 --ntasks-per-node=48 --wrap hostname<br>
submission denied, got jobid 199805<br>
<br>
The only difference, found in the logs between these two jobs was:<br>
199801: <br>
debug3: ntasks_per_node=-1 ntasks_per_socket=-1
ntasks_per_core=-1<br>
199805:<br>
debug3: ntasks_per_node=48 ntasks_per_socket=-1
ntasks_per_core=-1<br>
<br>
Nonetheless, slurm schedules 199801 onto one host:<br>
sched: Allocate JobId=199801 NodeList=ncm0288 #CPUs=48
Partition=c18m<br>
<br>
<br>
Best<br>
Marcus<br>
<br>
<br>
<div class="moz-cite-prefix">On 2/19/19 2:58 PM, Prentice Bisbal
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:a6295737-9021-a73a-9cf8-ed1bc535f556@pppl.gov">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<p>--ntasks-per-node is meant to be used in conjunction with
--nodes option. From <a class="moz-txt-link-freetext"
href="https://slurm.schedmd.com/sbatch.html"
moz-do-not-send="true">https://slurm.schedmd.com/sbatch.html</a>:</p>
<p> </p>
<blockquote type="cite">
<dl compact="compact">
<dt><b>--ntasks-per-node</b>=<<i>ntasks</i>></dt>
<dd> Request that <i>ntasks</i> be invoked on each node. If
used with the <b>--ntasks</b> option, the <b>--ntasks</b>
option will take precedence and the <b>--ntasks-per-node</b>
will be treated as a <i>maximum</i> count of tasks per
node. Meant to be used with the <b>--nodes</b> option...</dd>
</dl>
</blockquote>
<p>If you don't specify --ntasks, it defaults to --ntasks=1, as
Andreas said. <a class="moz-txt-link-freetext"
href="https://slurm.schedmd.com/sbatch.html"
moz-do-not-send="true">https://slurm.schedmd.com/sbatch.html</a>:
<br>
</p>
<p> </p>
<blockquote type="cite">
<dl compact="compact">
<dt><b>-n</b>, <b>--ntasks</b>=<<i>number</i>></dt>
<dd> sbatch does not launch tasks, it requests an allocation
of resources and submits a batch script. This option advises
the Slurm controller that job steps run within the
allocation will launch a maximum of <i>number</i> tasks and
to provide for sufficient resources. The default is one task
per node, but note that the <b>--cpus-per-task</b> option
will change this default. </dd>
</dl>
</blockquote>
<p>So the correct way to specify your job is either like this</p>
<p>--ntasks=48</p>
<p>or <br>
</p>
<p>--nodes=1 --ntasks-per-node=48</p>
<p>Specifying both --ntasks-per-node and --ntasks at the same time
is not correct. <br>
</p>
<p><br>
</p>
<pre class="moz-signature" cols="72">Prentice </pre>
<div class="moz-cite-prefix">On 2/14/19 1:09 AM, Henkel, Andreas
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:58774C62-1104-4A76-8DC5-BE4BCF2F534D@uni-mainz.de">
<pre class="moz-quote-pre" wrap="">Hi Marcus,
What just came to my mind: if you don’t set —ntasks isn’t the default just 1? All examples I know using ntasks-per-node also set ntasks with ntasks >= ntasks-per-node.
Best,
Andreas
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">Am 14.02.2019 um 06:33 schrieb Marcus Wagner <a class="moz-txt-link-rfc2396E" href="mailto:wagner@itc.rwth-aachen.de" moz-do-not-send="true"><wagner@itc.rwth-aachen.de></a>:
Hi all,
I have narrowed this down a little bit.
the really astonishing thing is, that if I use
--ntasks=48
I can submit the job, it will be scheduled onto one host:
NumNodes=1 NumCPUs=48 NumTasks=48 CPUs/Task=1 ReqB:S:C:T=0:0:*:*
TRES=cpu=48,mem=182400M,node=1,billing=48
but as soon as I change --ntasks to --ntasks-per-node (which should be the same, as --ntasks=48 schedules onto one host), I get the error:
sbatch: error: CPU count per node can not be satisfied
sbatch: error: Batch job submission failed: Requested node configuration is not available
Is there no one else, who observes this behaviour?
Any explanations?
Best
Marcus
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">On 2/13/19 1:48 PM, Marcus Wagner wrote:
Hi all,
I have a strange behaviour here.
We are using slurm 18.08.5-2 on CentOS 7.6.
Let me first describe our computenodes:
NodeName=ncm[0001-1032] CPUs=48 Sockets=4 CoresPerSocket=12 ThreadsPerCore=2 RealMemory=185000 Feature=skx8160,hostok,hpcwork Weight=10541 State=UNKNOWN
we have the following config set:
$>scontrol show config | grep -i select
SelectType = select/cons_res
SelectTypeParameters = CR_CORE_MEMORY,CR_ONE_TASK_PER_CORE
So, I have 48 cores on one node. According to the manpage of sbatch, I should be able to do the following:
#SBATCH --ntasks=48
#SBATCH --ntasks-per-node=48
But I get the following error:
sbatch: error: Batch job submission failed: Requested node configuration is not available
Has anyone an explanation for this?
Best
Marcus
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">--
Marcus Wagner, Dipl.-Inf.
IT Center
Abteilung: Systeme und Betrieb
RWTH Aachen University
Seffenter Weg 23
52074 Aachen
Tel: +49 241 80-24383
Fax: +49 241 80-624383
<a class="moz-txt-link-abbreviated" href="mailto:wagner@itc.rwth-aachen.de" moz-do-not-send="true">wagner@itc.rwth-aachen.de</a>
<a class="moz-txt-link-abbreviated" href="http://www.itc.rwth-aachen.de" moz-do-not-send="true">www.itc.rwth-aachen.de</a>
</pre>
</blockquote>
</blockquote>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Marcus Wagner, Dipl.-Inf.
IT Center
Abteilung: Systeme und Betrieb
RWTH Aachen University
Seffenter Weg 23
52074 Aachen
Tel: +49 241 80-24383
Fax: +49 241 80-624383
<a class="moz-txt-link-abbreviated" href="mailto:wagner@itc.rwth-aachen.de">wagner@itc.rwth-aachen.de</a>
<a class="moz-txt-link-abbreviated" href="http://www.itc.rwth-aachen.de">www.itc.rwth-aachen.de</a>
</pre>
</body>
</html>