<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle18
{mso-style-type:personal;
font-family:"Calibri",sans-serif;
color:#1F497D;}
span.EmailStyle19
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Howdy,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Yes there is a difference between those two submissions. You are correct that usually ntasks is for mpi and cpus-per-task is for multithreading, but let’s look
at your commands.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">For your first example, the “sbatch --ntasks 24 […]”, this will allocated a job with 24 tasks. These tasks in this case are only 1 CPUs, but may be split across
multiple nodes. So you get a total of 24 CPUs across multiple nodes.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">For your second example, the “sbatch --ntasks 1 --cpus-per-task 24 [...]”, this will allocated a job with 1 task and 24 CPUs for that task. So you will get a
total of 24 CPUs on a single node.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">So in other words, a task cannot be split across multiple nodes. So using --cpus-per-task will ensure it gets allocated to the same node, while using --ntasks
can and may allocate it to multiple nodes.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Hope this helps.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Matthew<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> slurm-users [mailto:slurm-users-bounces@lists.schedmd.com]
<b>On Behalf Of </b>Miguel Gutiérrez Páez<br>
<b>Sent:</b> Thursday, February 22, 2018 3:07 AM<br>
<b>To:</b> Slurm User Community List <slurm-users@lists.schedmd.com><br>
<b>Subject:</b> Re: [slurm-users] ntasks and cpus-per-task<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi,<o:p></o:p></p>
</div>
<p class="MsoNormal">That was just I thought, ntasks for mpi and cpus-per-task for multithreading.<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">So, for example, if every node has 24 cores, is there any difference between these commands?<o:p></o:p></p>
</div>
<p class="MsoNormal">sbatch --ntasks 24 [...]<o:p></o:p></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">sbatch --ntasks 1 --cpus-per-task 24 [...]<o:p></o:p></p>
</div>
<p class="MsoNormal">regards.<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="MsoNormal">El jue., 22 feb. 2018 a las 9:52, Loris Bennett (<<a href="mailto:loris.bennett@fu-berlin.de">loris.bennett@fu-berlin.de</a>>) escribió:<o:p></o:p></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in">
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi Chris,<br>
<br>
Christopher Samuel <<a href="mailto:chris@csamuel.org" target="_blank">chris@csamuel.org</a>> writes:<br>
<br>
> On 22/02/18 18:49, Miguel Gutiérrez Páez wrote:<br>
><br>
>> What's the real meaning of ntasks? Has cpus-per-task and ntasks the<br>
>> same meaning in sbatch and srun?<br>
><br>
> --ntasks is for parallel distributed jobs, where you can run lots of<br>
> independent processes that collaborate using some form of communication<br>
> between the processes (usually MPI for HPC).<br>
><br>
> So inside your batch script you would use "srun" to start up the tasks.<br>
><br>
> However, unless you code is written to make use of that interface then<br>
> it's not really going to help you, and so for any multithreaded<br>
> application you need to use --cpus-per-task instead.<br>
<br>
[snip (11 lines)]<br>
<br>
But does it make any difference for a multithreaded program if I have<br>
<br>
#SBATCH --ntasks=4<br>
#SBATCH --nodes=1-1<br>
<br>
rather than<br>
<br>
#SBATCH --ntasks=1<br>
#SBATCH --cpus-per-task=4<br>
<br>
Up to now I have only thought of --cpus-per-task in connection with<br>
hybrid MPI/OpenMP jobs, which we don't actually have. Thus I tend to<br>
tell users to think always in terms of tasks, regardless of whether<br>
these are MPI processes or just threads.<br>
<br>
One downside of my approach is that if the user forgets to specify<br>
--nodes and --ntasks is greater than 1, non-MPI jobs can be assigned to<br>
multiple nodes.<br>
<br>
Cheers,<br>
<br>
Loris<br>
<br>
--<br>
Dr. Loris Bennett (Mr.)<br>
ZEDAT, Freie Universität Berlin Email <a href="mailto:loris.bennett@fu-berlin.de" target="_blank">
loris.bennett@fu-berlin.de</a><o:p></o:p></p>
</blockquote>
</div>
</div>
</body>
</html>