<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body dir="auto">
The end of the MPICH section at [1] shows an example using salloc [2].
<div><br>
</div>
<div>Worst case, you should be able to use the output of “scontrol show hostnames” [3] and use that data to make mpiexec command parameters to run one rank per node, similar to what’s shown at the end of the synopsis section of [4].</div>
<div>
<div><br>
</div>
<div>[1] <a href="https://slurm.schedmd.com/mpi_guide.html#mpich2">https://slurm.schedmd.com/mpi_guide.html#mpich2</a></div>
<div>[2] <a href="https://slurm.schedmd.com/salloc.html">https://slurm.schedmd.com/salloc.html</a></div>
<div>[3] <a href="https://slurm.schedmd.com/scontrol.html">https://slurm.schedmd.com/scontrol.html</a><br>
[4] <a href="https://www.mpich.org/static/docs/v3.1/www1/mpiexec.html">https://www.mpich.org/static/docs/v3.1/www1/mpiexec.html</a><br>
<br>
<div dir="ltr">
<div><span style="background-color: rgba(255, 255, 255, 0);">--</span></div>
<span style="background-color: rgba(255, 255, 255, 0);">Mike Renfro, PhD / HPC Systems Administrator, Information Technology Services<br>
<a href="tel:931%20372-3601" dir="ltr" x-apple-data-detectors="true" x-apple-data-detectors-type="telephone" x-apple-data-detectors-result="0/1" style="-webkit-text-decoration-color: rgba(0, 0, 0, 0.258824);">931 372-3601</a> / Tennessee Tech University</span></div>
<div dir="ltr"><br>
<blockquote type="cite">On Nov 25, 2021, at 12:45 PM, Mccall, Kurt E. (MSFC-EV41) <kurt.e.mccall@nasa.gov> wrote:<br>
<br>
</blockquote>
</div>
<blockquote type="cite">
<div dir="ltr">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<p align="center" style="text-align:center;
background:white;margin:0px"><b><span style="font-size:12.0pt;color:red;background:white;
font-family:"Calibri",sans-serif">External Email Warning</span></b></p>
<p align="center" style="text-align:center;
background:white;margin:0px 12pt"><b><span style="font-size:12.0pt;color:red;font-family:"Calibri",sans-serif">This email originated from outside the university. Please use caution when opening attachments, clicking
links, or responding to requests.</span></b><span style="font-size:12.0pt"></span></p>
<hr>
<div>
<div class="WordSection1">
<p class="MsoNormal">I want to launch an MPICH job with sbatch with one task per node (each a manager), while also reserving a certain number of cores on each node for the managers to fill up with spawned workers (via MPI_Comm_spawn). I’d like to avoid using
–exclusive.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">I tried the arguments –ntasks=20 –cpus-per-task=24, but it appears that 20 * 24 tasks will be launched. Is there a way to reserve cores without immediately launching tasks on them? Thanks for any help.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">sbatch: defined options<o:p></o:p></p>
<p class="MsoNormal">sbatch: -------------------- --------------------<o:p></o:p></p>
<p class="MsoNormal">sbatch: cpus-per-task : 24<o:p></o:p></p>
<p class="MsoNormal">sbatch: ignore-pbs : set<o:p></o:p></p>
<p class="MsoNormal">sbatch: ntasks : 20<o:p></o:p></p>
<p class="MsoNormal">sbatch: test-only : set<o:p></o:p></p>
<p class="MsoNormal">sbatch: verbose : 1<o:p></o:p></p>
<p class="MsoNormal">sbatch: -------------------- --------------------<o:p></o:p></p>
<p class="MsoNormal">sbatch: end of defined options<o:p></o:p></p>
<p class="MsoNormal">sbatch: Linear node selection plugin loaded with argument 4<o:p></o:p></p>
<p class="MsoNormal">sbatch: select/cons_res loaded with argument 4<o:p></o:p></p>
<p class="MsoNormal">sbatch: Cray/Aries node selection plugin loaded<o:p></o:p></p>
<p class="MsoNormal">sbatch: select/cons_tres loaded with argument 4<o:p></o:p></p>
<p class="MsoNormal">sbatch: Job 34274 to start at 2021-11-25T12:15:05 using <b><span style="color:red">480 processors</span></b><span style="color:red">
</span>on nodes n[001-020] in partition normal<o:p></o:p></p>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</body>
</html>