<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="DE" link="#0563C1" vlink="#954F72" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Hello,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span lang="EN-US">one of our applications need hybrid OpenMPI and OpenMP Job-Submit.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">Only one task is allowed on one node, but this task should use all cores of the node.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US">So, for example I made:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><i><span lang="EN-US">#!/bin/bash<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:35.4pt"><i><span lang="EN-US"><o:p> </o:p></span></i></p>
<p class="MsoNormal" style="margin-left:35.4pt"><i><span lang="EN-US">#SBATCH --nodes=5<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:35.4pt"><i><span lang="EN-US">#SBATCH --ntasks=5<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:35.4pt"><i><span lang="EN-US">#SBATCH --cpus-per-task=44<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:35.4pt"><i><span lang="EN-US">#SBATCH --export=ALL<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:35.4pt"><i><span lang="EN-US"><o:p> </o:p></span></i></p>
<p class="MsoNormal" style="margin-left:35.4pt"><i><span lang="EN-US">export OMP_NUM_THREADS=44<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:35.4pt"><i><span lang="EN-US">mpiexec PreonNode test.prscene<o:p></o:p></span></i></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black;mso-fareast-language:DE">But the job does not take more than one thread:<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black;mso-fareast-language:DE"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><span lang="EN-US" style="color:black;mso-fareast-language:DE">…<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:35.4pt"><i><span lang="EN-US" style="color:black;mso-fareast-language:DE">Thread binding will be disabled because the full machine is not available for the process.<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:35.4pt"><b><i><span lang="EN-US" style="color:black;mso-fareast-language:DE">Detected 44 CPU threads</span></i></b><i><span lang="EN-US" style="color:black;mso-fareast-language:DE">, 2 l3 caches and 2 packages on the
machine.<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:35.4pt"><b><i><span lang="EN-US" style="color:black;mso-fareast-language:DE">Number of CPU processors reported by OpenMP: 1<o:p></o:p></span></i></b></p>
<p class="MsoNormal" style="margin-left:35.4pt"><b><i><span lang="EN-US" style="color:black;mso-fareast-language:DE">Maximum number of CPU threads reported by OpenMP: 44<o:p></o:p></span></i></b></p>
<p class="MsoNormal" style="margin-left:35.4pt"><i><span lang="EN-US" style="color:black;mso-fareast-language:DE"><o:p> </o:p></span></i></p>
<p class="MsoNormal" style="margin-left:35.4pt"><i><span lang="EN-US" style="color:black;mso-fareast-language:DE">Warning:
<b>OMP_NUM_THREADS was set to 44, which is higher than the number of available processors of
</b>1. Will use 1 threads now.<o:p></o:p></span></i></p>
<p class="MsoNormal" style="margin-left:35.4pt"><i><span lang="EN-US" style="color:black;mso-fareast-language:DE">…<o:p></o:p></span></i></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black;mso-fareast-language:DE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black;mso-fareast-language:DE">What did I wrong?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black;mso-fareast-language:DE">Does anyone have any idea why OpenMP thinks it can only use one thread per node?<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black;mso-fareast-language:DE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black;mso-fareast-language:DE">Thanks !<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black;mso-fareast-language:DE"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black;mso-fareast-language:DE">Best regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-US" style="color:black;mso-fareast-language:DE">Brigitte Selch<o:p></o:p></span></p>
<p class="MsoNormal"><b><span lang="EN-US" style="color:black;mso-fareast-language:DE"><o:p> </o:p></span></b></p>
<p class="MsoNormal"><b><span lang="EN-US" style="color:black;mso-fareast-language:DE">MAN Truck & Bus SE<o:p></o:p></span></b></p>
<p class="MsoNormal"><span style="color:black;mso-fareast-language:DE">IT Produktentwicklung Simulation (FIDD)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black;mso-fareast-language:DE">Vogelweiher Str. 33<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black;mso-fareast-language:DE">90441 Nürnberg<o:p></o:p></span></p>
</div>
<br>
<hr>
<font face="Arial" color="Gray" size="1"><br>
MAN Truck & Bus SE<br>
Sitz der Gesellschaft: München<br>
Registergericht: Amtsgericht München, HRB 247520<br>
Vorsitzender des Aufsichtsrats: Christian Levin, Vorstand: Alexander Vlaskamp (Vorsitzender), Murat Aksel, Friedrich-W. Baumann, Michael Kobriger, Inka Koljonen, Arne Puls, Dr. Frederik Zohm<br>
<br>
You can find information about how we process your personal data and your rights in our data protection notice: www.man.eu/data-protection-notice<br>
<br>
This e-mail (including any attachments) is confidential and may be privileged.<br>
If you have received it by mistake, please notify the sender by e-mail and delete this message from your system.<br>
Any unauthorised use or dissemination of this e-mail in whole or in part is strictly prohibited.<br>
Please note that e-mails are susceptible to change.<br>
MAN Truck & Bus SE (including its group companies) shall not be liable for the improper or incomplete transmission of the information contained in this communication nor for any delay in its receipt.<br>
MAN Truck & Bus SE (or its group companies) does not guarantee that the integrity of this communication has been maintained nor that this communication is free of viruses, interceptions or interference.<br>
<br>
</font>
</body>
</html>