<div dir="ltr"><div>I didn't receive the last mail from Mahmood but Marcus is right, Mahmood's heterogeneous job submission seems to be working now.</div><div>Well, separating each pack in the srun command and asking for the correct number of tasks to be launched for each pack is the way I figured the heterogeneous jobs worked with SLURM v18.08.0 (I didn't test it with more recent SLURM versions).<br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Le jeu. 28 mars 2019 à 08:23, Marcus Wagner <<a href="mailto:wagner@itc.rwth-aachen.de">wagner@itc.rwth-aachen.de</a>> a écrit :<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
Hi Mahmood,<br>
<br>
<div class="gmail-m_-4783933080688307837moz-cite-prefix">On 3/28/19 7:33 AM, Mahmood Naderan
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_default" style="font-family:tahoma,sans-serif">>srun
--pack-group=0 --ntasks=2 : --pack-group=1 --ntasks=4 pw.x
-i <a href="http://mos2.rlx.in" target="_blank">mos2.rlx.in</a>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif">Still only one node
is running the processes</div>
</div>
</div>
</div>
</blockquote>
no, the processes are running as had been asked for.<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div class="gmail_default" style="font-family:tahoma,sans-serif">$ squeue<br>
JOBID PARTITION NAME USER ST
TIME NODES NODELIST(REASON)<br>
755+1 QUARTZ myQE ghatee R
0:47 1 rocks7<br>
755+0 QUARTZ myQE ghatee R
0:47 1 compute-0-2<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
compute-0-2 is the first pack (755+0), it should run 2 tasks<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_default" style="font-family:tahoma,sans-serif">$ rocks run host
compute-0-2 "ps aux | grep pw.x"<br>
ghatee 541 0.1 0.0 582048 7604 ? Sl
02:29 0:00 srun --pack-group=0 --ntasks=2 :
--pack-group=1 --ntasks=4 pw.x -i <a href="http://mos2.rlx.in" target="_blank">mos2.rlx.in</a><br>
ghatee 542 0.0 0.0 46452 748 ? S
02:29 0:00 srun --pack-group=0 --ntasks=2 :
--pack-group=1 --ntasks=4 pw.x -i <a href="http://mos2.rlx.in" target="_blank">mos2.rlx.in</a><br>
ghatee 559 99.6 0.1 1930560 129728 ? Rl
02:29 0:52 /home/ghatee/QuantumEspresso621/bin/pw.x -i <a href="http://mos2.rlx.in" target="_blank">mos2.rlx.in</a><br>
ghatee 560 99.7 0.1 1930560 129720 ? Rl
02:29 0:52 /home/ghatee/QuantumEspresso621/bin/pw.x -i <a href="http://mos2.rlx.in" target="_blank">mos2.rlx.in</a><br>
ghatee 590 0.0 0.0 113132 1588 ? Ss
02:30 0:00 bash -c ps aux | grep pw.x<br>
ghatee 629 0.0 0.0 112668 960 ? S
02:30 0:00 grep pw.x<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
process ids 559 and 560<br>
<br>
<br>
rocks7 is the second pack (755+1), it should run 4 tasks<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_default" style="font-family:tahoma,sans-serif">$ rocks run host
rocks7 "ps aux | grep pw.x"<br>
ghatee 16219 99.0 0.1 1930484 127764 ? Rl
10:59 1:00 /home/ghatee/QuantumEspresso621/bin/pw.x -i <a href="http://mos2.rlx.in" target="_blank">mos2.rlx.in</a><br>
ghatee 16220 99.1 0.1 1930524 127764 ? Rl
10:59 1:00 /home/ghatee/QuantumEspresso621/bin/pw.x -i <a href="http://mos2.rlx.in" target="_blank">mos2.rlx.in</a><br>
ghatee 16221 99.0 0.1 1930484 127760 ? Rl
10:59 1:00 /home/ghatee/QuantumEspresso621/bin/pw.x -i <a href="http://mos2.rlx.in" target="_blank">mos2.rlx.in</a><br>
ghatee 16222 99.1 0.1 1930496 127760 ? Rl
10:59 1:00 /home/ghatee/QuantumEspresso621/bin/pw.x -i <a href="http://mos2.rlx.in" target="_blank">mos2.rlx.in</a><br>
ghatee 16391 0.0 0.0 316388 26652 pts/16 Sl+
11:00 0:00 /opt/rocks/bin/python /opt/rocks/bin/rocks
run host rocks7 ps aux | grep pw.x<br>
ghatee 16394 0.0 0.0 113132 1368 pts/16 S+
11:00 0:00 bash -c ps aux | grep pw.x<br>
ghatee 16396 0.0 0.0 112664 952 pts/16 S+
11:00 0:00 grep pw.x<br>
<br>
</div>
</div>
</div>
</div>
</blockquote>
<br>
process ids 16219, 16220, 16221 and 16222<br>
<br>
Or did I miss something?<br>
<br>
<br>
Best<br>
Marcus<br>
<br>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div class="gmail_default" style="font-family:tahoma,sans-serif"><br>
</div>
<div>
<div dir="ltr" class="gmail-m_-4783933080688307837gmail_signature">
<div dir="ltr"><font face="tahoma,sans-serif">Regards,<br>
Mahmood</font><br>
<br>
<br>
</div>
</div>
</div>
<br>
</div>
</div>
<br>
</div>
</blockquote>
<br>
<pre class="gmail-m_-4783933080688307837moz-signature" cols="72">--
Marcus Wagner, Dipl.-Inf.
IT Center
Abteilung: Systeme und Betrieb
RWTH Aachen University
Seffenter Weg 23
52074 Aachen
Tel: +49 241 80-24383
Fax: +49 241 80-624383
<a class="gmail-m_-4783933080688307837moz-txt-link-abbreviated" href="mailto:wagner@itc.rwth-aachen.de" target="_blank">wagner@itc.rwth-aachen.de</a>
<a class="gmail-m_-4783933080688307837moz-txt-link-abbreviated" href="http://www.itc.rwth-aachen.de" target="_blank">www.itc.rwth-aachen.de</a>
</pre>
</div>
</blockquote></div>