<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
</head>
<body bgcolor="#FFFFFF" text="#000000">
Hi Merlin, <br>
<br>
thanks for the answer, but our user is not in need of a high index,
but in fact in need of 100k taskids.<br>
<br>
<br>
Best<br>
Marcus<br>
<br>
<br>
<div class="moz-cite-prefix">On 2/26/19 3:50 PM, Merlin Hartley
wrote:<br>
</div>
<blockquote type="cite"
cite="mid:B1121CEE-DD80-452E-B8C5-8F449A0484E6@mrc-mbu.cam.ac.uk">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<dt style="box-sizing: border-box; color: rgb(70, 84, 92);
font-family: "Source Sans Pro", Helvetica, Arial,
sans-serif; font-size: 20px; font-variant-ligatures: normal;
orphans: 2; widows: 2; background-color: rgb(255, 255, 255);"
class=""><b style="box-sizing: border-box; margin: 0px; padding:
0px; border: 0px; font-style: inherit; font-variant-ligatures:
inherit; font-variant-position: inherit; font-variant-caps:
inherit; font-variant-numeric: inherit;
font-variant-alternates: inherit; font-variant-east-asian:
inherit; font-stretch: inherit; font-size: inherit;
line-height: inherit; font-family: inherit; vertical-align:
baseline;" class="">max_array_tasks</b></dt>
<dd style="box-sizing: border-box; color: rgb(70, 84, 92);
font-family: "Source Sans Pro", Helvetica, Arial,
sans-serif; font-size: 20px; font-variant-ligatures: normal;
orphans: 2; widows: 2; background-color: rgb(255, 255, 255);"
class="">Specify the maximum number of tasks that be included in
a job array. The default limit is MaxArraySize, but this option
can be used to set a lower limit. For example,
max_array_tasks=1000 and MaxArraySize=100001 would permit a
maximum task ID of 100000, but limit the number of tasks in any
single job array to 1000.</dd>
<div class=""><a href="https://slurm.schedmd.com/slurm.conf.html"
class="" moz-do-not-send="true">https://slurm.schedmd.com/slurm.conf.html</a></div>
<div class=""><br class="">
</div>
<div class="">SchedulerParameters=max_array_tasks=1000</div>
<div class=""><br class="">
</div>
<div class="">MaxArraySize=100000</div>
<div class=""><br class="">
</div>
<div class="">See commit:</div>
<a
href="https://github.com/SchedMD/slurm/commit/09c13fb292a4a6a56b4078de840aae0d4db70309"
class="" moz-do-not-send="true">https://github.com/SchedMD/slurm/commit/09c13fb292a4a6a56b4078de840aae0d4db70309</a>
<div class=""><br class="">
<div class=""><br class="webkit-block-placeholder">
</div>
<div class=""><br class="webkit-block-placeholder">
</div>
<div class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal;
text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; word-spacing: 0px;
-webkit-text-stroke-width: 0px; word-wrap: break-word;
-webkit-nbsp-mode: space; -webkit-line-break:
after-white-space;" class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal;
text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; word-spacing: 0px;
-webkit-text-stroke-width: 0px; word-wrap: break-word;
-webkit-nbsp-mode: space; -webkit-line-break:
after-white-space;" class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal;
text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; word-spacing: 0px;
-webkit-text-stroke-width: 0px; word-wrap: break-word;
-webkit-nbsp-mode: space; -webkit-line-break:
after-white-space;" class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal;
text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; word-spacing: 0px;
-webkit-text-stroke-width: 0px; word-wrap: break-word;
-webkit-nbsp-mode: space; -webkit-line-break:
after-white-space;" class="">--<br class="">
Merlin Hartley<br class="">
Computer Officer</div>
<div style="color: rgb(0, 0, 0); letter-spacing: normal;
text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; word-spacing: 0px;
-webkit-text-stroke-width: 0px; word-wrap: break-word;
-webkit-nbsp-mode: space; -webkit-line-break:
after-white-space;" class="">MRC Mitochondrial
Biology Unit</div>
<div style="color: rgb(0, 0, 0); letter-spacing: normal;
text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; word-spacing: 0px;
-webkit-text-stroke-width: 0px; word-wrap: break-word;
-webkit-nbsp-mode: space; -webkit-line-break:
after-white-space;" class="">University of Cambridge</div>
<div style="color: rgb(0, 0, 0); letter-spacing: normal;
text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; word-spacing: 0px;
-webkit-text-stroke-width: 0px; word-wrap: break-word;
-webkit-nbsp-mode: space; -webkit-line-break:
after-white-space;" class="">Cambridge, CB2 0XY</div>
<div style="color: rgb(0, 0, 0); letter-spacing: normal;
text-align: start; text-indent: 0px; text-transform:
none; white-space: normal; word-spacing: 0px;
-webkit-text-stroke-width: 0px; word-wrap: break-word;
-webkit-nbsp-mode: space; -webkit-line-break:
after-white-space;" class="">United Kingdom</div>
</div>
</div>
</div>
</div>
<div><br class="">
<blockquote type="cite" class="">
<div class="">On 26 Feb 2019, at 14:27, Jeffrey Frey <<a
href="mailto:frey@udel.edu" class=""
moz-do-not-send="true">frey@udel.edu</a>> wrote:</div>
<br class="Apple-interchange-newline">
<div class="">
<div style="word-wrap: break-word; -webkit-nbsp-mode:
space; -webkit-line-break: after-white-space;" class="">
<div class="">Also see "<a
href="https://slurm.schedmd.com/slurm.conf.html"
class="" moz-do-not-send="true">https://slurm.schedmd.com/slurm.conf.html</a>"
for MaxArraySize/MaxJobCount.</div>
<div class=""><br class="">
</div>
We just went through a user-requested adjustment to
MaxArraySize to bump it from 1000 to 10000; as the
documentation states, since each index of an array job
is essentially "a job," you must be sure to also
adjust MaxJobCount (from 10000 to 100000 in our case).
Adjusting MaxJobCount requires a restart of slurmctld;
though the documentation doesn't state it, so does
adjustment of MaxArraySize (scontrol reconfigure will
succeed but leave the previous limit in effect, see "<a
href="https://bugs.schedmd.com/show_bug.cgi?id=6553"
class="" moz-do-not-send="true">https://bugs.schedmd.com/show_bug.cgi?id=6553</a>").
<div class="">
<div class=""><br class="">
</div>
<div class="">The "MaxArraySize" is a bit of a
misnomer since it's really 1 + the top of the valid
range of indices -- "MaxArrayIndex" would be more
apt. Our users were very happy with Grid Engine's
allowance of any index range and striding that
produces no more than "max_aj_tasks" indices; since
moving to Slurm they're forced to come up with their
own index-mapping functionality at times, but the
relatively low MaxArraySize versus what we had in
GridEngine (75000) has been especially frustrating
for them.</div>
<div class=""><br class="">
</div>
<div class="">So far the 10000/100000 combo hasn't
come close to exhausting resources on our slurmctld
nodes; but we haven't actually submitted a couple
10000-index array jobs and enough other jobs to hit
100000 active jobs, so current memory usage isn't an
adequate measure of usage under load. Since the
slurm.conf documentation states:</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
</div>
<blockquote style="margin: 0 0 0 40px; border: none;
padding: 0px;" class="">
<div class="">
<div class="">Performance can suffer with more than
a few hundred thousand jobs. </div>
</div>
</blockquote>
<div class="">
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class="">we're reluctant to increase MaxJobCount
too much higher.</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
</div>
<div class=""><br class="">
<blockquote type="cite" class="">On Feb 26, 2019, at
3:18 AM, Ole Holm Nielsen <<a
href="mailto:Ole.H.Nielsen@fysik.dtu.dk"
class="" moz-do-not-send="true">Ole.H.Nielsen@fysik.dtu.dk</a>>
wrote:<br class="">
<br class="">
On 2/26/19 9:07 AM, Marcus Wagner wrote:<br
class="">
<blockquote type="cite" class="">Does anyone know,
why per default the number of array elements is
limited to 1000?<br class="">
We have one user, who would like to have 100k
array elements!<br class="">
What is more difficult for the scheduler, one
array job with 100k elements or 100k non-array
jobs?<br class="">
Where did you set the limit? Do your users use
array jobs at all?<br class="">
</blockquote>
<br class="">
Google is your friend :-)<br class="">
<br class="">
<a href="https://slurm.schedmd.com/job_array.html"
class="" moz-do-not-send="true">https://slurm.schedmd.com/job_array.html</a><br
class="">
<br class="">
<blockquote type="cite" class="">A new
configuration parameter has been added to
control the maximum job array size:
MaxArraySize. The smallest index that can be
specified by a user is zero and the maximum
index is MaxArraySize minus one. The default
value of MaxArraySize is 1001. The maximum
MaxArraySize supported in Slurm is 4000001. Be
mindful about the value of MaxArraySize as job
arrays offer an easy way for users to submit
large numbers of jobs very quickly.<br class="">
</blockquote>
<br class="">
/Ole<br class="">
<br class="">
</blockquote>
<br class="">
<div class=""><br class="">
::::::::::::::::::::::::::::::::::::::::::::::::::::::<br class="">
Jeffrey T. Frey, Ph.D.<br class="">
Systems Programmer V / HPC Management<br class="">
Network & Systems Services / College of
Engineering<br class="">
University of Delaware, Newark DE 19716<br
class="">
Office: (302) 831-6034 Mobile: (302) 419-4976<br
class="">
::::::::::::::::::::::::::::::::::::::::::::::::::::::<br class="">
<br class="">
<br class="">
<br class="">
</div>
<br class="">
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br class="">
</div>
</blockquote>
<br>
<pre class="moz-signature" cols="72">--
Marcus Wagner, Dipl.-Inf.
IT Center
Abteilung: Systeme und Betrieb
RWTH Aachen University
Seffenter Weg 23
52074 Aachen
Tel: +49 241 80-24383
Fax: +49 241 80-624383
<a class="moz-txt-link-abbreviated" href="mailto:wagner@itc.rwth-aachen.de">wagner@itc.rwth-aachen.de</a>
<a class="moz-txt-link-abbreviated" href="http://www.itc.rwth-aachen.de">www.itc.rwth-aachen.de</a>
</pre>
</body>
</html>