<html><head></head><body><div dir="auto">Thank you Sam for this example. I will try to apply this procedure to my study case,<br></div>
<div dir="auto">regards<br><br></div>
<div dir="auto">Alfredo<br><br></div>
<div dir="auto"><!-- tmjah_g_1299s -->Enviado desde <!-- tmjah_g_1299e --><a href="http://www.bluemail.me/r?b=14063"><!-- tmjah_g_1299s -->BlueMail<!-- tmjah_g_1299e --></a><!-- tmjah_g_1299s --> <!-- tmjah_g_1299e --></div>
<div class="gmail_quote" >En 20 de diciembre de 2018, en 00:11, Sam Hawarden <<a href="mailto:sam.hawarden@otago.ac.nz" target="_blank">sam.hawarden@otago.ac.nz</a>> escribió:<blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<p>Hi <span style="color: rgb(33, 33, 33); font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 18.6667px; background-color: rgb(255, 255, 255);">Alfredo</span>,<br> </p>
<p><br> </p>
<p>Beyond what is already suggested, I have used the following script to run a set number of jobs simultaneously within a batch script.<br> </p>
<p><br> </p>
<p></p>
<div>
<span style="font-family: Consolas, monospace; font-size: 10pt;">for thing in ${arrayOfThings[@]}; do echo $<span style="font-family: Consolas, monospace; font-size: 13.3333px; background-color: rgb(255, 255, 255);"><span style="font-family: Consolas, monospace; font-size: 13.3333px; background-color: rgb(255, 255, 255);">thing</span></span>; done | (</span>
</div>
<div>
<span style="font-size: 10pt;"><span style="white-space: pre; font-family: Consolas, monospace;"></span><span style="font-family: Consolas, monospace;">srun -J JobName xargs -I{} --max-procs ${SLURM_JOB_CPUS_PER_NODE} bash -c '{</span></span>
</div>
<div>
<span style="font-size: 10pt;"><span style="white-space: pre; font-family: Consolas, monospace;"></span><span style="font-family: Consolas, monospace;">someCommand -args -and -options {}</span></span>
</div>
<div>
<span style="font-size: 10pt;"><span style="white-space: pre; font-family: Consolas, monospace;"></span><span style="font-family: Consolas, monospace;">}'</span></span>
</div>
<div>
<span style="font-family: Consolas, monospace; font-size: 10pt;">)</span>
</div>
<div>
<span style="font-family: Consolas, monospace; font-size: 10pt;"><br> </span>
</div>
<p>Where {} is $thing<br> </p>
<p><br> </p>
<p>In operation <a href="https://github.com/shawarden/Fastq2VCF/blob/master/slurm-scripts/blockalign.sl#L215"> here</a><br> </p>
<p><br> </p>
<p>Cheers,<br> </p>
<p> Sam<br> </p>
<div id="Signature">
<div name="divtagdefaultwrapper" style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:; margin:0">
<div class="BodyFragment">
<hr style="font-family:Calibri,Arial,Helvetica,sans-serif; text-align:center">
<div class="PlainText">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; text-align:right">
<span style="font-size:large">Sam Hawarden</span>
</div>
<font size="4">
</font><div style="font-family:Calibri,Arial,Helvetica,sans-serif; text-align:right"><font size="4">
Assistant Research Fellow
</font></div><div style="font-family:Calibri,Arial,Helvetica,sans-serif; text-align:right"><font size="4">
Pathology Department
</font></div><div style="font-family:Calibri,Arial,Helvetica,sans-serif; text-align:right"><font size="4">
Dunedin School of Medicine
</font></div><div style="font-family:Calibri,Arial,Helvetica,sans-serif; text-align:right"><font size="4">
sam.hawarden(at)otago.ac.nz
<br>
</font></div><div style="text-align:right"><font size="4"></font></div>
</div>
</div>
</div>
</div>
<div style="color: rgb(33, 33, 33);">
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr">
<font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> slurm-users <slurm-users-bounces@lists.schedmd.com> on behalf of Carlos Fenoy <minibit@gmail.com><br> <b>Sent:</b> Thursday, 20 December 2018 04:59<br> <b>To:</b> Slurm User Community List<br> <b>Subject:</b> Re: [slurm-users] requesting resources and afterwards launch an array of calculations</font>
<div>
</div>
</div>
<div>
<div dir="ltr">
<div dir="ltr">
Hi Alfredo,
<div>
<br>
</div>
<div>
You can have a look at using
<a href="https://github.com/eth-cscs/GREASY">https://github.com/eth-cscs/GREASY</a> . It was developed before array-jobs were supported in slurm and it will do exactly what you want.
</div>
<div>
<br>
</div>
<div>
Regards,
<br> Carlos
</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr">
On Wed, Dec 19, 2018 at 3:33 PM Alfredo Quevedo <
<a href="mailto:maquevedo.unc@gmail.com">maquevedo.unc@gmail.com</a>> wrote:
<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
thank you Michael for the feedback, my scenario is the following: I want
<br> to run a job array of (lets say) 30 jobs. So I setted the slurm input as
<br> follows:
<br>
<br> #SBATCH --array=1-104%30
<br> #SBATCH --ntasks=1
<br>
<br> however only 4 jobs within the array are launched at a time due to the
<br> allowed max number of jobs as setted in the slurm configuration (4). As
<br> a workaround to the issued, the sysadmin suggested me to request the
<br> resources, and afterwards distribute the resources asigned into a
<br> multiple set of single CPU task. I believe that with the solution you
<br> mentioned only 30 (out of the 104) jobs will be finished?
<br>
<br> thanks
<br>
<br> Alfredo
<br>
<br>
<br> El 19/12/2018 a las 11:15, Renfro, Michael escribió:
<br> > Literal job arrays are built into Slurm:
<a href="https://slurm.schedmd.com/job_array.html" rel="noreferrer" target="_blank"> https://slurm.schedmd.com/job_array.html</a>
<br> >
<br> > Alternatively, if you wanted to allocate a set of CPUs for a parallel task, and then run a set of single-CPU tasks in the same job, something like:
<br> >
<br> > #!/bin/bash
<br> > #SBATCH --ntasks=30
<br> > srun --ntasks=${SLURM_NTASKS} hostname
<br> >
<br> > is one way of doing it. If that’s not what you’re looking for, some other details would be needed.
<br> >
<br>
<br>
</blockquote>
</div>
<br clear="all">
<div>
<br>
</div> --
<br>
<div dir="ltr" class="gmail_signature">
--
<br> Carles Fenoy
<br>
</div>
</div>
</div></blockquote></div></body></html>