<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Helvetica, sans-serif, "EmojiFont", "Apple Color Emoji", "Segoe UI Emoji", NotoColorEmoji, "Segoe UI Symbol", "Android Emoji", EmojiSymbols;">
<p>Sure ;) My example was just for fast reproductivity.<br>
</p>
<p>The complete job farm script is (if that's of interest):<br>
</p>
<p>-----------------------------------------></p>
<p></p>
<div>#!/bin/bash<br>
#SBATCH -J jobfarm_test<br>
#SBATCH -o log.%x.%j.%N.out<br>
#SBATCH -D ./<br>
#SBATCH --mail-type=NONE<br>
#SBATCH --time=00:05:00<br>
#SBATCH --export=NONE<br>
#SBATCH --get-user-env<br>
#SBATCH --clusters=...<br>
#SBATCH --partition=..<br>
#SBATCH --qos=lrz_admin<br>
#SBATCH --nodes=2<br>
#SBATCH --ntasks=4 # needed for SLURM_NTASKS on parallel below<br>
<br>
module load slurm_setup # LRZ specific<br>
module load parallel # GNU parallel<br>
<br>
# Hyperthreading<br>
export OMP_NUM_THREADS=28<br>
export MY_SLURM_PARAMS="-N 1 -n 1 -c 28 --threads-per-core=2 --mem=27G --exact --export=ALL --cpu_bind=verbose,cores --mpi=none"<br>
</div>
<div><br>
</div>
<div>## Not Hyperthreading<br>
#export OMP_NUM_THREADS=14<br>
#export MY_SLURM_PARAMS="-N 1 -n 1 -c 28 --threads-per-core=2 --mem=27G --exact --export=ALL --cpu_bind=verbose,cores --mpi=none"<br>
<br>
export MYEXEC=/lrz/sys/tools/placement_test_2021/bin/placement-test.omp_only<br>
export PARAMS=("-d 20" "-d 10" "-d 20" "-d 10" "-d 20" "-d 10" "-d 20" "-d 10")<br>
<br>
task() {<br>
echo "srun $MY_SLURM_PARAMS $MYEXEC $2 &> log2.$1"<br>
srun $MY_SLURM_PARAMS $MYEXEC $2 &> log2.$1<br>
}<br>
export -f task<br>
<br>
parallel -P $SLURM_NTASKS task {#} {} ::: "${PARAMS[@]}"<br>
</div>
-----------------------------------------><br>
<p></p>
<p>The good thing here is that users may only need to modify the SBATCH header and the exported environment variables.</p>
<p><br>
</p>
<p>But Magnus (Thanks for the Link!) is right. This is still far away from a feature rich job- or task-farming concept, where at least some overview of the passed/failed/missing task statistics is available etc.
<br>
</p>
<p>But as a solution for about some dozens of tasks, the above is imho a feasible and flexible solution (as long as srun keeps playing by the rules).<br>
</p>
<p><br>
</p>
<p>For 1000 tasks, sure, something else is needed. I played with julia's pmap (<a href="https://doku.lrz.de/display/PUBLIC/FAQ%3A+Julia+on+SuperMUC-NG+and+HPC+Systems#FAQ:JuliaonSuperMUCNGandHPCSystems-MoreExamplesandExampleUseCases" class="OWAAutoLink">https://doku.lrz.de/display/PUBLIC/FAQ%3A+Julia+on+SuperMUC-NG+and+HPC+Systems#FAQ:JuliaonSuperMUCNGandHPCSystems-MoreExamplesandExampleUseCases</a>)
what however also reacted quite negative on the srun-interface changes. So, I was a bit diverting from it again. Maybe too easily :scratch_head:</p>
<p>Anyway, Magnus, I will try it.</p>
<p><br>
</p>
<p>Huge thanks you all!</p>
<p>Kind regards,</p>
<p>Martin<br>
</p>
<p><br>
</p>
<p><br>
</p>
<p><br>
</p>
<br>
<br>
<div style="color:rgb(0,0,0)">
<div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>Von:</b> slurm-users <slurm-users-bounces@lists.schedmd.com> im Auftrag von Ward Poelmans <ward.poelmans@vub.be><br>
<b>Gesendet:</b> Mittwoch, 18. Januar 2023 15:00<br>
<b>An:</b> slurm-users@lists.schedmd.com<br>
<b>Betreff:</b> Re: [slurm-users] srun jobfarming hassle question</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt">
<div class="PlainText">Hi Martin,<br>
<br>
Just a tip: use gnu parallel instead of a for loop. Much easier and more powerful.<br>
<br>
Like:<br>
<br>
parallel -j $SLURM_NTASKS srun -N 1 -n 1 -c 1 --exact <command> ::: *.input<br>
<br>
<br>
Ward<br>
</div>
</span></font></div>
</div>
</body>
</html>