<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p>Alright. I didn't see that option for GNU parallel. Retrying a task that failed for good reasons, makes maybe not much sense (e.g. due to OOM). And if the farming job timed out, on restart that job, GNU parallel does not start from the former state, does
it? I guess book-keeping is an extra issue, why Magnus probably also used some server including some data base or so.</p>
<p>But ok. GNU parallel's documentation is indeed quite vast. I try to parse it's other/new features (it is also still developed on ... ).</p>
<p><br>
</p>
<p>Concerning Dask ... I heard of it. But never tried ('cos Intel advertised it ... <span>😏</span> ).<br>
</p>
<p>Maybe I should reconsider that.</p>
<p><br>
</p>
<p>Thank you for this input!</p>
<p>KR, Martin<br>
</p>
<br>
<br>
<div style="color: rgb(0, 0, 0);">
<div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>Von:</b> slurm-users <slurm-users-bounces@lists.schedmd.com> im Auftrag von Ward Poelmans <ward.poelmans@vub.be><br>
<b>Gesendet:</b> Mittwoch, 18. Januar 2023 15:35<br>
<b>An:</b> slurm-users@lists.schedmd.com<br>
<b>Betreff:</b> Re: [slurm-users] srun jobfarming hassle question</font>
<div> </div>
</div>
</div>
<font size="2"><span style="font-size:10pt;">
<div class="PlainText">On 18/01/2023 15:22, Ohlerich, Martin wrote:<br>
> But Magnus (Thanks for the Link!) is right. This is still far away from a feature rich job- or task-farming concept, where at least some overview of the passed/failed/missing task statistics is available etc.<br>
<br>
GNU parallel has log output and options to retry failed jobs.<br>
<br>
If you want really fancy stuff, maybe look at dask combined with slurm plugins? It has dashboards for jupyter I believe.<br>
<br>
Ward<br>
</div>
</span></font></div>
</div>
</body>
</html>