<div dir="ltr">On 27 May 2018 at 18:56, Nadav Toledo <span dir="ltr"><<a href="mailto:nadavtoledo@cs.technion.ac.il" target="_blank">nadavtoledo@cs.technion.ac.il</a>></span> wrote:<br><div class="gmail_extra"><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div style="direction:ltr" bgcolor="#FFFFFF">
Hey Lachlan,<br>
<br>
Can you specify how/where you set the walltime and which factor you
use in the accounting system to deprioritse?<br>
<br></div></blockquote><div><br></div><div>walltime is set in slurm.conf per Partition. You can set DefaultTime or MaxTime or both. Search for those terms here</div><div><a href="https://slurm.schedmd.com/slurm.conf.html">https://slurm.schedmd.com/slurm.conf.html</a><br></div><div><br></div><div>Accounting system is using FairShare/Fair Tree</div><div><a href="https://slurm.schedmd.com/fair_tree.html">https://slurm.schedmd.com/fair_tree.html</a></div><div>PDF of presentation -> <a href="https://slurm.schedmd.com/SC14/BYU_Fair_Tree.pdf">https://slurm.schedmd.com/SC14/BYU_Fair_Tree.pdf</a></div><div><br></div><div>Cheers</div><div>L.<br></div><div><br></div><div><br></div><div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div style="direction:ltr" bgcolor="#FFFFFF">
Thanks, Nadav<div><div class="gmail-h5"><br>
<br>
<div class="gmail-m_4691625924338474037moz-cite-prefix">On 27/05/2018 11:34, Lachlan Musicman
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">On 27 May 2018 at 18:23, Nadav Toledo <span dir="ltr"><<a href="mailto:nadavtoledo@cs.technion.ac.il" target="_blank">nadavtoledo@cs.technion.ac.il</a><wbr>></span>
wrote:<br>
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div style="direction:ltr" bgcolor="#FFFFFF"> Hello forum,<br>
<br>
I am trying to deal with idle session for some time, and
haven't found a solution i am happy with.<br>
The scenario is as follow: users using srun for
jupyter-lab(which is fine and even encouraged by me) on
image processing cluster with gpus.<br>
<br>
problem is, I am trying to have some kind of solution to
email/cancel their job if their session is idle for X
amount of hours.<br>
<br>
the w command or xprintidle cannot be used , since they
both work with ssh but not with slurm(checked that)<br>
<br>
Writing a script is not as easy as one might think, If i
run a script in admin user scope, i need later on to
figure out which idle gpu belong to which slurm job.<br>
running a script in the user scope is probably better
idea, but in which way? crontab is running even user is
not logged, how can i force users to run something only
when the job start?<br>
<br>
perhaps some combination of sreport and tres?<br>
</div>
</blockquote>
<div><br>
</div>
<div>Hmm. We address this with accounting. A tight walltime
( 40 minutes) means that most jobs run without worrying
about walltime. But some will need to set it. The
accounting system keeps people honest by making "hogging"
of resources bad for a users job priority - in so much as
their next job will be deprioritsed.</div>
<div><br>
</div>
<div>Letting people know that their next job will not be
de-prioritised if they waste the resources, we find our
users behave responsibly.</div>
<div><br>
</div>
<div>L.<br>
</div>
<div><br>
</div>
<div><br>
</div>
<div> </div>
</div>
</div>
</div>
</blockquote>
<br>
</div></div></div>
</blockquote></div><br></div></div>