<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
span.EmailStyle19
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">I'd probably write a shell function that would calculate the time required, and add it as a command-line parameter to sbatch. We do a similar thing for easier interactive shells in our /etc/profile.d folder on the login node:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">function hpcshell() {<o:p></o:p></p>
<p class="MsoNormal"> srun --partition=interactive $@ --pty bash -i<o:p></o:p></p>
<p class="MsoNormal">}<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">So something like (untested):<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">weekendjob () {<o:p></o:p></p>
<p class="MsoNormal"> WEEKRESEND=$(scontrol show res | head -n1 | awk '{print $3}' | cut -f2 -d= | xargs -I {} date +%s --date "{}" )<o:p></o:p></p>
<p class="MsoNormal"> NOW=$(date +%s)<o:p></o:p></p>
<p class="MsoNormal"> HOURS=$(((WEEKRESEND - NOW) / 3600))<o:p></o:p></p>
<p class="MsoNormal"> sbatch --time=${HOURS}:00:00 $@<o:p></o:p></p>
<p class="MsoNormal">}<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">and:<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"> weekendjob myjob.sh<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">could work, with the possible downside that anyone using this early in the reservation period would potentially reserve resources for the entire weekend.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="margin-bottom:12.0pt"><b><span style="font-size:12.0pt;color:black">From:
</span></b><span style="font-size:12.0pt;color:black">slurm-users <slurm-users-bounces@lists.schedmd.com> on behalf of Jeremy Fix <Jeremy.Fix@centralesupelec.fr><br>
<b>Date: </b>Tuesday, March 30, 2021 at 2:24 AM<br>
<b>To: </b>Florian Zillner <fzillner@lenovo.com>, slurm-users@lists.schedmd.com <slurm-users@lists.schedmd.com><br>
<b>Subject: </b>Re: [slurm-users] [External] Autoset job TimeLimit to fit in a reservation<o:p></o:p></span></p>
</div>
<p align="center" style="text-align:center;background:white"><b><span style="font-size:12.0pt;color:red;background:white">External Email Warning</span></b><o:p></o:p></p>
<p align="center" style="mso-margin-top-alt:0in;margin-right:12.0pt;margin-bottom:0in;margin-left:12.0pt;margin-bottom:.0001pt;text-align:center;background:white">
<b><span style="font-size:12.0pt;color:red">This email originated from outside the university. Please use caution when opening attachments, clicking links, or responding to requests.</span></b><o:p></o:p></p>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="0" width="100%" align="center">
</div>
<div>
<div>
<p class="MsoNormal">Hi Florian, and first of all , thanks for this single line command which is already quite a lot; To give it a bit more sense, I will compute the remaining time until the reservation ends, so just changing $2 in $3 in your command<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"># WEEKRESEND=$(scontrol show res | head -n1 | awk '{print $3}' | cut -f2 -d= | xargs -I {} date +%s --date "{}" )
<o:p></o:p></p>
<div>
<p class="MsoNormal"># NOW=$(date +%s)<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"># echo "$(((WEEKRESEND - NOW) / 3600)) hours left until reservation ends"<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">178 hours left until reservation ends<o:p></o:p></p>
</div>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Trying to give some sense, here is the use case. We use slurm for allocating resources for teaching purposes. We want to allow the students to allocate as they wish during the week ends, starting friday evening up to monday early morning.
For that we think about creating a reservation that is regularly scheduled every week and assigned to a partition with part of the nodes we want the students to be able to work with. Then comes the reason why we need to set the timelimit. The partition
have their timelimit and we set it to, say, 2 days and a half for a job submitted on friday evening to be allowed to run until monday morning. Now, suppose a user starts a job on sunday evening without specifying the timelimit of its jobs, the default will
be the partition's one (2 days and a half) and if there is a reservation scheduled on monday morning, for a practical say, if I'm not wrong, slurm will not allow the allocation because it will consider that sunday evening + the default partition timelimit
will overlap with a planned reservation, which makes sense. The first answer could be to ask the user to specify the timelimit on its job so that it's allocation does not overlap with the next reservation but we thought about automatically setting that,
hence my question. <o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">So, in our use case, it's not a problem to kill a job by the end of the reservation even if the job is not completed because we may need the resources for a practical on monday morning.
<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Now, back to your proposal; I was thinking about putting that line in a job prolog but .... maybe that's was you meant by "putting the cart before the horse", I may run into troubles : for the prolog to be executed, the job has to the
allocated but It will not with the default timelimit of the partition because slurm may not allow it (given a practical may be reserved on monday morning) . Do you think there is any other way than asking the user to set the timelimit himself in its srun/sbatch
command ?<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Best;<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">Jeremy.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal">On 29/03/2021 22:09, Florian Zillner wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<div>
<p class="MsoNormal"><span style="color:black">Hi,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">well, I think you're putting the cart before the horse, but anyway, you could write a script that extracts the next reservation and does some simple math to display the time in hours or else to the user. It's the
users job to set the time their job needs to finish. Auto-squeezing a job that takes 2 days to complete into a remaining 2 hour window until the reservation starts doesn't make any sense to me.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"># NEXTRES=$(scontrol show res | head -n1 | awk '{print $2}' | cut -f2 -d= | xargs -I {} date +%s --date "{}" )
<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="color:black"># NOW=$(date +%s)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"># echo "$(((NEXTRES - NOW) / 3600)) hours left until reservation begins"<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">178 hours left until reservation begins<o:p></o:p></span></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">Cheers,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">Florian<o:p></o:p></span></p>
</div>
<div id="Signature">
<div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</div>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="0" width="100%" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> slurm-users
<a href="mailto:slurm-users-bounces@lists.schedmd.com"><slurm-users-bounces@lists.schedmd.com></a> on behalf of Jeremy Fix
<a href="mailto:Jeremy.Fix@centralesupelec.fr"><Jeremy.Fix@centralesupelec.fr></a><br>
<b>Sent:</b> Monday, 29 March 2021 10:48<br>
<b>To:</b> <a href="mailto:slurm-users@lists.schedmd.com">slurm-users@lists.schedmd.com</a>
<a href="mailto:slurm-users@lists.schedmd.com"><slurm-users@lists.schedmd.com></a><br>
<b>Subject:</b> [External] [slurm-users] Autoset job TimeLimit to fit in a reservation</span>
<o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi,<br>
<br>
I'm wondering if there is any built-in option to autoset a job TimeLimit<br>
to fit within a defined reservation.<br>
<br>
For now, it seems to me that the timelimit must be explicitely provided,<br>
in a agreement with the deadline of the reservation, by a user when<br>
invoking the srun or sbatch command while I would find it comfortable to<br>
let slurm calculate the remaining time of the reservation on which<br>
srun/sbatch is executed and fill in the TimeLimit accordingly;<br>
<br>
Best;<br>
<br>
Jeremy.<br>
<br>
<o:p></o:p></p>
</div>
</div>
</blockquote>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
</body>
</html>