<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hello Alain,</p>
<p>as an alternative to job_container/tmpfs, you may also try your
luck with the 'auto_tmpdir' SPANK plugin:
<a class="moz-txt-link-freetext" href="https://github.com/University-of-Delaware-IT-RCI/auto_tmpdir">https://github.com/University-of-Delaware-IT-RCI/auto_tmpdir</a></p>
<p>We've been using using that on our small HPC cluster (Slurm
22.05) and it does what it's supposed to. One thing one has to
remember is that it requires a recompile after every Slurm update.</p>
<p>Kind regards,<br>
René Sitt<br>
</p>
<div class="moz-cite-prefix">Am 21.11.23 um 14:07 schrieb Arsene
Marian Alain:<br>
</div>
<blockquote type="cite"
cite="mid:DB4PR03MB8609435B5E3532BA61AD398AF5BBA@DB4PR03MB8609.eurprd03.prod.outlook.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="Generator"
content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]-->
<style>@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}p.xmsonormal, li.xmsonormal, div.xmsonormal
{mso-style-name:x_msonormal;
margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}span.EstiloCorreo24
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;
mso-ligatures:none;}div.WordSection1
{page:WordSection1;}</style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="WordSection1">
<p class="MsoNormal"><span style="mso-fareast-language:EN-US">Thanks
Sean. I’ve tried using slurm prolog/epilog scripts but
without any success. That's why I decided to look for other
solutions and job_container/tmpfs plugin seemed like a good
alternative.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div
style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>De:</b> slurm-users
<a class="moz-txt-link-rfc2396E" href="mailto:slurm-users-bounces@lists.schedmd.com"><slurm-users-bounces@lists.schedmd.com></a>
<b>En nombre de </b>Sean Mc Grath<br>
<b>Enviado el:</b> martes, 21 de noviembre de 2023 12:57<br>
<b>Para:</b> Slurm User Community List
<a class="moz-txt-link-rfc2396E" href="mailto:slurm-users@lists.schedmd.com"><slurm-users@lists.schedmd.com></a><br>
<b>Asunto:</b> Re: [slurm-users] slurm job_container/tmpfs<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div
style="border:solid #9C6500 1.0pt;padding:2.0pt 2.0pt 2.0pt 2.0pt;tegn:xt-ali left">
<p class="MsoNormal"
style="line-height:12.0pt;background:#E3F2FD"><strong><span
style="font-size:10.0pt;font-family:"Calibri",sans-serif;color:black">ATENCIÓN</span></strong><span
style="font-size:10.0pt;color:#9C6500">:</span><span
style="font-size:10.0pt;color:black"> Este correo
electrónico se envió desde fuera de la UAH. No haga clic
en enlaces ni abra archivos adjuntos a menos que reconozca
al remitente y sepa que el contenido es seguro.<o:p></o:p></span></p>
</div>
<div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black">Would a prolog
script,
<a href="https://slurm.schedmd.com/prolog_epilog.html"
moz-do-not-send="true" class="moz-txt-link-freetext">https://slurm.schedmd.com/prolog_epilog.html</a>,
do what you need? Sorry if you have already considered
that and I missed it.</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><span
style="font-size:12.0pt;color:black"><o:p> </o:p></span></p>
</div>
<div id="Signature">
<div>
<p class="MsoNormal" style="background:white"><span
style="font-size:12.0pt;color:black">---</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span
style="font-size:12.0pt;color:black">Sean McGrath</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span
style="font-size:12.0pt;color:black">Senior Systems
Administrator, IT Services</span><o:p></o:p></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><o:p> </o:p></p>
</div>
</div>
<div class="MsoNormal" style="text-align:center"
align="center">
<hr width="98%" size="2" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span
style="color:black"> slurm-users <<a
href="mailto:slurm-users-bounces@lists.schedmd.com"
moz-do-not-send="true" class="moz-txt-link-freetext">slurm-users-bounces@lists.schedmd.com</a>>
on behalf of Arsene Marian Alain <<a
href="mailto:alain.arsene@uah.es"
moz-do-not-send="true" class="moz-txt-link-freetext">alain.arsene@uah.es</a>><br>
<b>Sent:</b> Tuesday 21 November 2023 09:58<br>
<b>To:</b> Slurm User Community List <<a
href="mailto:slurm-users@lists.schedmd.com"
moz-do-not-send="true" class="moz-txt-link-freetext">slurm-users@lists.schedmd.com</a>><br>
<b>Subject:</b> Re: [slurm-users] slurm
job_container/tmpfs</span> <o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="xmsonormal">Hello Brian,<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">Thanks for your answer. With the
job_container/tmpfs plugin I don't really create the
directory manually.<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">I just give my Basepath=/scratch (a
local directory for each node that is already mounted
with 1777 permissions) in job_container.conf. The plugin
automatically generates for each job a directory with
the "JOB_ID", for example: /scratch/1805<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">The only problem is that directory
1805 is generated with root owner and permissions 700.
So the user who submitted the job cannot write/read
inside directory 1805.<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">Is there a way for the owner of
directory 1805 to be the user who submitted the job and
not root?<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<div>
<div
style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="xmsonormal"><b>De:</b> slurm-users <<a
href="mailto:slurm-users-bounces@lists.schedmd.com"
moz-do-not-send="true"
class="moz-txt-link-freetext">slurm-users-bounces@lists.schedmd.com</a>>
<b>En nombre de </b>Brian Andrus<br>
<b>Enviado el:</b> lunes, 20 de noviembre de 2023
23:29<br>
<b>Para:</b> <a
href="mailto:slurm-users@lists.schedmd.com"
moz-do-not-send="true"
class="moz-txt-link-freetext">slurm-users@lists.schedmd.com</a><br>
<b>Asunto:</b> Re: [slurm-users] slurm
job_container/tmpfs<o:p></o:p></p>
</div>
</div>
<p class="xmsonormal"> <o:p></o:p></p>
<div
style="border:solid #9C6500 1.0pt;padding:2.0pt 2.0pt 2.0pt 2.0pt">
<p class="xmsonormal"
style="line-height:12.0pt;background:#E3F2FD"><strong><span
style="font-size:10.0pt;font-family:"Calibri",sans-serif;color:black">ATENCIÓN</span></strong><span
style="font-size:10.0pt;color:#9C6500">:</span><span
style="font-size:10.0pt;color:black"> Este correo
electrónico se envió desde fuera de la UAH. No haga
clic en enlaces ni abra archivos adjuntos a menos
que reconozca al remitente y sepa que el contenido
es seguro.</span><o:p></o:p></p>
</div>
<div>
<p>How do you 'manually create a directory'? That would
be when the ownership of root would be occurring.
After creating it, you can chown/chmod it as well.<o:p></o:p></p>
<p>Brian Andrus<o:p></o:p></p>
<div>
<p class="xmsonormal">On 11/18/2023 7:35 AM, Arsene
Marian Alain wrote:<o:p></o:p></p>
</div>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="xmsonormal">Dear slurm community,<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">I run slurm 21.08.1 under Rocky
Linux 8.5 on my small HPC cluster and am trying to
configure job_container/tmpfs to manage the
temporary directories.<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">I have a shared nfs drive
"/home" and a local "/scratch" (with permissions
1777) on each node.<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">For each submitted job I
manually create a directory with the "JOB_ID.$USER"
in the local "/scratch" which is where all the temp
files for the job will be generated. Now, I would
like to do these automatically (especially to remove
the directory when the job finishes or is canceled):<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">I added the following parameters
in my /etc/slurm.conf:<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">JobContainerType=job_container/tmpfs<o:p></o:p></p>
<p class="xmsonormal">PrologFlags=contain<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">So, I have created the
"job_container.conf" in the directory "/etc/slurm"<o:p></o:p></p>
<p class="xmsonormal">with the following
configuration:<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">AutoBasePath=false<o:p></o:p></p>
<p class="xmsonormal">BasePath=/scratch<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">Then, I replicated the changes
to all nodes and restarted the slurm daemons.<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">Finally, when I launch the job a
directory with the "JOB_ID" is created in the local
"/scratch" of the compute node. The only problem is
that the owner of the directory is "root" and the
user who submitted the job doesn’t have read and
write permissions to that directory (other users do
not either).<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">I would like that:<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">1) The name of the automatically
created directory will be: "JOB_ID.$USER"<o:p></o:p></p>
<p class="xmsonormal">2) The owner of the directory
will be the user who submitted the job, not "root".<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">Please, could someone help me?<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">Thanks a lot.<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">Best regards,<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">Alain<o:p></o:p></p>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</blockquote>
<pre class="moz-signature" cols="72">--
Dipl.-Chem. René Sitt
Hessisches Kompetenzzentrum für Hochleistungsrechnen
Philipps-Universität Marburg
Hans-Meerwein-Straße
35032 Marburg
Tel. +49 6421 28 23523
<a class="moz-txt-link-abbreviated" href="mailto:sittr@hrz.uni-marburg.de">sittr@hrz.uni-marburg.de</a>
<a class="moz-txt-link-abbreviated" href="http://www.hkhlr.de">www.hkhlr.de</a></pre>
</body>
</html>