[slurm-users] slurm job_container/tmpfs

Tue Nov 21 13:35:22 UTC 2023

Hello Alain,

as an alternative to job_container/tmpfs, you may also try your luck 
with the 'auto_tmpdir' SPANK plugin: 
https://github.com/University-of-Delaware-IT-RCI/auto_tmpdir

We've been using using that on our small HPC cluster (Slurm 22.05) and 
it does what it's supposed to. One thing one has to remember is that it 
requires a recompile after every Slurm update.

Kind regards,
René Sitt

Am 21.11.23 um 14:07 schrieb Arsene Marian Alain:
>
> Thanks Sean. I’ve tried using slurm prolog/epilog scripts but without 
> any success. That's why I decided to look for other solutions and 
> job_container/tmpfs plugin seemed like a good alternative.
>
> *De:* slurm-users <slurm-users-bounces at lists.schedmd.com> *En nombre 
> de *Sean Mc Grath
> *Enviado el:* martes, 21 de noviembre de 2023 12:57
> *Para:* Slurm User Community List <slurm-users at lists.schedmd.com>
> *Asunto:* Re: [slurm-users] slurm job_container/tmpfs
>
> *ATENCIÓN*:Este correo electrónico se envió desde fuera de la UAH. No 
> haga clic en enlaces ni abra archivos adjuntos a menos que reconozca 
> al remitente y sepa que el contenido es seguro.
>
> Would a prolog script, https://slurm.schedmd.com/prolog_epilog.html, 
> do what you need? Sorry if you have already considered that and I 
> missed it.
>
> ---
>
> Sean McGrath
>
> Senior Systems Administrator, IT Services
>
> ------------------------------------------------------------------------
>
> *From:*slurm-users <slurm-users-bounces at lists.schedmd.com> on behalf 
> of Arsene Marian Alain <alain.arsene at uah.es>
> *Sent:* Tuesday 21 November 2023 09:58
> *To:* Slurm User Community List <slurm-users at lists.schedmd.com>
> *Subject:* Re: [slurm-users] slurm job_container/tmpfs
>
> Hello Brian,
>
> Thanks for your answer. With the job_container/tmpfs plugin I don't 
> really create the directory manually.
>
> I just give my Basepath=/scratch (a local directory for each node that 
> is already mounted with 1777 permissions) in job_container.conf. The 
> plugin automatically generates for each job a directory with the 
> "JOB_ID", for example: /scratch/1805
>
> The only problem is that directory 1805 is generated with root owner 
> and permissions 700. So the user who submitted the job cannot 
> write/read inside directory 1805.
>
> Is there a way for the owner of directory 1805 to be the user who 
> submitted the job and not root?
>
> *De:* slurm-users <slurm-users-bounces at lists.schedmd.com> *En nombre 
> de *Brian Andrus
> *Enviado el:* lunes, 20 de noviembre de 2023 23:29
> *Para:* slurm-users at lists.schedmd.com
> *Asunto:* Re: [slurm-users] slurm job_container/tmpfs
>
> *ATENCIÓN*:Este correo electrónico se envió desde fuera de la UAH. No 
> haga clic en enlaces ni abra archivos adjuntos a menos que reconozca 
> al remitente y sepa que el contenido es seguro.
>
> How do you 'manually create a directory'? That would be when the 
> ownership of root would be occurring. After creating it, you can 
> chown/chmod it as well.
>
> Brian Andrus
>
> On 11/18/2023 7:35 AM, Arsene Marian Alain wrote:
>
>     Dear slurm community,
>
>     I run slurm 21.08.1 under Rocky Linux 8.5 on my small HPC cluster
>     and am trying to configure job_container/tmpfs to manage the
>     temporary directories.
>
>     I have a shared nfs drive "/home" and a local "/scratch" (with
>     permissions 1777) on each node.
>
>     For each submitted job I manually create a directory with the
>     "JOB_ID.$USER" in the local "/scratch" which is where all the temp
>     files for the job will be generated. Now, I would like to do these
>     automatically (especially to remove the directory when the job
>     finishes or is canceled):
>
>     I added the following parameters in my /etc/slurm.conf:
>
>     JobContainerType=job_container/tmpfs
>
>     PrologFlags=contain
>
>     So, I have created the "job_container.conf" in the directory
>     "/etc/slurm"
>
>     with the following configuration:
>
>     AutoBasePath=false
>
>     BasePath=/scratch
>
>     Then, I replicated the changes to all nodes and restarted the
>     slurm daemons.
>
>     Finally, when I launch the job a directory with the "JOB_ID" is
>     created in the local "/scratch" of the compute node. The only
>     problem is that the owner of the directory is "root" and the user
>     who submitted the job doesn’t have read and write permissions to
>     that directory (other users do not either).
>
>     I would like that:
>
>     1) The name of the automatically created directory will be:
>     "JOB_ID.$USER"
>
>     2) The owner of the directory will be the user who submitted the
>     job, not "root".
>
>     Please, could someone help me?
>
>     Thanks a lot.
>
>     Best regards,
>
>     Alain
>
-- 
Dipl.-Chem. René Sitt
Hessisches Kompetenzzentrum für Hochleistungsrechnen
Philipps-Universität Marburg
Hans-Meerwein-Straße
35032 Marburg

Tel. +49 6421 28 23523
sittr at hrz.uni-marburg.de
www.hkhlr.de
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20231121/2ccc1821/attachment.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4239 bytes
Desc: Kryptografische S/MIME-Signatur
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20231121/2ccc1821/attachment.bin>