[slurm-users] prolog not passing env var to job

[Brian Andrus]

I think it isn't running how you think or there is something not 
provided in the description.


You have:

         export
    TMP="/local/scratch/${SLURM_ARRAY_JOB_ID}.${SLURM_ARRAY_TASK_ID}"


Notice that period in there.
Then you have:

    node001::~$ echo $TMP
    /local/scratch/80472

There is no period.
In fact, SLURM_ARRAY_JOB_ID should be blank too if you are not running 
as an array session.

However, to your desire for a job-specific tmp directory:
Check out the mktemp command. It should do just what you want. I use it 
for interactive desktop sessions for users to create the temp directory 
that is used for X sessions.
You just need to make sure the user has write access to the directory 
you are creating the directory in (chmod 1777 for the parent directory 
is good)

Brian Andrus

On 3/4/2021 9:03 AM, Chin,David wrote:
> Hi, Brian:
>
> So, this is my SrunProlog script -- I want a job-specific tmp dir, 
> which makes for easy cleanup at end of job:
>
>     #!/bin/bash
>     if [[ -z ${SLURM_ARRAY_JOB_ID+x} ]]
>     then
>         export TMP="/local/scratch/${SLURM_JOB_ID}"
>         export TMPDIR="${TMP}"
>         export LOCAL_TMPDIR="${TMP}"
>         export BEEGFS_TMPDIR="/beegfs/scratch/${SLURM_JOB_ID}"
>     else
>         export
>     TMP="/local/scratch/${SLURM_ARRAY_JOB_ID}.${SLURM_ARRAY_TASK_ID}"
>         export TMPDIR="${TMP}"
>         export LOCAL_TMPDIR="${TMP}"
>         export
>     BEEGFS_TMPDIR="/beegfs/scratch/${SLURM_ARRAY_JOB_ID}.${SLURM_ARRAY_TASK_ID}"
>     fi
>
>     echo DEBUG srun_set_tmp.sh
>     echo I am `whoami`
>
>     /usr/bin/mkdir -p ${TMP}
>     chmod 700 ${TMP}
>     /usr/bin/mkdir -p ${BEEGFS_TMPDIR}
>     chmod 700 ${BEEGFS_TMPDIR}
>
>
> And this is my srun session:
>
>     picotte001::~$ whoami
>     dwc62
>     picotte001::~$ srun -p def --mem 1000 -n 4 -t 600 --pty /bin/bash
>     DEBUG srun_set_tmp.sh
>     I am dwc62
>     node001::~$ echo $TMP
>     /local/scratch/80472
>     node001::~$ ll !$
>     ll $TMP
>     /bin/ls: cannot access '/local/scratch/80472': No such file or
>     directory
>     node001::~$ mkdir $TMP
>     node001::~$ ll -d !$
>     ll -d $TMP
>     drwxrwxr-x 2 dwc62 dwc62 6 Mar  4 11:52 /local/scratch/80472/
>     node001::~$ exit
>
>
> So, the "echo" and "whoami" statements are executed by the prolog 
> script, as expected, but the mkdir commands are not?
>
> Thanks,
>     Dave
>
> --
> David Chin, PhD (he/him)   Sr. SysAdmin, URCF, Drexel
> dwc62 at drexel.edu  215.571.4335 (o)
> For URCF support: urcf-support at drexel.edu
> https://proteusmaster.urcf.drexel.edu/urcfwiki
> github:prehensilecode
>
> ------------------------------------------------------------------------
> *From:* slurm-users <slurm-users-bounces at lists.schedmd.com> on behalf 
> of Brian Andrus <toomuchit at gmail.com>
> *Sent:* Thursday, March 4, 2021 10:12
> *To:* slurm-users at lists.schedmd.com <slurm-users at lists.schedmd.com>
> *Subject:* Re: [slurm-users] prolog not passing env var to job
>
> External.
>
>
> It seems to me, if you are using srun directly to get an interactive 
> shell, you can just run the script once you get your shell.
>
>
> You can set the variables and then run srun. It automatically exports 
> the environment.
>
> If you want to change a particular one (or more), use something like 
> --export=ALL,MYVAR=othervalue
>
> do 'man srun' and look at the --export option
>
>
> Brian Andrus
>
>
>
>
> On 3/3/2021 9:28 PM, Chin,David wrote:
>> ahmet.mercan at uhem.itu.edu.tr <mailto:ahmet.mercan at uhem.itu.edu.tr> wrote:
>> > Prolog and TaskProlog are different parameters and scripts. You should
>> > use the TaskProlog script to set env. variables.
>>
>> Can you tell me how to do this for srun? E.g. users request an 
>> interactive shell:
>>
>> srun -n 1 -t 600 --pty /bin/bash
>>
>> but the shell on the compute node does not have the env variables set.
>>
>> I use the same prolog script as TaskProlog, which sets it properly 
>> for jobs submitted
>> with sbatch.
>>
>> Thanks in advance,
>> Dave Chin
>>
>> --
>> David Chin, PhD (he/him)   Sr. SysAdmin, URCF, Drexel
>> dwc62 at drexel.edu <mailto:dwc62 at drexel.edu>                   
>>  215.571.4335 (o)
>> For URCF support: urcf-support at drexel.edu 
>> <mailto:urcf-support at drexel.edu>
>> https://proteusmaster.urcf.drexel.edu/urcfwiki 
>> <https://proteusmaster.urcf.drexel.edu/urcfwiki>
>> github:prehensilecode
>>
>>
>> ------------------------------------------------------------------------
>> *From:* slurm-users <slurm-users-bounces at lists.schedmd.com> 
>> <mailto:slurm-users-bounces at lists.schedmd.com> on behalf of mercan 
>> <ahmet.mercan at uhem.itu.edu.tr> <mailto:ahmet.mercan at uhem.itu.edu.tr>
>> *Sent:* Friday, February 12, 2021 16:27
>> *To:* Slurm User Community List <slurm-users at lists.schedmd.com> 
>> <mailto:slurm-users at lists.schedmd.com>; Herc Silverstein 
>> <herc.silverstein at schrodinger.com> 
>> <mailto:herc.silverstein at schrodinger.com>; slurm-users at schedmd.com 
>> <mailto:slurm-users at schedmd.com> <slurm-users at schedmd.com> 
>> <mailto:slurm-users at schedmd.com>
>> *Subject:* Re: [slurm-users] prolog not passing env var to job
>> External.
>>
>> Hi;
>>
>> Prolog and TaskProlog are different parameters and scripts. You should
>> use the TaskProlog script to set env. variables.
>>
>> Regards;
>>
>> Ahmet M.
>>
>>
>>
>> Drexel Internal Data
>>
>
> Drexel Internal Data
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20210304/bfd63916/attachment-0001.htm>