[slurm-users] $TMPDIR does not honor "TmpFS"

[Michael Gutteridge]

I don't think that's a bug.  As far as I've ever known, TmpFS is only used
to tell slurmd where to look for available space (reported as TmpDisk for
the node).  The manpage only indicates that, not any additional
functionality.  We set TMPDIR in a task prolog:

#!/bin/bash
echo "export TMPDIR=/loc/scratch/${SLURM_JOB_ID}"
echo "export SCRATCH_LOCAL=/loc/scratch/${SLURM_JOB_ID}"
echo "export SCRATCH=/net/scratch/${SLURM_JOB_ID}"

- Michael


On Wed, Nov 21, 2018 at 6:52 AM Shenglong Wang <sw77 at nyu.edu> wrote:

> We have TMPDIR setup inside prolog file. Hope users do not have absolute
> path /tmp inside their scripts.
>
> #!/bin/bash
>
> SLURM_BIN="/opt/slurm/bin"
>
> SLURM_job_tmp=/state/partition1/job-${SLURM_JOB_ID}
>
> mkdir -m 700 -p $SLURM_job_tmp
> chown $SLURM_JOB_USER $SLURM_job_tmp
>
> echo "export SLURM_JOBTMP=$SLURM_job_tmp"
> echo "export SLURM_JOB_TMP=$SLURM_job_tmp"
> echo "export SLURM_JOB_TMPDIR=$SLURM_job_tmp"
> echo "export TMPDIR=$SLURM_job_tmp”
>
> Best.
> Shenglong
>
> On Nov 21, 2018, at 9:44 AM, Roger Moye <rmoye at quantlab.com> wrote:
>
> We are having the exact same problem with $TMPDIR.   I wonder if a bug has
> crept in?    I spoke to the SchedMD guys at SC18 last week and they were
> not aware of a bug but since more than one person is having this difficulty
> something must be wrong somewhere.
>
> -Roger
>
> *From:* slurm-users [mailto:slurm-users-bounces at lists.schedmd.com
> <slurm-users-bounces at lists.schedmd.com>] *On Behalf Of *Douglas Duckworth
> *Sent:* Wednesday, November 21, 2018 7:38 AM
> *To:* slurm-users at lists.schedmd.com
> *Subject:* [slurm-users] $TMPDIR does not honor "TmpFS"
>
> Hi
>
> We are setting TmpFS=/scratchLocal in /etc/slurm/slurm.conf on nodes and
> controller.  However $TMPDIR value seems to be /tmp not /scratchLocal.  As
> a result users are writing to /tmp which we do not want.
>
> We are not setting $TMPDIR anywhere else such as /etc/profile.d nor do
> users have it defined in their ~/.bashrc or ~/.bash_profile.
>
> We do not see any error messages anywhere which could indicate why the
> default value of /tmp overrides our value of of TmpFS.
>
> As I understand prolog scripts can change this value though, if that's the
> case, then what's the purpose of setting TmpFS in /etc/slurm/slurm.conf?
>
>
> Thanks,
>
> Douglas Duckworth, MSc, LFCS
> HPC System Administrator
> Scientific Computing Unit
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__scu.med.cornell.edu&d=DwMGaQ&c=slrrB7dE8n7gBJbeO0g-IQ&r=gI7YPDDMIksUN29Y65_UCg&m=VfqOl-yXqx9492N0dCQZ7TafygrL8Ev0zem-MEqEYEY&s=H8jTI5dijpEftNyF3D1oZZOBrn4sRgY0wXSxgqUgD2U&e=>
> Weill Cornell Medicine
> 1300 York Avenue
> New York, NY 10065
> E: doug at med.cornell.edu
> O: 212-746-6305
> F: 212-746-8690
>
>
> -----------------------------------------------------------------------------------
>
> The information in this communication and any attachment is confidential
> and intended solely for the attention and use of the named addressee(s).
> All information and opinions expressed herein are subject to change without
> notice. This communication is not to be construed as an offer to sell or
> the solicitation of an offer to buy any security. Any such offer or
> solicitation can only be made by means of the delivery of a confidential
> private offering memorandum (which should be carefully reviewed for a
> complete description of investment strategies and risks). Any reliance one
> may place on the accuracy or validity of this information is at their own
> risk. Past performance is not necessarily indicative of the future results
> of an investment. All figures are estimated and unaudited unless otherwise
> noted. If you are not the intended recipient, or a person responsible for
> delivering this to the intended recipient, you are not authorized to and
> must not disclose, copy, distribute, or retain this message or any part of
> it. In this case, please notify the sender immediately at 713-333-5440
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20181121/b1db520b/attachment.html>