[slurm-users] How can jobs request a minimum available (free) TmpFS disk space?
b.h.mevik at usit.uio.no
Tue Sep 3 10:36:24 UTC 2019
Juergen Salk <juergen.salk at uni-ulm.de> writes:
> We are also going to implement disk quotas for the amount of local
> scratch space that has been allocated for the job by means of generic
> resources (e.g. `--gres=scratch:100´ for 100GB). This is especially
> important when several users share a node.
> This leads me to ask how you plan to determine the amount of local
> scratch space allocated for the job from within its prolog and epilog
> I already thought about running `scontrol show job $SLURM_JOB_ID´ from
> within the prolog/epilog scripts in order to get that piece of
This is exactly what we do. :)
> This line could eventually be parsed to get the amount of scratch
> allocated for this job (and then further used to increase/decrease the
> quota limits for the corresponding $SLURM_JOB_USER in the
> prolog/epilog scripts).
If you use separate directories for each job, and use "project" quotas
(a.k.a folder quotas), then you don't have to adjust the quota when a
new job arrives, even if it is from the same user.
> However, this still looks kind of clumsily to me and I wonder, if I
> have just overlooked a more obvious, cleaner or more robust solution.
Nope. (We could have done something more elegant like writing a
small Perl utility that extracted just the needed parts, but never got
around to it.)
I _think_ another option would be to write a SPANK plugin for the gres,
and let that create/remove the scratch directory and set the quota, but
I haven't looked into that. That would probably count as a more elegant
> Since this is probably not an unusual requirement, I suppose this is
> something that many other sites have already solved for
> themselves. No?
Yes, please, let us know how you've solved this!
Bjørn-Helge Mevik, dr. scient,
Department for Research Computing, University of Oslo
-------------- next part --------------
A non-text attachment was scrubbed...
Size: 832 bytes
Desc: not available
More information about the slurm-users