[slurm-users] Feature request: create a job id before job submission

[Reuti]

> Am 07.05.2019 um 16:00 schrieb Mark Hahn <hahn at mcmaster.ca>:
> 
>> Some cluster sites need the creation of a workspace for the job in a
>> scratch area before the actual job submission, and on the other hand they
>> don't accept all characters as name of the workspace. Hence the plain job
>> name often can't be used here.
> 
> puzzled two ways: why not use the numeric jobid,

I don't know the job id before the actual job submission. Hence I would like to get some kind of place holder, and `scommit` the job later with the actual resource requirements as comments in an usual jobscript.


> and why would configuring
> the scratch space be too slow to perform in the job prolog?

The access to /home is highly discouraged from the nodes, instead the users should prepare an area in /scratch beforehand (copy all the files for the job thereto) and submit the job from there. So the working directory of the job is automatically in the /scratch area (fast parallel file system) – no further file staging needed. Essentially the nodes could work without a mounted /home.

Putting it in the prolog would at least mean for the admins to prepare some kind of file staging too, as all users's files need to be transferred thereto too (short access to /home) and the working directory adjusted. With many different types of jobs this might not be easy.

Sure, `sblank` which would provide a reserved job id could have some prolog and prepare the workspace to tell the user: please put your files in /scratch/job-id-task-id. For the users this would mean to issue:

sblank
copy files to the given location(s) fro  the login node
scommit

and the users can be sure to find the job's files in /scratch/job-id-task-id, and the admins can be sure that there is no access to /home slowing down the cluster and interactive work on the login node.

-- Reuti