[slurm-users] Getting --gpus -request in job_submit.lua

[Christopher Benjamin Coffey]

Hi Niels,

Have you found a solution? I just noticed this recently as well. We've traditionally told our users to use --gres:gpu:tesla:# for requesting gpus. Then, our job submit plugin would detect the gres ask, specifically gpu, and set a a qos, and partition accordingly. Unforutnately I started pushing folks to use -G1, or --gpus=1 for simplicity and just realized our plugin does not pick up gpu stuff anymore. Looking at the docs here:

https://slurm.schedmd.com/job_submit_plugins.html

The lua portion says that the function: " _get_job_req_field()" should highlight the attributes available. Yet, the gpu request specifics don't appear to be there in the code:

https://github.com/SchedMD/slurm/blob/master/src/plugins/job_submit/lua/job_submit_lua.c

Here is hoping slurm devs can add them, or point to the correct attributes to use. I did try "gpus_per_task" but that didn't work.

Best,
Chris
 
-- 
Christopher Coffey
High-Performance Computing
Northern Arizona University
928-523-1167
 
 

On 9/30/20, 6:13 AM, "slurm-users on behalf of Niels Carl Hansen" <slurm-users-bounces at lists.schedmd.com on behalf of ncwh at cscaa.dk> wrote:

    I am trying to retrieve the number of requested GPUs in job_submit.lua

    If the job is submitted with a --gres -flag, as in "sbatch 
    --gres=gpu:2...", I can get the
    information in job_submit.lua via the variable 'job_desc.tres_per_node'.

    But if the job is submitted with the --gpus -flag, as in "sbatch 
    --gpus=2", then 'job_desc.tres_per_node'
    is nil.

    How can I dig out the number of requested GPUs in job_submit.lua in the 
    latter case?
    I am running Slurm 20.02.5.

    Thanks in advance.

    Niels Carl Hansen
    Aarhus University, Denmark