[slurm-users] Getting --gpus -request in job_submit.lua

[Niels Carl Hansen]

Hi Chris

Yes, we're using constructions like this in our job_submit.lua:

--    For --gres=gpu:N requests
         if job_desc.tres_per_node ~= nil then
           slurm.user_msg("Tres_per_node   => " .. job_desc.tres_per_node)
         else
           slurm.user_msg("Tres_per_node   => is nil")
         end

--    For --gpus=N requests
         if job_desc.tres_per_job ~= nil then
           slurm.user_msg("Tres_per_job    => " .. job_desc.tres_per_job)
         else
           slurm.user_msg("Tres_per_job    => is nil")
         end


Best
Niels Carl Hansen
University of Aarhus
Denmark


On 25/11/2020 18.35, Christopher Benjamin Coffey wrote:
> Hi Niels,
>
> Have you found a solution? I just noticed this recently as well. We've traditionally told our users to use --gres:gpu:tesla:# for requesting gpus. Then, our job submit plugin would detect the gres ask, specifically gpu, and set a a qos, and partition accordingly. Unforutnately I started pushing folks to use -G1, or --gpus=1 for simplicity and just realized our plugin does not pick up gpu stuff anymore. Looking at the docs here:
>
> https://slurm.schedmd.com/job_submit_plugins.html
>
> The lua portion says that the function: " _get_job_req_field()" should highlight the attributes available. Yet, the gpu request specifics don't appear to be there in the code:
>
> https://github.com/SchedMD/slurm/blob/master/src/plugins/job_submit/lua/job_submit_lua.c
>
> Here is hoping slurm devs can add them, or point to the correct attributes to use. I did try "gpus_per_task" but that didn't work.
>
> Best,
> Chris
>