[slurm-users] Re: GPU usage in fair share factor

Indeed I guess the right syntax is:

TRESBillingWeights=CPU=1.0,Mem=0.25G,gres/gpu:nvidia-h100=20.0,gres/gpu:nvidia-l40s=10.0

Thanks again !

Regards, Massimo

On Fri, Dec 12, 2025 at 4:10 PM Paul Edmon pedmon@cfa.harvard.edu wrote:

I will admit I have never tried to bill on a mixed way like this for gpu. I have you tried doing gres/gpu:nvidia-l40s=10.0 I think the extra = is throwing things off.

-Paul Edmon- On 12/12/25 9:25 AM, Massimo Sgaravatto wrote:

Thanks a lot, but I am unable to set different weights for different partitions. I tried with:

PartitionName=DEFAULT TRESBillingWeights="CPU=1.0,Mem=0.25G,gres/gpu=nvidia-h100=20.0,gres/gpu=nvidia-l40s=10.0"

but:

[2025-12-12T14:59:14.741] error: Invalid unit type 'n'. Possible options are 'KMGTP' [2025-12-12T14:59:14.741] fatal: failed to parse tres weights str 'CPU=1.0,Mem=0.25G,gres/gpu=nvidia-h100=20.0,gres/gpu=nvidia-l40s=10.0'

while:

PartitionName=DEFAULT TRESBillingWeights="CPU=1.0,Mem=0.25G,gres/gpu=20"

works (or at least it is accepted)

Thanks, Massimo

On Wed, Dec 10, 2025 at 3:40 PM Paul Edmon via slurm-users < slurm-users@lists.schedmd.com> wrote:

...

That is correct. You need to set the TRESBillingWeight to get billing for gpus.

-Paul Edmon-

On 12/10/2025 9:04 AM, Massimo Sgaravatto via slurm-users wrote:

...

Dear all

I have these settings in my slurm.conf:

PriorityType=priority/multifactor PriorityDecayHalfLife=7-0 PriorityFavorSmall=NO PriorityMaxAge=10-0 PriorityWeightAge=100000 PriorityWeightFairshare=1000000 PriorityWeightJobSize=100000 PriorityWeightPartition=100000 PriorityWeightQOS=100000 PriorityFlags=ACCRUE_ALWAYS,FAIR_TREE AccountingStorageTRES=gres/gpu,gres/gpu:nvidia-h100,gres/gpu:nvidia-l40s

If I have got it right, with these settings, when calculating the Job_priority (I am referring to the formula in https://slurm.schedmd.com/priority_multifactor.html) the fair-share_factor is calculated only considering cores*seconds. So if a job also used some GPUs, this is not taken into account. Am I right ?

If I want to take memory and GPUs into account as well (with different weights for different GPU models), my understanding is that I should use the TRESBillingWeights attribute, setting e.g:

TRESBillingWeights="CPU=1.0,Mem=0.25G,gres/gpu=nvidia-h100=10.0,gres/gpu=nvidia-l40s=5.0"

...

Is this correct ?

Thanks, Massimo

-- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-leave@lists.schedmd.com