[slurm-users] NoDecay on accounts (or on GrpTRESMins in general)

[Sebastian T Smith]

Hi,

We're setting GrpTRESMins on the account association and have NoDecay QOS's for different user classes.  All user associations with a GrpTRESMins-limited account are assigned a NoDecay QOS.  I'm not sure if it's a better approach... but it's an option.

Our GrpTRESMins limits are applied to account hierarchies (in some cases).  For example, we have a student research service that allows groups of students to gain access to HPC resources.  This service is backed by a central fund, and we want each student group to have equal access to the system (student_GrpTRESMins ~= central_budget / num_student_groups) while not exceeding the central budget.  We create a parent account and set its GrpTRESMins equivalent to the central fund budget, and child accounts with their own GrpTRESMins below that for each student group.  This will shut down student workloads if they exceed their group budget, and the aggregated student workload if the central budget would be exceeded.

I'm also interested in what others have come up with.  There's a lot of config permutations...

Thanks,

Sebastian

--

[University of Nevada, Reno]<http://www.unr.edu/>
Sebastian Smith
High-Performance Computing Engineer
Office of Information Technology
1664 North Virginia Street
MS 0291

work-phone: 775-682-5050<tel:7756825050>
email: stsmith at unr.edu<mailto:stsmith at unr.edu>
website: http://rc.unr.edu<http://rc.unr.edu/>

________________________________
From: slurm-users <slurm-users-bounces at lists.schedmd.com> on behalf of Yair Yarom <irush at cs.huji.ac.il>
Sent: Tuesday, November 17, 2020 6:42 AM
To: Slurm User Community List <slurm-users at lists.schedmd.com>
Subject: [slurm-users] NoDecay on accounts (or on GrpTRESMins in general)

Hi all,

We have around 50 accounts, each has its own GrpTRES limits. We want to add another set of accounts (probably another 50) with different priority which will have GrpTRESMins, such that users could "buy" TRES*minutes with higher priority.

For that we require that the GrpTRESMins won't get decayed. We do want the normal multifactor priority mechanism to work with decaying usage, and we don't want to reset the usage of GrpTRESMins periodically.

Currently the only solution I found is to create a new QOS with NoDecay for each such new account. As we also have multiple clusters on the same database, this also requires a new QOS for each account for each cluster (as QOS appears to be shared among clusters).

Is there a downside of adding many QOS? (besides the management headache).
Has anyone else done something similar and have some insights? or another solution?

Thanks in advance,
    Yair.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20201119/64b60000/attachment.htm>