[slurm-users] [External] Re: Per-user TRES summary?

[Ole Holm Nielsen]

Hi Mike,

That sounds great!  It seems to me that "showuserlimits -q <qos>" would 
also print the QOS information, but maybe this is not what you are 
after?  Have you tried this -q option, or should the script perhaps be 
generalized to cover your needs?

/Ole


On 29-11-2022 14:39, Pacey, Mike wrote:
> Hi Ole (and Jeffrey),
> 
> Thanks for the pointer - those are some very useful scripts. I couldn't get showslurmlimits or showslurmjobs to get quite what I was after (it wasn't showing me memory usage). However, it pointed me in the right direction - the scontrol command. I can run the following:
> 
> scontrol show assoc_mgr flags=qos
> 
> and part of the output reads:
> 
>      User Limits
>        [myuid]
> 	MaxJobsPU=N(2) MaxJobsAccruePU=N(0) MaxSubmitJobsPU=N(2)
> 	MaxTRESPU=cpu=80(2),mem=327680(1000),energy=N(0),node=N(1),billing=N(2),fs/disk=N(0),vmem=N(0),pages=N(0)
> 
> Which is exactly what I'm looking for. The values outside the brackets are the qos limit, and the values within are the current usage.
> 
> Regards,
> Mike
> 
> -----Original Message-----
> From: slurm-users <slurm-users-bounces at lists.schedmd.com> On Behalf Of Ole Holm Nielsen
> Sent: 28 November 2022 18:58
> To: slurm-users at lists.schedmd.com
> Subject: [External] Re: [slurm-users] Per-user TRES summary?
> 
> This email originated outside the University. Check before clicking links or attachments.
> 
> Hi Mike,
> 
> Would the "showuserlimits" tool give you the desired information?  Check out https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FOleHolmNielsen%2FSlurm_tools%2Ftree%2Fmaster%2Fshowuserlimits&data=05%7C01%7Cpacey%40live.lancs.ac.uk%7Cbea74c16c0b34468c68908dad174dacc%7C9c9bcd11977a4e9ca9a0bc734090164a%7C0%7C0%7C638052597059366026%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=9pKyND95SW9Z1E%2BjGPsGKUKwTZIqj3juwWGQ4d5AWRw%3D&reserved=0
> 
> /Ole
> 
> 
> On 28-11-2022 16:16, Pacey, Mike wrote:
>> Does anyone have suggestions as to how to produce a summary of a
>> user's TRES resources for running jobs? I'd like to able to see how
>> each user is fairing against their qos resource limits. (I'm looking
>> for something functionally equivalent to Grid Engine's qquota
>> command). The info must be in the scheduler somewhere in order for it
>> to enforce qos TRES limits, but as a SLURM novice I've not found any way to do this.
>>
>> To summarise TRES qos limits I can do this:
>>
>> % sacctmgr list qos format=Name,MaxTRESPerUser%50
>>
>>         Name                                          MaxTRESPU
>>
>> ---------- --------------------------------------------------
>>
>>       normal                                    cpu=80,mem=320G
>>
>> But to work out what a user is currently using in currently running
>> jobs, the nearest I can work out is:
>>
>> % sacct -X -s R --units=G -o User,ReqTRES%50
>>
>>        User                                            ReqTRES
>>
>> --------- --------------------------------------------------
>>
>>       pacey                   billing=1,cpu=1,mem=0.49G,node=1
>>
>>       pacey                   billing=1,cpu=1,mem=0.49G,node=1
>>
>> With a little scripting I can sum those up, but there might be a
>> neater way to do this?
>