<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;">
Once you added enough to ingest the Slurm logs into Influx or whatever, it could be similar. XDMoD already has the pieces in place to dig through your hierarchy of PIs, users, etc. Plus some built-in queries for correlating job size to wait time, for example:
<div class=""><br class="">
</div>
<div class=""><img apple-inline="yes" id="3A7BD06E-26E7-49D9-ACDC-40B86B1812DA" src="cid:0F1CF9CC-D46B-4464-A386-5C5BF11B59D9@tntech.edu" class=""></div>
<div class=""><br class="">
</div>
<div class="">I’ve also started using XDMoD as my data source for some short one-slide presentations, where I extract out a graph of the historical resource usage and overlay our total job count and total CPU-hours used.</div>
<div class="">
<div class=""><br class="">
</div>
<blockquote type="cite" class="">On Nov 26, 2019, at 10:21 AM, Ricardo Gregorio <<a href="mailto:ricardo.gregorio@rothamsted.ac.uk" class="">ricardo.gregorio@rothamsted.ac.uk</a>> wrote:<br class="">
<br class="">
External Email Warning<br class="">
<br class="">
This email originated from outside the university. Please use caution when opening attachments, clicking links, or responding to requests.<br class="">
<br class="">
________________________________<br class="">
<br class="">
Mike,<br class="">
<br class="">
It sounds interesting...In fact I had come across XDMoD this morning while "searching" for further info...<br class="">
<br class="">
Would Grafana do similar job as XDMoD?<br class="">
<br class="">
<br class="">
<br class="">
-----Original Message-----<br class="">
From: slurm-users <<a href="mailto:slurm-users-bounces@lists.schedmd.com" class="">slurm-users-bounces@lists.schedmd.com</a>> On Behalf Of Renfro, Michael<br class="">
Sent: 26 November 2019 16:14<br class="">
To: Slurm User Community List <<a href="mailto:slurm-users@lists.schedmd.com" class="">slurm-users@lists.schedmd.com</a>><br class="">
Subject: Re: [slurm-users] slurm reporting<br class="">
<br class="">
<blockquote type="cite" class="">• Total number of jobs submitted by user (daily/weekly/monthly)<br class="">
• Average queue time per user (daily/weekly/monthly)<br class="">
• Average job run time per user (daily/weekly/monthly)<br class="">
</blockquote>
<br class="">
Open XDMoD for these three. <a href="https://eur01.safelinks.protection.outlook.com/?" class="">
https://eur01.safelinks.protection.outlook.com/?</a>url=https%3A%2F%<a href="http://2Fgithub.com" class="">2Fgithub.com</a>%2Fubccr%2Fxdmod&data=01%7C01%7Cricardo.gregorio%<a href="http://40rothamsted.ac.uk" class="">40rothamsted.ac.uk</a>%7C460de352693741c7399508d7728bfb68%7Cb688362589414342b0e37b8cc8392f64%7C1&sdata=ePMpRET56c241GOCIU%2Bt3qMkR1vDUeFHv9DLKNb0cVo%3D&reserved=0
, plus <a href="https://eur01.safelinks.protection.outlook.com/?" class="">https://eur01.safelinks.protection.outlook.com/?</a>url=https%3A%2F%<a href="http://2Fxdmod.ccr.buffalo.edu" class="">2Fxdmod.ccr.buffalo.edu</a>&data=01%7C01%7Cricardo.gregorio%<a href="http://40rothamsted.ac.uk" class="">40rothamsted.ac.uk</a>%7C460de352693741c7399508d7728bfb68%7Cb688362589414342b0e37b8cc8392f64%7C1&sdata=DkFnQBRkfAkzpIb6naqsPWXiVvBoRpC1zNr8CRsRpRA%3D&reserved=0
(unfortunately their SSL certificate expired yesterday, so you’ll get a warning).<br class="">
<br class="">
<blockquote type="cite" class="">• %time partitions were in-use and idle<br class="">
</blockquote>
<br class="">
Not sure how you’d want to define this, plus our partitions have substantial overlap on resources (our partitions are primarily to separate GPU or large memory jobs from others, and to balance priorities and limits on different classes of jobs).<br class="">
<br class="">
<blockquote type="cite" class="">• min/mx/avg number of nodes/cpus/mem used per user/job<br class="">
</blockquote>
<br class="">
Open XDMoD for CPUs and nodes, and probably Open XDMoD plus SUPREMM for memory (haven’t used this one myself, but I plan to).<br class="">
<br class="">
--<br class="">
Mike Renfro, PhD / HPC Systems Administrator, Information Technology Services<br class="">
931 372-3601 / Tennessee Tech University<br class="">
<br class="">
<blockquote type="cite" class="">On Nov 26, 2019, at 10:02 AM, Ricardo Gregorio <<a href="mailto:ricardo.gregorio@rothamsted.ac.uk" class="">ricardo.gregorio@rothamsted.ac.uk</a>> wrote:<br class="">
<br class="">
External Email Warning<br class="">
This email originated from outside the university. Please use caution when opening attachments, clicking links, or responding to requests.<br class="">
Hi all,<br class="">
<br class="">
I am new to both HPC and SLURM.<br class="">
<br class="">
I have been trying to run some usage reports (using sreport and sacct); but I cannot find a way to get the following info:<br class="">
<br class="">
• Total number of jobs submitted by user (daily/weekly/monthly)<br class="">
• Average queue time per user (daily/weekly/monthly)<br class="">
• Average job run time per user (daily/weekly/monthly)<br class="">
• %time partitions were in-use and idle<br class="">
• min/mx/avg number of nodes/cpus/mem used per user/job<br class="">
<br class="">
Is this doable?<br class="">
<br class="">
Regards,<br class="">
Ricardo Gregorio<br class="">
Research and Systems Administrator<br class="">
<br class="">
<br class="">
Rothamsted Research is a company limited by guarantee, registered in England at Harpenden, Hertfordshire, AL5 2JQ under the registration number 2393175 and a not for profit charity number 802038.<br class="">
</blockquote>
<br class="">
<br class="">
Rothamsted Research is a company limited by guarantee, registered in England at Harpenden, Hertfordshire, AL5 2JQ under the registration number 2393175 and a not for profit charity number 802038.<br class="">
</blockquote>
<br class="">
</div>
</body>
</html>