<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=gb2312">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
Hi,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
I think the `Elapsed` or `ElapsedRaw` field is what you're looking for. Selected example from my system:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<span>$ sacct -X --allusers --format="AllocCPUS,Elapsed,ElapsedRaw,CPUTime,CPUTimeRAW"<br>
</span>
<div><br>
<span>AllocCPUS Elapsed ElapsedRaw CPUTime CPUTimeRAW<br>
</span>
<div>--------------------------------------------------------------------------------------------------------<br>
</div>
<div>64 10-00:00:29 864029 640-00:30:56 55297856</div>
<div><span>640 10-05:09:08 882548 6537-09:25:20 564830720<br>
</span>
<div>128 5-23:23:21 516201 764-17:48:48 66073728</div>
<div>...</div>
<div>...</div>
<div>...<br>
</div>
<span></span><br>
</div>
</div>
</div>
<div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Divide `CPUTimeRAW` by `ElapsedRaw` and you get the allocated CPUs for the job. Be careful with requested resources vs allocated resources if you have SMT enabled on your system. The values can be different depending on the options of your user's jobs.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
- Sebastian<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="Signature">
<div>
<div></div>
<div></div>
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica, sans-serif;">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px">
<div class="BodyFragment"><font size="2"><span style="font-size:10pt">
<div class="PlainText">
<div id="emailSignature" style="display:block">--</div>
<div id="emailSignature" style="display:block"><br>
<table style="box-sizing: border-box; border-collapse: collapse; border-spacing: 0px; background-color: rgb(255, 255, 255); max-width: 100%; text-align: start; font-family: Helvetica, Arial, san-serif; width: 480px;">
<tbody style="box-sizing:border-box">
<tr style="box-sizing:border-box">
<td style="box-sizing:border-box; vertical-align:top" width="75"><a href="http://www.unr.edu/" style="box-sizing: border-box; background: transparent none repeat scroll 0% 0%; color: rgb(46, 108, 162); text-decoration: underline;"><img alt="University of Nevada, Reno" style="box-sizing:border-box; vertical-align:middle" width="75" src="https://www.unr.edu/assets/images/blockn-100x100.png"></a></td>
<td style="box-sizing:border-box" width="20"> </td>
<td style="box-sizing:border-box; vertical-align:top" width="385">
<table style="box-sizing:border-box; border-collapse:collapse; max-width:100%; width:385px; text-align:left">
<tbody style="box-sizing:border-box">
<tr style="box-sizing:border-box">
<td style="box-sizing:border-box; vertical-align:top"><span style="box-sizing:border-box"><span style="box-sizing: border-box; color: rgb(0, 51, 102);"><strong style="box-sizing:border-box; font-weight:bold"><span style="box-sizing:border-box">Sebastian Smith<br style="box-sizing:border-box">
</span></strong></span><span style="box-sizing:border-box"><span style="box-sizing:border-box">High-Performance Computing Engineer<br style="box-sizing:border-box">
</span><span style="box-sizing:border-box">Office of Information Technology<br style="box-sizing:border-box">
</span><span style="box-sizing:border-box">1664 North Virginia Street<br style="box-sizing:border-box">
</span><span style="box-sizing:border-box">MS 0291<br style="box-sizing:border-box">
</span><span style="box-sizing:border-box"></span><span style="box-sizing:border-box"></span><br style="box-sizing:border-box">
<strong style="box-sizing:border-box; font-weight:bold">work-phone:<span> </span></strong><span style="box-sizing:border-box"><a href="tel:7756825050" style="box-sizing: border-box; background: transparent none repeat scroll 0% 0%; text-decoration: underline;">775-682-5050</a><br style="box-sizing:border-box">
</span><strong style="box-sizing:border-box; font-weight:bold"></strong><span style="box-sizing:border-box"></span><strong style="box-sizing:border-box; font-weight:bold">email:<span> </span></strong><span style="box-sizing:border-box"><a href="mailto:stsmith@unr.edu" style="box-sizing: border-box; background: transparent none repeat scroll 0% 0%; text-decoration: underline;">stsmith@unr.edu</a><br style="box-sizing:border-box">
</span><strong style="box-sizing:border-box; font-weight:bold">website:<span> </span></strong><span style="box-sizing:border-box"><a href="http://rc.unr.edu/" style="box-sizing: border-box; background: transparent none repeat scroll 0% 0%; text-decoration: underline;">http://rc.unr.edu</a><br style="box-sizing:border-box">
</span></span><br style="box-sizing:border-box">
</span></td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</span></font></div>
</div>
</div>
</div>
</div>
</div>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> slurm-users <slurm-users-bounces@lists.schedmd.com> on behalf of ФÕý¸Õ <guru.novice@gmail.com><br>
<b>Sent:</b> Thursday, July 16, 2020 8:15 PM<br>
<b>To:</b> slurm-users@lists.schedmd.com <slurm-users@lists.schedmd.com>; Mark Hahn <hahn@mcmaster.ca><br>
<b>Subject:</b> Re: [slurm-users] how to know the real utilization of a node when oversubscribe is set to FORCE (Mark Hahn)</font>
<div> </div>
</div>
<div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">Hi, Hahn</div>
<div>I mean the elapsed time.</div>
<div>In the example I mentioned earlier£¬a job run 10s £¬the elapsed time from sacct is 640s(10s*64), so how do i get the real elapsed time from sacct or other command line tools?</div>
<div><br>
</div>
<div>As you mentioned, i checked usercpu/systemcpu/totalcpu, but they all zero</div>
<div>Query command: sacct -T -X -S 2020-07-16T00:00:00 -E 2020-07-16T23:59:59 -r $partition --format=cputimeraw,usercpu,systemcpu,totalcpu,alloccpus,allocnodes,AllocTRES%40</div>
<div>
<div>CPUTimeRAW UserCPU SystemCPU TotalCPU AllocCPUS AllocNodes AllocTRES</div>
<div>---------- ---------- ---------- ---------- ---------- ---------- ----------------------------------------</div>
<div> 640 00:00:00 00:00:00 00:00:00 64 1 billing=64,cpu=64,node=1</div>
<div> 640 00:00:00 00:00:00 00:00:00 64 1 billing=64,cpu=64,node=1</div>
<div> 640 00:00:00 00:00:00 00:00:00 64 1 billing=64,cpu=64,node=1</div>
<div> 640 00:00:00 00:00:00 00:00:00 64 1 billing=64,cpu=64,node=1</div>
<div> 640 00:00:00 00:00:00 00:00:00 64 1 billing=64,cpu=64,node=1</div>
<div> 640 00:00:00 00:00:00 00:00:00 64 1 billing=64,cpu=64,node=1</div>
<div> 640 00:00:00 00:00:00 00:00:00 64 1 billing=64,cpu=64,node=1</div>
<div> </div>
</div>
<div>Jobs use 1/2/4/8/16/32/64 cores, but elapsed time are the same from sacct. </div>
<div>The real elasped time should be 10/20/40/80/160/320/640</div>
<div><br>
</div>
<div>regards.</div>
<div><br>
</div>
<div class="x_gmail_quote">
<blockquote class="x_gmail_quote" style="margin:0px 0px 0px 0.8ex; border-left:1px solid rgb(204,204,204); padding-left:1ex">
<br>
<br>
----------------------------------------------------------------------<br>
<br>
Message: 1<br>
Date: Thu, 16 Jul 2020 11:03:07 -0400 (EDT)<br>
From: Mark Hahn <<a href="mailto:hahn@mcmaster.ca" target="_blank">hahn@mcmaster.ca</a>><br>
To: Slurm User Community List <<a href="mailto:slurm-users@lists.schedmd.com" target="_blank">slurm-users@lists.schedmd.com</a>><br>
Subject: Re: [slurm-users] how to know the real utilization of a node<br>
when oversubscribe is set to FORCE<br>
Message-ID:<br>
<<a href="mailto:alpine.LFD.2.02.2007161036410.16842@coffee.psychology.mcmaster.ca" target="_blank">alpine.LFD.2.02.2007161036410.16842@coffee.psychology.mcmaster.ca</a>><br>
Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed<br>
<br>
> srun -N 1 -n 1 -p testA sleep 10<br>
> then the cpurawtime of this job recorded by slurm is 640s, but actually<br>
> this job only used 10s;<br>
> so, I want to know are there any way to get the real cputime used by this<br>
> job in slurm.<br>
<br>
if you really mean cpu time (compute-bound, not elapsed),<br>
then don't you just want usercpu, systemcpu and totalcpu from sacct?<br>
<br>
cputime/cputimeraw is just ncpus * elapsed.<br>
<br>
regards,<br>
-- <br>
operator may differ from spokesperson. <a href="mailto:hahn@mcmaster.ca" target="_blank">
hahn@mcmaster.ca</a><br>
<br>
<br>
<br>
------------------------------<br>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>