<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p>Hi</p>
<p><br>
</p>
<p>While doing some statistics on efficient CPU usage, I realized that sacct is reporting inexplicable (at least for me) high values for TotalCPU, UserCPU and SystemCPU. Here is a simple example (each job step is a infinite while loop): </p>
<p><br>
</p>
<p><i>sacct -j 64338003 --format=jobid,elapsed,ncpus,cputime,totalcpu,usercpu,systemcpu,nodelist</i></p>
<p> JobID Elapsed NCPUS <b>CPUTime </b> <b>TotalCPU</b> UserCPU SystemCPU NodeList</p>
<p>------------ ---------- ---------- ---------- ---------- ---------- ---------- ---------------</p>
<p>64338003 00:02:29 4 00:09:56 13:19:41<b> </b>13:19:36 00:05.054 anode033</p>
<p>64338003.ba+ 00:02:31 4 00:10:04 00:09.017 00:04.003 00:05.014 anode033</p>
<p>64338003.ex+ 00:02:30 4 00:10:00 00:00.001 00:00:00 00:00.001 anode033</p>
<p>64338003.0 00:02:32 1 <b> 00:02:32</b> <b>03:19:52 </b>
03:19:52 00:00.013 anode033</p>
<p>64338003.1 00:02:32 1 <b>00:02:32</b> <b>03:19:54 </b>
03:19:54 00:00.008 anode033</p>
<p>64338003.2 00:02:32 1 <b> 00:02:32</b> <b>03:19:53 </b> 03:19:53 00:00.010 anode033</p>
<p>64338003.3 00:02:32 1 <b>00:02:32 </b> <b>03:19:52 </b>
03:19:52 00:00.007 anode033</p>
<p><br>
</p>
<p><b>I would expect CPUTime to be the upper limit for TotalCPU.</b></p>
<p><br>
</p>
<p>Looking at <span style="font-size: 12pt;"><i>cpuacct.stat</i> for job step3:</span></p>
<p><br>
</p>
<p><i>cat </i><span style="font-size: 12pt;"><i>/cgroup/cpuacct/slurm/uid_6994/job_64338003/step_3/cpuacct.stat</i></span></p>
<p>user <b>14902 </b>(~149 = 00:02:29) </p>
<p>system 0</p>
<p><br>
</p>
<p>This value corresponds to the expected CPU usage of a single job step.</p>
<p><br>
</p>
<p>We are running Slurm 18.08.4 with</p>
<p>JobAcctGatherType=jobacct_gather/cgroup<br>
</p>
<p><br>
</p>
<p>Does anyone have an explanation for those high values reported by sacct?<br>
</p>
<p><br>
</p>
<p><br>
</p>
<p>Best,</p>
<p>Nico</p>
<p><br>
</p>
<div id="Signature">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px">
<div style="font-family:Tahoma; font-size:13px"><font class="Apple-style-span" size="3"><span class="Apple-style-span" style="font-size: 12px;">Universitaet Bern</span></font>
<div style="font-family:Helvetica; font-size:medium"><font class="Apple-style-span" size="3"><span class="Apple-style-span" style="font-size:12px">Abt. Informatikdienste<br>
<br>
Nico Färber<br>
High Performance Computing<br>
</span></font></div>
<br>
<div style="font-family:Helvetica; font-size:medium"><font class="Apple-style-span" size="3"><span class="Apple-style-span" style="font-size:12px">Gesellschaftsstrasse 6</span></font></div>
<div style="font-family:Helvetica; font-size:medium"><font class="Apple-style-span" size="3"><span class="Apple-style-span" style="font-size:12px">CH-3012 Bern<br>
Raum 104<br>
</span></font><font class="Apple-style-span" size="3"><span class="Apple-style-span" style="font-size:12px">Tel. +41 (0)31 631 51 89</span></font><br>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>