<div dir="ltr"><div>Hi Loris, <br></div><div><br></div><div>No output is truncated. Here is the snapshot of the output. <br></div><div><br></div><div>#sacct -X --account=chemistry --user=j.mira --format=jobid,user,ElapsedRaw,state,AllocGRES,ncpus --starttime=2021-05-01 --endtime=2021-08-31 --noheader<br>23269 j.mira 1209627 TIMEOUT gpu:1 1 <br>25853 j.mira 1200060 CANCELLED+ gpu:1 1 <br>27335 j.mira 2 COMPLETED gpu:1 1 <br>27336 j.mira 0 COMPLETED gpu:1 1 <br>27339 j.mira 90 COMPLETED gpu:1 1 <br>27564 j.mira 0 CANCELLED+ gpu:1 1 <br>27565 j.mira 0 CANCELLED+ gpu:1 1 <br>30865 j.mira 0 CANCELLED+ gpu:1 1 <br>31575 j.mira 929809 COMPLETED gpu:1 1 <br>31576 j.mira 918413 COMPLETED gpu:1 1 <br>31573 j.mira 699059 COMPLETED gpu:1 1 <br>36060 j.mira 1207085 CANCELLED+ gpu:1 1 <br>40654 j.mira 682311 RUNNING gpu:1 1 <br>[root@gpu-login ~]# sacct -X --account=chemistry --user=j.mira --format=jobid,user,ElapsedRaw,state,AllocGRES,ncpus --starttime=2021-05-01 --endtime=2021-08-31 --noheader | awk '{sum += $3} END {print sum}'<br><b>6846556</b></div><div><b><br></b></div><div><b>It still showing a similar results.</b><br></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Sep 1, 2021 at 6:57 PM Loris Bennett <<a href="mailto:loris.bennett@fu-berlin.de">loris.bennett@fu-berlin.de</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">Dear Jeherul,<br>
<br>
Jeherul Islam <<a href="mailto:jeherul@gmail.com" target="_blank">jeherul@gmail.com</a>> writes:<br>
<br>
> Dear Loris, <br>
><br>
> When we grep it by the user name "j.mira" it will strike out the multiple counts. Again sacct is showing fewer gpu minutes than sreport. <br>
<br>
Yes, you are right, although instead of <br>
<br>
sacct --account=chemistry --format=jobid,user,ElapsedRaw,state,AllocGRES,ncpus --starttime=2021-05-01 --endtime=2021-08-31 | grep j.mira<br>
<br>
it would be more elegant just to write<br>
<br>
sacct --account=chemistry --user=j.mira --format=jobid,user,ElapsedRaw,state,AllocGRES,ncpus --starttime=2021-05-01 --endtime=2021-08-31 --noheader<br>
<br>
However, your problem might be caused by the fact that the default width<br>
of the 'AllocGRES' field is to small for the values. This will cause<br>
the values to be truncated, so your 'grep gpu' might miss some entries.<br>
You might need something like<br>
<br>
--format=jobid,user,ElapsedRaw,state,AllocGRES%60,ncpus<br>
<br>
Cheers,<br>
<br>
Loris<br>
<br>
<br>
> On Wed, 1 Sep, 2021, 6:03 PM Loris Bennett, <<a href="mailto:loris.bennett@fu-berlin.de" target="_blank">loris.bennett@fu-berlin.de</a>> wrote:<br>
><br>
> Dear Jeherul,<br>
><br>
> Jeherul Islam <<a href="mailto:jeherul@gmail.com" target="_blank">jeherul@gmail.com</a>> writes:<br>
><br>
> > Dear Loris, <br>
> ><br>
> > Thanks for your reply. Here is the output for the same period but the result is not matching.<br>
> ><br>
> > #sacct --account=chemistry --format=jobid,user,ElapsedRaw,state,AllocGRES,ncpus --starttime=2021-05-01 --endtime=2021-08-31 | grep j.mira| grep gpu| awk '{sum += $3} END {print sum}'<br>
><br>
> I think you need the option '-X' for 'sacct'. This will give you one<br>
> line per job rather than including the steps. Without '-X' you are<br>
> counting the usage multiple times for each job.<br>
><br>
> Cheers,<br>
><br>
> Loris <br>
><br>
> > 6835053 (6835053/60 = 113917 )<br>
> ><br>
> > # sreport cluster AccountUtilizationByUser cluster=**** user=j.mira start=2021-05-01 end=2021-08-31 --tres="gres/gpu" <br>
> > --------------------------------------------------------------------------------<br>
> > Cluster/Account/User Utilization 2021-05-01T00:00:00 - 2021-08-30T23:59:59 (10540800 secs)<br>
> > Usage reported in TRES Minutes<br>
> > --------------------------------------------------------------------------------<br>
> > Cluster Account Login Proper Name TRES Name Used <br>
> > --------- --------------- --------- --------------- -------------- -------- <br>
> > ******** chemistry j.mira j.mira gres/gpu 149434 <br>
> ><br>
> > On Wed, Sep 1, 2021 at 5:27 PM Loris Bennett <<a href="mailto:loris.bennett@fu-berlin.de" target="_blank">loris.bennett@fu-berlin.de</a>> wrote:<br>
> ><br>
> > Dear Jeherul,<br>
> ><br>
> > Jeherul Islam <<a href="mailto:jeherul@gmail.com" target="_blank">jeherul@gmail.com</a>> writes:<br>
> ><br>
> > > Dear All, <br>
> > ><br>
> > > Please share the correct way of calculating the GPU usages.<br>
> > > I am confused with sreport and sacct cmd. I am getting a different result. <br>
> > ><br>
> > > # sreport cluster AccountUtilizationByUser cluster=**** user=j.mira start=2021-05-01 end=2021-08-31 --tres="gres/gpu" <br>
> ><br>
> > Here you have:<br>
> ><br>
> > end=2021-08-31<br>
> ><br>
> > > --------------------------------------------------------------------------------<br>
> > > Cluster/Account/User Utilization 2021-05-01T00:00:00 - 2021-08-30T23:59:59 (10540800 secs)<br>
> > > Usage reported in TRES Minutes<br>
> > > --------------------------------------------------------------------------------<br>
> > > Cluster Account Login Proper Name TRES Name Used <br>
> > > --------- --------------- --------- --------------- -------------- -------- <br>
> > > **** chemistry j.mira j.mira gres/gpu 149434 <br>
> > ><br>
> > > # sacct --account=chemistry --format=jobid,user,ElapsedRaw,state,AllocGRES,ncpus --starttime=2021-05-01 --endtime=2021-08-01 | grep j.mira| grep gpu| awk '{sum += $3} END {print sum}'<br>
> ><br>
> > whereas here you have<br>
> ><br>
> > --endtime=2021-08-01<br>
> ><br>
> > > 4957060<br>
> > ><br>
> > > Please share the correct way.<br>
> > ><br>
> > > With Thanks and regards<br>
> ><br>
> > so, without having checked your sacct/awk logic I would not expect the results to be the same.<br>
> ><br>
> > Cheers,<br>
> ><br>
> > Loris<br>
> ><br>
> > -- <br>
> > Dr. Loris Bennett (Hr./Mr.)<br>
> > ZEDAT, Freie Universität Berlin Email <a href="mailto:loris.bennett@fu-berlin.de" target="_blank">loris.bennett@fu-berlin.de</a><br>
> -- <br>
> Dr. Loris Bennett (Hr./Mr.)<br>
> ZEDAT, Freie Universität Berlin Email <a href="mailto:loris.bennett@fu-berlin.de" target="_blank">loris.bennett@fu-berlin.de</a><br>
><br>
-- <br>
Dr. Loris Bennett (Hr./Mr.)<br>
ZEDAT, Freie Universität Berlin Email <a href="mailto:loris.bennett@fu-berlin.de" target="_blank">loris.bennett@fu-berlin.de</a><br>
<br>
</blockquote></div><br clear="all"><br>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div>Jeherul Islam <br>Technical Officer Grade I <br></div><div>Data Centre and High Performance Computing<br></div><div>Computer Centre<br>Indian Institute of Technology Guwahati<br>Guwahati-39<br>India<br>Office No :+91-361-258-3353<br><br></div></div></div>