[slurm-users] Calculate the GPU usages

[Juergen Salk]

Dear Jeherul,

sacct is for job accounting, sreport for cluster usage accounting. 

Did you maybe had any resource reservations for this user in place during 
that period of time? 

To my very best knowledge, resource reservations for one or more users 
do count in terms of cluster usage as reported by sreport for that user(s), 
even if there were no jobs submitted into the reservation, while sacct
reports the resources actually allocated by jobs but does not count idle
reservation time.

Best regards,
Jürgen


* Jeherul Islam <jeherul at gmail.com> [210901 18:24]:
> Dear Loris,
> 
> When we grep it by the user name "j.mira" it will strike out the multiple
> counts. Again sacct is showing fewer gpu minutes than sreport.
> 
> On Wed, 1 Sep, 2021, 6:03 PM Loris Bennett, <loris.bennett at fu-berlin.de>
> wrote:
> 
> > Dear Jeherul,
> >
> > Jeherul Islam <jeherul at gmail.com> writes:
> >
> > > Dear Loris,
> > >
> > > Thanks for your reply. Here is the output for the same period but the
> > result is not matching.
> > >
> > > #sacct --account=chemistry
> > --format=jobid,user,ElapsedRaw,state,AllocGRES,ncpus --starttime=2021-05-01
> > --endtime=2021-08-31  | grep j.mira| grep gpu| awk '{sum += $3} END {print
> > sum}'
> >
> > I think you need the option '-X' for 'sacct'.  This will give you one
> > line per job rather than including the steps.  Without '-X' you are
> > counting the usage multiple times for each job.
> >
> > Cheers,
> >
> > Loris
> >
> >
> > > 6835053          (6835053/60 = 113917 )
> > >
> > > # sreport cluster AccountUtilizationByUser cluster=**** user=j.mira
> > start=2021-05-01 end=2021-08-31 --tres="gres/gpu"
> > >
> > --------------------------------------------------------------------------------
> > > Cluster/Account/User Utilization 2021-05-01T00:00:00 -
> > 2021-08-30T23:59:59 (10540800 secs)
> > > Usage reported in TRES Minutes
> > >
> > --------------------------------------------------------------------------------
> > >   Cluster         Account     Login     Proper Name      TRES Name
> >  Used
> > > --------- --------------- --------- --------------- --------------
> > --------
> > > ********       chemistry    j.mira          j.mira       gres/gpu
> >  149434
> > >
> > > On Wed, Sep 1, 2021 at 5:27 PM Loris Bennett <loris.bennett at fu-berlin.de>
> > wrote:
> > >
> > >  Dear Jeherul,
> > >
> > >  Jeherul Islam <jeherul at gmail.com> writes:
> > >
> > >  > Dear All,
> > >  >
> > >  > Please share the correct way of calculating the GPU usages.
> > >  > I am confused with sreport and sacct cmd. I am getting a different
> > result.
> > >  >
> > >  > # sreport cluster AccountUtilizationByUser cluster=**** user=j.mira
> > start=2021-05-01 end=2021-08-31 --tres="gres/gpu"
> > >
> > >  Here you have:
> > >
> > >    end=2021-08-31
> > >
> > >  >
> > --------------------------------------------------------------------------------
> > >  > Cluster/Account/User Utilization 2021-05-01T00:00:00 -
> > 2021-08-30T23:59:59 (10540800 secs)
> > >  > Usage reported in TRES Minutes
> > >  >
> > --------------------------------------------------------------------------------
> > >  >   Cluster         Account     Login     Proper Name      TRES Name
> >  Used
> > >  > --------- --------------- --------- --------------- --------------
> > --------
> > >  > ****       chemistry    j.mira          j.mira       gres/gpu
> >  149434
> > >  >
> > >  > # sacct --account=chemistry
> > --format=jobid,user,ElapsedRaw,state,AllocGRES,ncpus --starttime=2021-05-01
> > --endtime=2021-08-01  | grep j.mira| grep gpu| awk '{sum += $3} END {print
> > sum}'
> > >
> > >  whereas here you have
> > >
> > >    --endtime=2021-08-01
> > >
> > >  > 4957060
> > >  >
> > >  > Please share the correct way.
> > >  >
> > >  > With Thanks and regards
> > >
> > >  so, without having checked your sacct/awk logic I would not expect the
> > results to be the same.
> > >
> > >  Cheers,
> > >
> > >  Loris
> > >
> > >  --
> > >  Dr. Loris Bennett (Hr./Mr.)
> > >  ZEDAT, Freie Universität Berlin         Email
> > loris.bennett at fu-berlin.de
> > --
> > Dr. Loris Bennett (Hr./Mr.)
> > ZEDAT, Freie Universität Berlin         Email loris.bennett at fu-berlin.de
> >
> >

-- 
GPG A997BA7A | 87FC DA31 5F00 C885 0DC3  E28F BD0D 4B33 A997 BA7A