[slurm-users] Calculate the GPU usages

[Loris Bennett]

Dear Jeherul,

Jeherul Islam <jeherul at gmail.com> writes:

> Dear Loris, 
>
> Thanks for your reply. Here is the output for the same period but the result is not matching.
>
> #sacct --account=chemistry --format=jobid,user,ElapsedRaw,state,AllocGRES,ncpus --starttime=2021-05-01 --endtime=2021-08-31  | grep j.mira| grep gpu| awk '{sum += $3} END {print sum}'

I think you need the option '-X' for 'sacct'.  This will give you one
line per job rather than including the steps.  Without '-X' you are
counting the usage multiple times for each job.

Cheers,

Loris   


> 6835053          (6835053/60 = 113917 )
>
> # sreport cluster AccountUtilizationByUser cluster=**** user=j.mira start=2021-05-01 end=2021-08-31 --tres="gres/gpu" 
> --------------------------------------------------------------------------------
> Cluster/Account/User Utilization 2021-05-01T00:00:00 - 2021-08-30T23:59:59 (10540800 secs)
> Usage reported in TRES Minutes
> --------------------------------------------------------------------------------
>   Cluster         Account     Login     Proper Name      TRES Name     Used 
> --------- --------------- --------- --------------- -------------- -------- 
> ********       chemistry    j.mira          j.mira       gres/gpu   149434 
>
> On Wed, Sep 1, 2021 at 5:27 PM Loris Bennett <loris.bennett at fu-berlin.de> wrote:
>
>  Dear Jeherul,
>
>  Jeherul Islam <jeherul at gmail.com> writes:
>
>  > Dear All, 
>  >
>  > Please share the correct way of calculating the GPU usages.
>  > I am confused with sreport and sacct cmd. I am getting a different result. 
>  >
>  > # sreport cluster AccountUtilizationByUser cluster=**** user=j.mira start=2021-05-01 end=2021-08-31 --tres="gres/gpu" 
>
>  Here you have:
>
>    end=2021-08-31
>
>  > --------------------------------------------------------------------------------
>  > Cluster/Account/User Utilization 2021-05-01T00:00:00 - 2021-08-30T23:59:59 (10540800 secs)
>  > Usage reported in TRES Minutes
>  > --------------------------------------------------------------------------------
>  >   Cluster         Account     Login     Proper Name      TRES Name     Used 
>  > --------- --------------- --------- --------------- -------------- -------- 
>  > ****       chemistry    j.mira          j.mira       gres/gpu   149434 
>  >
>  > # sacct --account=chemistry --format=jobid,user,ElapsedRaw,state,AllocGRES,ncpus --starttime=2021-05-01 --endtime=2021-08-01  | grep j.mira| grep gpu| awk '{sum += $3} END {print sum}'
>
>  whereas here you have
>
>    --endtime=2021-08-01
>
>  > 4957060
>  >
>  > Please share the correct way.
>  >
>  > With Thanks and regards
>
>  so, without having checked your sacct/awk logic I would not expect the results to be the same.
>
>  Cheers,
>
>  Loris
>
>  -- 
>  Dr. Loris Bennett (Hr./Mr.)
>  ZEDAT, Freie Universität Berlin         Email loris.bennett at fu-berlin.de
-- 
Dr. Loris Bennett (Hr./Mr.)
ZEDAT, Freie Universität Berlin         Email loris.bennett at fu-berlin.de