[slurm-users] slurmdbd not showing job accounting

[Steven Dick]

I've found that when creating a new cluster, slurmdbd does not
function correctly right away.  It may be necessary to restart
slurmdbd at several points during the slurm installation process to
get everything working correctly.

Also, slurmctld will buffer the accounting data until slurmdbd starts
functioning correctly, so it is possible if you restart slurmdbd you
will find that all your missing accounting data shows up at once.

One of the critical steps is   sacctmgr create cluster $clustername
where $clustername matches the ClusterName parameter in slurm.conf

You probably need to restart slurmdbd after doing that to make sure it
takes full effect..

On Fri, Oct 12, 2018 at 4:10 PM Dave Botsch <botsch at cnf.cornell.edu> wrote:
>
> Hi.
>
> I am setting up a new slurm cluster instance. And I just went through
> what I thought were the right steps to get job accounting going with
> slurmdbd.
>
> So I know that slurmdbd itself works as I can use the sacctmgr commands
> to add users and accounts, and the users cannot run jobs unless I first
> add them with sacctmgr .
>
> What's interesting is that sreport is not showing any of the job
> information pieces at all.
>
> Eg:
>
> So, I use srun to run a quick test job, and then:
>
> $ sreport cluster UserUtilizationByAccount End=11/01/18
>
> comes back empty.
>
> sacct show information which is evidently still going to
> /var/log/slurm_jobacct.log ...
>
> even though in slurm.conf I have:
>
> AccountingStorageType=accounting_storage/slurmdbd
>
>
> Now what's even more interesting is that as I was writing this email,
> the output of sreport has updated to show more info, as if some
> scheduled job someplace updated something. I don't see anything in cron,
> though. As far as I can tell, the only thing I did was to use sacctmgr
> to delete and readd a user.
>
> Any thoughts on what is causing sreport's output to update/not update?
>
> Thanks!
>
>
> --
> ********************************
> David William Botsch
> Programmer/Analyst
> @CNFComputing
> botsch at cnf.cornell.edu
> ********************************
>