[slurm-users] slurmdbd not showing job accounting

[Dave Botsch]

So the only mention of this I can find is a "rollup" mention in
https://slurm.schedmd.com/slurmdbd.conf.html , now that I specificaly
googled for "slurmdbd rollup" .

So if this hourly summary is the right behaviour, I'd request it be
better documented -- nothing at all is mentioned in
https://slurm.schedmd.com/accounting.html

Thanks.

On Sun, Oct 14, 2018 at 02:18:48PM -0700, Douglas Jacobsen wrote:
> Sreport shows data that is summarized hourly. Restarting slurmdbd can delay
> this process.  If some jobs are missing end records it can massively slow
> the process because it may need to pick a much earlier start time in the
> past to summarize.
> 
> Sacctmgr show runawayjobs can help identify if you are in this situation
> 
> On Sun, Oct 14, 2018 at 2:05 PM Dave Botsch <botsch at cnf.cornell.edu> wrote:
> 
> > This seems to reflect what I am seeing. Someone earlier mentioned
> > multiple restarts of slurmdbd... those restarts never made data appear
> > unless right around on the hour.
> >
> > It's as if instead of data getting sent right through slurmdbd that
> > something in slurmdbd is just doing an hourly check of the text based
> > sacct records (which I don't understand why those are even there if not
> > configured in slurm.conf).
> >
> > Thanks.
> >
> > On Sun, Oct 14, 2018 at 12:08:10PM +0100, Antony Cleave wrote:
> > > I have noticed on several clusters that sreport can be upto one hour out
> > of
> > > date i.e. it will update on the hour every hour.
> > >
> > > sacct does not behave this way and is always up to date.
> > >
> > > I cannot see this stated in the docs or see any config settings to
> > control
> > > this but it happens on the last 17.02 cluster I checked.
> > >
> > > Antony
> > >
> > > On 14 Oct 2018 11:58, "Steven Dick" <kg4ydw at gmail.com> wrote:
> > >
> > > It is documented that you need to create the cluster in the database.
> > >
> > > It is not documented that the accounting system won't work until you
> > > restart slurmdbd multiple times before it starts collecting accounting
> > > records.
> > >
> > > Also, none of the necessary restarts are needed on an upgrade -- only
> > > when slurm is initialized for a new cluster.
> > >
> > >
> > > On Sun, Oct 14, 2018 at 4:12 AM Ole Holm Nielsen
> > > <Ole.H.Nielsen at fysik.dtu.dk> wrote:
> > > > Correct, and this is documented in the Slurm accounting setup page:
> > > > https://slurm.schedmd.com/accounting.html#database-configuration
> > > >
> >
> > --
> > ********************************
> > David William Botsch
> > Programmer/Analyst
> > @CNFComputing
> > botsch at cnf.cornell.edu
> > ********************************
> >
> > --
> Sent from Gmail Mobile

-- 
********************************
David William Botsch
Programmer/Analyst
@CNFComputing
botsch at cnf.cornell.edu
********************************