[slurm-users] [External] Re: can't lengthen my jobs log

[Ryan Novosielski]

As root, -a is effectively applied to every command I’m aware of.

--
#BlackLivesMatter
____
|| \\UTGERS,  	 |---------------------------*O*---------------------------
||_// the State	 |         Ryan Novosielski - novosirj at rutgers.edu
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
||  \\    of NJ	 | Office of Advanced Research Computing - MSB C630, Newark
     `'

> On Dec 4, 2020, at 1:54 PM, Prentice Bisbal <pbisbal at pppl.gov> wrote:
> 
> I know I'm very late to this thread, but were/are you using the --allusers flag to sacct? If not, sacct only returns results for the user running the command (not sure if this is the case for root - I never need to run sacct as root). This minor detail tripped me up a few days ago when I was expecting hundreds of thousands of results, and only got a couple hundred, which were *my* jobs from the period I was searching, not *all* jobs. I was about to have a heart attack because I thought someone purged the SlurmDB. 
> 
> For those wondering, I was using  -o to get only a few pieces of data per job  from my query, and userid wasn't one of them, which is why my mistake wasn't so obvious at first. 
> 
> Prentice
> 
> On 11/12/2020 4:49 PM, Erik Bryer wrote:
>> That worked pretty well in that I got more data than I ever have before by a lot. It only goes back about 18 days, but I'm not sure why. The slurmdbd.conf back then contained no directives on retaining logs, which is supposed to mean it defaults to retaining them indefinitely. On another test cluster it shows records back 2 days, which is about when I started fiddling with the settings. Could that have wiped the previous records, if they existed, or have my changes started the saving of older data. Still, this is progress.
>> 
>> Erik
>> From: slurm-users <slurm-users-bounces at lists.schedmd.com> on behalf of Sebastian T Smith <stsmith at unr.edu>
>> Sent: Thursday, November 12, 2020 2:32 PM
>> To: slurm-users at lists.schedmd.com <slurm-users at lists.schedmd.com>
>> Subject: Re: [slurm-users] can't lengthen my jobs log
>>  
>> Hi John,
>> 
>> Have you tried specifying a start time?  The default is 00:00:00 of the current day (depending on other options).  Example:
>> 
>> sacct -S 2020-11-01T00:00:00
>> 
>> Our accounting database retains all job data from the epoch of our system.
>> 
>> Best,
>> 
>> Sebastian
>> 
>> --
>> 
>> 	 	
>> Sebastian Smith
>> High-Performance Computing Engineer
>> Office of Information Technology
>> 1664 North Virginia Street
>> MS 0291
>> 
>> work-phone: 775-682-5050
>> email: stsmith at unr.edu
>> website: http://rc.unr.edu
>> 
>> From: slurm-users <slurm-users-bounces at lists.schedmd.com> on behalf of john abignail <abignailjohn at gmail.com>
>> Sent: Thursday, November 12, 2020 12:57 PM
>> To: slurm-users at lists.schedmd.com <slurm-users at lists.schedmd.com>
>> Subject: [slurm-users] can't lengthen my jobs log
>>  
>> Hi,
>> 
>> My jobs database empties after about 1 day. "sacct -a" returns no results. I've tried to lengthen that, but have been unsuccessful. I've tried adding the following to slurmdbd.conf and restarting slurmdbd:
>> ArchiveJobs=yes
>> PurgeEventAfter=1month
>> PurgeJobAfter=12month
>> PurgeResvAfter=1month
>> PurgeStepAfter=1month
>> PurgeSuspendAfter=1month
>> PurgeTXNAfter=12month
>> PurgeUsageAfter=24month
>> No job archives appear (in the default /tmp dir) either. What I'd like to do is have the slurm database retain information on jobs for at least a few weeks, writing out data beyond that threshold to files, but mainly I just want to keep job data in the database for longer.
>> 
>> Regards,
>> John
> -->