[slurm-users] Correct way to do logrotation

[Timony, Mick]

Schedmd has docs about how to do this at:

https://slurm.schedmd.com/slurm.conf.html#SECTION_LOGGING


Our config at HMS looks like this:


/var/log/slurm/slurmctld.log {
  create 0640 slurm root
  daily
  dateext
  nocompress
  notifempty
  rotate 10
  sharedscripts
  postrotate
    /bin/pkill -x --signal SIGUSR2 slurmctld >/dev/null 2>&1
  endscript
  prerotate
    /bin/sh /usr/local/bin/pre_rotate.sh >/dev/null 2>&1
  endscript
}

Our pre_rotate scripts rsync's the logs to an NFS mount.

Cheers
--Mick

________________________________
From: slurm-users <slurm-users-bounces at lists.schedmd.com> on behalf of Davide DelVento <davide.quantum at gmail.com>
Sent: Tuesday, October 17, 2023 10:26 AM
To: Slurm User Community List <slurm-users at lists.schedmd.com>
Subject: Re: [slurm-users] Correct way to do logrotation

I'd be interested in this too, and I'm reposting only because the message was flagged as both "dangerous email" and "spam", so people may not have seen it (hopefully my reply will not suffer the same downfall...)

On Mon, Oct 16, 2023 at 3:26 AM Taras Shapovalov <tshapovalov at nvidia.com<mailto:tshapovalov at nvidia.com>> wrote:
Hello,

In the past it was recommended to reconfigure slurm daemons in logrotate script, sending a signal I believe was also the way to go. But recently I retested manual logrotation and I see that a removal of log file (for slurmctld, slurmdbd or slurmd) does not affect the logging of the daemons. The dameons just recreate the log files and continue to write logs there. What is the right way to go in case of the modern Slurm versions?

Best regards,

Taras
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20231017/8d9f2327/attachment-0001.htm>