[slurm-users] SlurmDB Archive settings?

[Timony, Mick]

What I can tell you is that we have never had a problem reimporting the data back in that was dumped from older versions into a current version database.  So the import using sacctmgr must do the conversion from the older formats to the newer formats and handle the schema changes.
​That's the bit of info I was missing, I didn't realise that it outputs the data in a format that sacctmgr can read.

I will note that if you are storing job_scripts and envs those can eat up a ton of space in 21.08.  It looks like they've solved that problem in 22.05 but the archive steps on 21.08 took forever due to those scripts and envs.
​Yes, we are storing job_scripts with:

AccountingStoreFlags=job_script

I think when we made that decision, we decided that also saving the job_env would take up too much room as our DB is pretty big at the moment, at approx. 300GB with the o2_step_table and the o2_job_table taking up the most space for obvious reasons:

+----------------------------+-----------+
| Table                      | Size (GB) |
+----------------------------+-----------+
| o2_step_table              |    183.83 |
| o2_job_table               |    128.18 |


That's good advice Paul, much appreciated.

>took forever and actually caused issues with the archive process
I think that should be highlighted for other users!

For those interested, to find the tables sizes I did this:

SELECT table_name AS "Table", ROUND(((data_length + index_length) / 1024 / 1024 / 1024), 2) AS "Size (GB)" FROM information_schema.TABLES WHERE table_schema = "slurmdbd" ORDER BY (data_length + index_length) DESC;

Replace slurmdbd with the name of your database.

Cheers
--Mick


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20220714/13be2bc2/attachment-0001.htm>