Hi,
I am continuing to try to get thic clickhouse integration working.
I have a completed array job which shows up in sacct. The job id is 245385. If i do a select in the accounting db for id_job=245385 then I get one row. I then see that the job_db_inx for this job is 497857 and then selecting in the step table for this job_db_inx returns no rows.
So the question is that since the information for the job does not seem to be in the job or step table, where is sacct getting this info from ?
Does anyone have any information on recreating the output of sacct by using queries on the db ?
Regards,
Emyr James Head of Scientific IT CRG - Centre for Genomic Regulation
________________________________ From: Emyr James via slurm-users slurm-users@lists.schedmd.com Sent: 12 July 2024 11:51 To: slurm-users@lists.schedmd.com slurm-users@lists.schedmd.com Subject: [slurm-users] Job Step State
Dear all,
I am working on a script to take completed job accounting data from the slurm accounting database and insert the equivalent data into a clickhouse table for fast reporting
I can see that all the information is included in the cluster_job_table and cluster_job_step_table which seem to be joined on job_db_inx
To get the cpu usage and peak memory usage etc. I can see that I need to parse the tres columns in the job steps. I couldn't find any column called MaxRSS in the database even though the sacct command prints this. I then found some data in tres_table and assume that sacct is using this. Please correct me if I'm wrong and if sacct is getting information from somwhere other than the accounting database?
for the state column I get this...
select state, count(*) as num from crg_step_table group by state order by num desc limit 10;
+-------+--------+ | state | num | +-------+--------+ | 3 | 590635 | | 5 | 28345 | | 4 | 4401 | | 11 | 962 | | 1 | 8 | +-------+--------+
When I use sacct I see statuses seach as COMPLETED, OUT_OF_MEMORY etc. so there must be a mapping somewhere between these state ids and that text. Can someone prvide that mapping or point me to where it's defined in the database or in the code ?
Many thanks,
Emyr James Head of Scientific IT CRG - Centre for Genomic Regulation