[slurm-users] Get GPU usage from sacct?

Miguel Oliveira miguel.oliveira at crick.ac.uk
Fri Nov 15 15:06:58 UTC 2019


Thank! Nice code and just what I was needing! A few wrinkles:

a) on reading the Gres from scontrol for each job on my version this is on a TRES record not as an individual Gres. Possibly version/configuration issue.
b) converting pid2id from /proc/<pid>/cgroup is problematic on array jobs.

Again many thanks for sharing,

MAO

--
Miguel Afonso Oliveira
Senior HPC Systems Engineer
The Francis Crick Institute
1, Midland Road
London NW1 1AT
United Kingdom

T: 020 3796 3712
E: miguel.oliveira at crick.ac.uk
W: www.crick.ac.uk

>
> We have created
> https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FAaltoScienceIT%2Fansible-role-sacct_gpu%2F&data=02%7C01%7C%7C39c349510efb4c7a8d8608d769c3d25f%7C4eed7807ebad415aa7a99170947f4eae%7C0%7C0%7C637094162080068245&sdata=NIk60Oa3N6FYP84XubaW11ZauA28BzxpCc6DepaoGR8%3D&reserved=0 as a quick
> hack to put GPU utilization stats into the comment field at the end of
> the job.
>
> The above is an ansible role, but if you're not using ansible you can
> just pull the scripts from the "files" subdirectory.
>
> --
> Janne Blomqvist, D.Sc. (Tech.), Scientific Computing Specialist
> Aalto University School of Science, PHYS & NBE
> +358503841576 || janne.blomqvist at aalto.fi
>

The Francis Crick Institute Limited is a registered charity in England and Wales no. 1140062 and a company registered in England and Wales no. 06885462, with its registered office at 1 Midland Road London NW1 1AT



More information about the slurm-users mailing list