[slurm-users] seff: incorrect memory usage (18.08.5-2)

[Marcus Wagner]

Hi Loris,

ok, THAT seems really much.
What do you use for gathering these values? jobacct_gather/cgroup?

If I remember right, there was a discussion lately in this list 
regarding the JobAcctGatherType, yet I do not remember the outcame. I 
remember though, that someone pointed to SLUG18 (or 17?). Nonetheless 
there is the following whitepaper:

https://slurm.schedmd.com/SLUG18/field_notes2.pdf

My current recommended deployment has:
○
proctrack/cgroup, task/cgroup, jobacct_gather/cgroup
○
cgroup.conf - ConstrainCores, ContrainDevices, ConstrainRAMSpace,
ConstrainSwapSpace all enabled
○
PrologFlags=contain with pam_slurm_adopt setup appropriately
○
LaunchParameters=send_gids
○
ReconfigFlags=KeepPartInfo

Best
Marcus


On 2/26/19 1:20 PM, Loris Bennett wrote:
> Hi Marcus,
>
> Thanks for the response, but that doesn't seem to be the issue.  The
> problem seems to be that the raw data are incorrect:
>
> Slurm data: ... Ncpus Nnodes Ntasks    Reqmem PerNode   Cput Walltime                 Mem ExitStatus
> Slurm data: ...    50      2      1 102400000       0 503611    16310 1.8014398509482e+16          0
>
> Cheers,
>
> Loris
>
> Marcus Wagner <wagner at itc.rwth-aachen.de> writes:
>
>> Hi Loris,
>>
>> I assume, this job used FAIRLY few memory, in the kb range, might that be true?
>>
>> replace
>>
>> sub kbytes2str {
>>      my $kbytes = shift;
>>      if ($kbytes == 0) {
>>          return sprintf("%.2f %sB", 0.0, 'M');
>>      }
>>      my $mul = 1024;
>>
>>      my $exp = int(log($kbytes) / log($mul));
>>      my @pre = qw/ M G T P E /;
>>      my $pre = $pre[$exp-1];
>>
>>      return sprintf("%.2f %sB", ($kbytes / pow($mul, $exp)), $pre ? $pre : "");
>> }
>>
>>
>> with
>>
>>      my @pre = qw/ k M G T P E /;
>>      my $pre = $pre[$exp];
>>
>>
>> Best
>> Marcus
>>
>> On 2/26/19 10:08 AM, Loris Bennett wrote:
>>> 48.00 EB (estimated maximum)
>>>     Memory Efficiency: 26388279066.62% of 195.31 GB (1.95 GB/core)

-- 
Marcus Wagner, Dipl.-Inf.

IT Center
Abteilung: Systeme und Betrieb
RWTH Aachen University
Seffenter Weg 23
52074 Aachen
Tel: +49 241 80-24383
Fax: +49 241 80-624383
wagner at itc.rwth-aachen.de
www.itc.rwth-aachen.de

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20190226/1c457039/attachment.html>