[slurm-users] virtual memory limit exceeded

[Bjørn-Helge Mevik]

Noam Bernstein <noam.bernstein at nrl.navy.mil> writes:

> Can anyone shed some light on where the _virtual_ memory limit comes from?

Perhaps it comes from a VSizeFactor setting in slurm.conf:

       VSizeFactor
              Memory specifications in job requests apply to real memory size (also known as
              resident set size). It is possible to enforce virtual memory limits  for  both
              jobs  and  job  steps  by  limiting their virtual memory to some percentage of
              their real memory allocation. The VSizeFactor parameter specifies the job's or
              job  step's virtual memory limit as a percentage of its real memory limit. For
              example, if a job's real memory limit is 500MB and VSizeFactor is set  to  101
              then  the  job  will be killed if its real memory exceeds 500MB or its virtual
              memory exceeds 505MB (101 percent of the  real  memory  limit).   The  default
              value  is  0,  which disables enforcement of virtual memory limits.  The value
              may not exceed 65533 percent.

> 1. If I define DefMemPerCPU in the partition line, and the job doesn't request
> anything else, what memory measure should expect this to be the limit
> on? RSS?

Yes, RSS.

To test our memory settings, we often use a small program like the
attached C program.  It first allocates memory (to test vmem limit),
then fills it with values (to test rss limit), and finally re-reads the
memory (to check for swapping).

It is built like this

    gcc -DMB=$((30*1024)) -o malloc_30 malloc.c

for allocating 30 GiB.  (Yes, I could have made the size a command line
parameter, but I am lazy. :)

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: malloc.c
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20181109/b9c412d9/attachment.c>
-------------- next part --------------

-- 
Regards,
Bjørn-Helge Mevik, dr. scient,
Department for Research Computing, University of Oslo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 832 bytes
Desc: not available
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20181109/b9c412d9/attachment.sig>