[slurm-users] Virtual memory size requested by slurm

[Renfro, Michael]

On this part, I don’t think that’s always the case. On a node with 384 GB (with 2 GB reserved for the OS), we’ve got several jobs running under mem=32000:

=====

$ grep 'NodeName=gpunode\[00' /etc/slurm/slurm.conf
NodeName=gpunode[001-003]  CoresPerSocket=14 RealMemory=382000 Sockets=2 ThreadsPerCore=1 Weight=10011 Gres=gpu:2
$ squeue -t R | grep gpunode001
555699 bigme lstm_rel_w namartinda  R        16:17 *:*: 1     32000M     gpunode001       2020-01-28T08:41:31 2020-01-28T08:41:31 2020-01-28T14:41:31 N/A
555700 bigme lstm_rel_w namartinda  R        16:17 *:*: 1     32000M     gpunode001       2020-01-28T08:41:31 2020-01-28T08:41:31 2020-01-28T14:41:31 N/A
…
555709 bigme lstm_rel_w namartinda  R        16:17 *:*: 1     32000M     gpunode001       2020-01-28T08:41:31 2020-01-28T08:41:31 2020-01-28T14:41:32 N/A
555688 bigme lstm_rel_w namartinda  R        36:37 *:*: 1     32000M     gpunode001       2020-01-28T08:21:10 2020-01-28T08:21:11 2020-01-28T14:21:11 N/A
$

=====

This is with SelectType=select/cons_res , SelectTypeParameters=CR_Core_Memory , and cgroups enabled.

> On Jan 27, 2020, at 10:45 PM, Mahmood Naderan <mahmood.nt at gmail.com> wrote:
> 
> 1) --mem belongs to the physical memory which is requested by job and is later reserved for the job by slurm.
> So, on a 64GB node, if a user requests --mem=50GB, actually no one else can run a job with 10GB memory need.