[slurm-users] Virtual memory size requested by slurm
Renfro, Michael
Renfro at tntech.edu
Tue Jan 28 15:03:04 UTC 2020
On this part, I don’t think that’s always the case. On a node with 384 GB (with 2 GB reserved for the OS), we’ve got several jobs running under mem=32000:
=====
$ grep 'NodeName=gpunode\[00' /etc/slurm/slurm.conf
NodeName=gpunode[001-003] CoresPerSocket=14 RealMemory=382000 Sockets=2 ThreadsPerCore=1 Weight=10011 Gres=gpu:2
$ squeue -t R | grep gpunode001
555699 bigme lstm_rel_w namartinda R 16:17 *:*: 1 32000M gpunode001 2020-01-28T08:41:31 2020-01-28T08:41:31 2020-01-28T14:41:31 N/A
555700 bigme lstm_rel_w namartinda R 16:17 *:*: 1 32000M gpunode001 2020-01-28T08:41:31 2020-01-28T08:41:31 2020-01-28T14:41:31 N/A
…
555709 bigme lstm_rel_w namartinda R 16:17 *:*: 1 32000M gpunode001 2020-01-28T08:41:31 2020-01-28T08:41:31 2020-01-28T14:41:32 N/A
555688 bigme lstm_rel_w namartinda R 36:37 *:*: 1 32000M gpunode001 2020-01-28T08:21:10 2020-01-28T08:21:11 2020-01-28T14:21:11 N/A
$
=====
This is with SelectType=select/cons_res , SelectTypeParameters=CR_Core_Memory , and cgroups enabled.
> On Jan 27, 2020, at 10:45 PM, Mahmood Naderan <mahmood.nt at gmail.com> wrote:
>
> 1) --mem belongs to the physical memory which is requested by job and is later reserved for the job by slurm.
> So, on a 64GB node, if a user requests --mem=50GB, actually no one else can run a job with 10GB memory need.
More information about the slurm-users
mailing list