[slurm-users] swap size

[Raymond Wan]

Hi Ashton,

On Sat, Sep 22, 2018 at 5:34 AM A <andrealphus at gmail.com> wrote:
> So I'm wondering if 20% is enough, or whether it should scale by the number of single jobs I might be running at any one time. E.g. if I'm running 10 jobs that all use 20 gb of ram, and I suspend, should I need 200 gb of swap?


Perhaps I'm a bit clueless here, but maybe someone can correct me if I'm wrong.

I don't think swap space or a swap file is used like that.  If you
have 256 GB of memory and a 256 GB swap file (I don't suggest this
size...it just makes my math easier :-) ), then from the point of view
of the OS, it will appear there is 512 GB of memory.  So, this is
memory that is used while it is running...for reading in data, etc.

SLURM's ability to suspend jobs must be storing the state in a
location outside of this 512 GB.  So, you're not helping this by
allocating more swap.

What you are doing is perhaps allowing more jobs to run concurrently,
but I would caution against allocating more swap space.  After all,
disk read/write is much slower than memory.  If you can run 10 jobs
within 256 GB of memory but 20 jobs within 512 GB of (memory + swap
space), I think you should do some kind of test to see if it would be
faster to just let 10 jobs run.  Since disk I/O is slower, I doubt
you're going to get double the running time.

Personally, I still create swap space, but I agree with John that a
server with 256 GB of memory shouldn't need any swap at all.  With
what I run, if it uses more than the amount of memory that I have, I
tend to stop it and find another computer to run it.  If there isn't
one, I need to admit I can't do it.  Because once it exceeds the
amount of main memory, it will start thrashing and, thus, take a lot
of time to run.  i.e., a day versus a week or more...

On the other hand, we do have servers that double as desktops during
the day.  An alternative for you to consider is to only allocate 200
GB of memory to slurm, for example, leaving 56 GB for your own use.
Yes, this means that, at night, 56 GB of RAM is wasted, but during the
day, they can also continue running.  Of course, you should set aside
an amount that is enough for you...56 GB was chosen to make my math
easier as well.  :-)

If something I said here isn't quite correct, I'm happy to have
someone correct me...

Ray