[slurm-users] Python and R installation in a SLURM cluster

[Raymond Wan]

Hi Eric,


On 10/05/18 23:35, Eric F. Alemany wrote:
> I know this might sounds as a very basic question: where in
> the cluster should I install Python and R?
> Headnode?
> Execute nodes ?


I don't think there is a fixed rule for a question like this
and it depends on the compromise between what's easier for
you (as the system administrator) and your users.

We began by install it on every node using Ubuntu's apt-get
facility.  In fact, I'm sure Python is already installed
since it is required software for many of Debian/Ubuntu's
software.  However, we then had users with different
requirements.

For example, user A might have started on a research project
that lasted a couple of years and wants the software version
"frozen" as much as possible.  User B starts a new project a
year after user A and understandably doesn't want to be tied
down by user A, so s/he wants the latest version installed.

It was impossible to satisfy all users.  Of course, if it's
a bug fix or a security patch, we should apply it.  But I do
understand that some users want programs "frozen" since
doing research using a tool that updates regularly is difficult.

What we ended up doing now is to not install any software,
except those required by the system.  And then ask users to
install Miniconda/Anaconda in their home directories (which
is on an NFS share).  Then they look after the software
themselves.  For us, it's the only solution that worked...

There is duplication of software, but the duplication of a
binary file is better than trying to have to satisfy
multiple users.  At least we no longer have to make a
decision about which user to follow...

Your department's research is in cancer?  Then you might
have heard of Galaxy which *can* integrate with SLURM.  But
for the same reasons as above, we found it didn't work so well.

I guess what I've said isn't directly relevant to SLURM.
Sorry for going off on a tangent!

Ray