[slurm-users] Python and R installation in a SLURM cluster

[Eric F. Alemany]

HI Miguel,

Thank you for your comment. That sounds pretty straight forward.
you never had issues with programs relying on the system files or relying on the home directory location?

Thanks
Eric
_____________________________________________________________________________________________________

Eric F.  Alemany
System Administrator for Research

Division of Radiation & Cancer  Biology
Department of Radiation Oncology

Stanford University School of Medicine
Stanford, California 94305

Tel:1-650-498-7969<tel:1-650-498-7969>  No Texting
Fax:1-650-723-7382<tel:1-650-723-7382>



On May 10, 2018, at 11:55 PM, Miguel Gutiérrez Páez <mgutierrez at gmail.com<mailto:mgutierrez at gmail.com>> wrote:

Hi,

I install all my apps in a shared storage, and change environment variables (path, vars, etc.) with lmod. It's very useful.

Regards.

El vie., 11 may. 2018 a las 6:19, Eric F. Alemany (<ealemany at stanford.edu<mailto:ealemany at stanford.edu>>) escribió:
Hi Lachlan,

Thank you for sharing your environment. Everyone has their own set of rules and i appreciate everyone’s input.
It seems as if the NFS share is a great place to start.

Best,
Eric
_____________________________________________________________________________________________________

Eric F.  Alemany
System Administrator for Research

Division of Radiation & Cancer  Biology
Department of Radiation Oncology

Stanford University School of Medicine
Stanford, California 94305

Tel:1-650-498-7969<tel:1-650-498-7969>  No Texting
Fax:1-650-723-7382<tel:1-650-723-7382>



On May 10, 2018, at 4:23 PM, Lachlan Musicman <datakid at gmail.com<mailto:datakid at gmail.com>> wrote:

On 11 May 2018 at 01:35, Eric F. Alemany <ealemany at stanford.edu<mailto:ealemany at stanford.edu>> wrote:
Hi All,

I know this might sounds as a very basic question: where in the cluster should I install Python and R?
Headnode?
Execute nodes ?

And is there a particular directory (path) I need to install Python and R.

Background:
SLURM on Ubuntu 18.04
1 headnode
4 execute nodes
NFS shared drive among all nodes.


Eric,

To echo the others: we have a /binaries nfs share that utilises the standard Environment Modules software so that researchers can manipulate their $PATH on the fly with module load/module unload. That share is mounted on all the nodes.

For Python, I use virtualenv's but instead of activating, the path is changed by the Module file. Personally, I find conda doesn't work very well in a shared environment. It's fine on a personal level/

For R, we have resorted to only installing the main point release because we have >700 libraries installed within R and I don't want to reinstall them every time. We do also have packrat installed so researchers can install their own libraries locally as well.


Cheers
L.





-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20180512/c1bfa356/attachment-0001.html>