[slurm-users] Environment modules

[Brian Andrus]

I never understood the benefit of lmod vs environment modules, although 
I see a push to use lmod. Particularly for versioning, but I always 
managed versions by using directory hierarchies. For instance, OpenMPI:

mpi/openmpi/3.2.3
mpi/openmpi/4.0.2

Then set a conflict on the top path (mpi/openmpi), which forces only one 
version to be able to be loaded. I also set paths so specific versions 
of libraries become available depending on what environment you select 
(gcc vs intel for example).

Is there something besides versioning that lmod shines at?

Brian Andrus


On 11/24/2019 12:48 AM, Yair Yarom wrote:
> We also use lmod here. Very useful when different versions are needed 
> or for any software installations outside the distribution.
>
> However, our environment is heterogenous, and the software modules 
> might have different versions/paths on different nodes. This creates 
> an issue when users run 'module load something' on the submission 
> node, and then run srun/sbatch and the wrong module is loaded (or just 
> the wrong PATH is kept).
> To solve this (in an overly complicated manner..) we have a taskprolog 
> plugin and a spank plugin that: a. resets the modules at submission, 
> and b. let the user add "--module <something>" to the srun/sbatch so 
> that the appropriate module will be loaded on the nodes.
>
> On Sat, Nov 23, 2019 at 11:55 AM William Brown 
> <william at signalbox.org.uk <mailto:william at signalbox.org.uk>> wrote:
>
>     Agreed, I have just been setting up Lmod on a national compute
>     cluster where I am a non-privileged cluster and on an internal
>     cluster where I have full rights.  It works very well, and Lmod
>     can read theTcl module files also.  The most recent version has
>     some extra features specially for Slurm.  An I use EasyBuild,
>     saves hundreds of hours of effort.   I do quite often have to hand
>     create simple module files for software with no EasyConfig but I
>     can just copy the structure from module files created by EasyBuild
>     so it has never been a great problem.
>
>     The best bit of modules is being able to offer multiple
>     conflicting versions of software like Java, Perl, R etc.
>
>     William
>
>     On Sat, 23 Nov 2019 at 03:57, Chris Samuel <chris at csamuel.org
>     <mailto:chris at csamuel.org>> wrote:
>
>         On 22/11/19 9:37 am, Mariano.Maluf wrote:
>
>         > The cluster is operational but I need to install and configure
>         > environment modules.
>
>         If you use Easybuild to install your HPC software then it can
>         take care
>         of the modules too for you.  I'd also echo the recommendation
>         from
>         others to use Lmod.
>
>         Website: https://easybuilders.github.io/easybuild/
>         Documentation: https://easybuild.readthedocs.io/
>
>         All the best,
>         Chris
>         -- 
>           Chris Samuel  : http://www.csamuel.org/  : Berkeley, CA, USA
>
>
>
> -- 
>    /|        |
>    \/        |Yair Yarom | Senior DevOps Architect
>    []        |The Rachel and Selim Benin School
>    []  /\     |of Computer Science and Engineering
>    []//\\/   |The Hebrew University of Jerusalem
>    [//   \\   |T +972-2-5494522 | F +972-2-5494522
>    //     \   |irush at cs.huji.ac.il <mailto:irush at cs.huji.ac.il>
>   //         |
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20191124/4c6e3dbb/attachment.htm>