[slurm-users] Python and R installation in a SLURM cluster

[John Hearns]

Completely as an aside, the next question then is 'Aha - but what happens
when you have new users on the cluster'
I am currently working with sssd authentication and with the pam_mkhomedir
plugin.
I guess if an MPI job is launched using ssh then pam_mkhomedir would
automatically create the home directory too!


On 12 May 2018 at 22:02, John Hearns <hearnsj at googlemail.com> wrote:

> Well I DID say that you need 'what looks like a home directory'.
> So yes indeed you prove, correctly, that this works just fine!
>
> On 12 May 2018 at 20:17, Eric F. Alemany <ealemany at stanford.edu> wrote:
>
>>
>> Hi John,
>>
>> No worries at all. I take all ideas, comments and advice with the
>> greatest respect.
>> I know that my questions and knowledge of SULRM/cluster are very basics.
>> I have built very small and simple cluster. This is an opportunity for me
>> to learn at a bigger scale.
>>
>> Each node has a user home directory because I thought that all users must
>> have the same uid and guid across the nodes.
>> The two users (post-docs) who will use the cluster will only log in to
>> the headnode and install their programs and run their jobs from the
>> headnode.
>>
>> Thank you for your help.
>>
>> Best,
>> Eric
>>
>> .___________________________________________________________
>> __________________________________________
>>
>> * Eric F.  Alemany *
>> *System Administrator for Research*
>>
>> Division of Radiation & Cancer  Biology
>> Department of Radiation Oncology
>>
>> Stanford University School of Medicine
>> Stanford, California 94305
>>
>> Tel:1-650-498-7969  No Texting
>> Fax:1-650-723-7382
>>
>> On May 12, 2018, at 00:08, John Hearns <hearnsj at googlemail.com> wrote:
>>
>> Eric, I'm sorry to be a little prickly here.
>> Each node has an independent home directory for the user?
>> How then do applications update dot files?
>> How then would as a for instance do the users edit the .bashrc file to
>> bring Anaconda into their paths?
>>
>> Beofre anyone says it, a proper Modules system is the way forward.
>> But I know that when you install Anaconda as a user it adds the path to
>> your .bashrc
>> Which fouls up Gnomes dbus daemon, which is another tale.
>>
>>
>>
>>
>>
>>
>>
>> On 12 May 2018 at 07:09, Eric F. Alemany <ealemany at stanford.edu> wrote:
>>
>>> Hi Chris,
>>>
>>> Thank you for your comments. I will look at Easybuild. There are quite a
>>> few options to automate the creation of software modules.
>>>
>>> I will be doing lots of reading this week-end.
>>>
>>> By the way, i signed up to the Beowulf mailing list.
>>>
>>> Thank you,
>>>
>>> Eric
>>> ____________________________________________________________
>>> _________________________________________
>>>
>>> * Eric F.  Alemany *
>>> *System Administrator for Research*
>>>
>>> Division of Radiation & Cancer  Biology
>>> Department of Radiation Oncology
>>>
>>> Stanford University School of Medicine
>>> Stanford, California 94305
>>>
>>> Tel:1-650-498-7969  No Texting
>>> Fax:1-650-723-7382
>>>
>>>
>>>
>>> On May 11, 2018, at 12:56 AM, Chris Samuel <chris at csamuel.org> wrote:
>>>
>>> On Friday, 11 May 2018 5:11:38 PM AEST John Hearns wrote:
>>>
>>> Eric, my advice would be to definitely learn the Modules system and
>>> implement modules for your users.
>>>
>>>
>>> I will echo that, and the suggestion of shared storage (we use our
>>> Lustre
>>> filesystem for that).  I would also suggest looking at a system to help
>>> you
>>> automate building of software packages.   Not only does this help
>>> replicate
>>> builds, but it also gives you access to the community who write the
>>> recipes
>>> for them - and that itself can be very valuable.
>>>
>>> We use Easybuild (which also automates the creation of software modules
>>> - and
>>> I would suggest using the Lmod system for that):
>>>
>>> https://easybuilders.github.io/easybuild/
>>>
>>> But there's also Spack too:
>>>
>>> https://spack.io/
>>>
>>> As another resource (as we are going off topic from Slurm here), I would
>>> suggest the Beowulf list as a mailing list that deals with Linux based
>>> HPC
>>> systems of many different scales.  Disclosure: I now caretake the list,
>>> but
>>> it's been going since the 1990s.
>>>
>>> http://beowulf.org/
>>>
>>> All the best!
>>> Chris
>>> --
>>> Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC
>>>
>>>
>>>
>>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20180512/e8295e52/attachment-0001.html>