[slurm-users] Heterogeneous HPC

[Renfro, Michael]

MATLAB container at NVIDIA’s NGC: https://ngc.nvidia.com/catalog/containers/partners:matlab

Should be compatible with Docker and Singularity, but read the fine print on licensing.

> On Sep 19, 2019, at 8:22 AM, Thomas M. Payerle <payerle at umd.edu> wrote:
> 
> While I agree containers can be quite useful in HPC environments for
> dealing with applications requiring
> different library versions, there are limitations.  In particular, the
> kernel inside the container is the same
> as running outside the container.  Where this seems to be most
> problematic is when trying to use containers
> based on a much newer OS distribution than the distribution of the
> containing system.  I.e., one of our clusters
> is still running RHEL6, and while containers based on Ubuntu 16,
> Debian 8, or RHEL7 all appear to work properly,
> containers based on Ubuntu 18 or Debian 9 will die with "Kernel too
> old" errors.  (Basically, the glibc in those
> distros require a newer kernel than shipped with RHEL6).  VMs should
> not experience those issues, as the
> kernel running in the VM need not be the same kernel as running in the
> host system.
> 
> But I have found containers helpful (we use Singularity), particularly
> for applications.  Not as useful for software libraries,
> as those tend to not want to be "self-contained" and containers are
> all about "self-contained".
> 
> I am unaware of a container image for Matlab, but I suspect that is
> more due to licensing/support than technical issues.
> You could probably build a Matlab container based on some Mathworks
> supported distribution and run on a distribution
> not supported by Mathworks, but I doubt Mathworks would be willing to
> provide support for that mode of operation.
> 
> 
> 
> On Thu, Sep 19, 2019 at 6:55 AM Mahmood Naderan <mahmood.nt at gmail.com> wrote:
>> 
>> Thanks. Singularity seems to be interesting. I will try it.
>> 
>> Regards,
>> Mahmood
>> 
>> 
>> 
>> 
>> On Thu, Sep 19, 2019 at 2:49 PM Christoph Brüning <christoph.bruening at uni-wuerzburg.de> wrote:
>>> 
>>> Dear Mahmood,
>>> 
>>> Docker is somewhat tricky, because it needs a daemon running and there
>>> is no fine grained control over who is allowed to start and stop
>>> containers. Also getting the container on the node can be unpleasant
>>> (docker hub? private registry? build docker containers on the node
>>> before running them?). I would recommend against it!
>>> 
>>> However, there are projects like Singularity or Charliecloud designed to
>>> bring the "bring your own environment" idea to HPC.
>>> 
>>> We have Singularity installed, and some of our users use it. It seems to
>>> work reasonably well, as I have heard no complaint except that the
>>> available version is somewhat outdated...
>>> 
>>> Best,
>>> Christoph
>>> 
>>> 
>>> On 19/09/2019 10.08, Mahmood Naderan wrote:
>>>> Hi
>>>> The question is not directly related to Slurm, but is actually related
>>>> to the people in this community.
>>>> 
>>>> For heterogeneous environments, where different operating systems,
>>>> application and library versions are needed for HPC users, I would like
>>>> to know it using docker/containers is better than yielding virtual machines?
>>>> 
>>>> Actually, it is lighter than VM, however, I haven't seen a docker image
>>>> for Matlab for example. If that is possible, can Slurm be used to
>>>> schedule containers?
>>>> If someone has any experience using docker in HPC clusters, please let
>>>> me know.
>>>> 
>>>> 
>>>> Regards,
>>>> Mahmood
>>>> 
>>>> 
>>> 
>>> --
>>> Dr. Christoph Brüning
>>> Universität Würzburg
>>> Rechenzentrum
>>> Am Hubland
>>> D-97074 Würzburg
>>> Tel.: +49 931 31-80499
>>> 
> 
> 
> --
> Tom Payerle
> DIT-ACIGS/Mid-Atlantic Crossroads        payerle at umd.edu
> 5825 University Research Park               (301) 405-6135
> University of Maryland
> College Park, MD 20740-3831