[slurm-users] Heterogeneous HPC

[Thomas M. Payerle]

While I agree containers can be quite useful in HPC environments for
dealing with applications requiring
different library versions, there are limitations.  In particular, the
kernel inside the container is the same
as running outside the container.  Where this seems to be most
problematic is when trying to use containers
based on a much newer OS distribution than the distribution of the
containing system.  I.e., one of our clusters
is still running RHEL6, and while containers based on Ubuntu 16,
Debian 8, or RHEL7 all appear to work properly,
containers based on Ubuntu 18 or Debian 9 will die with "Kernel too
old" errors.  (Basically, the glibc in those
distros require a newer kernel than shipped with RHEL6).  VMs should
not experience those issues, as the
kernel running in the VM need not be the same kernel as running in the
host system.

But I have found containers helpful (we use Singularity), particularly
for applications.  Not as useful for software libraries,
as those tend to not want to be "self-contained" and containers are
all about "self-contained".

I am unaware of a container image for Matlab, but I suspect that is
more due to licensing/support than technical issues.
You could probably build a Matlab container based on some Mathworks
supported distribution and run on a distribution
not supported by Mathworks, but I doubt Mathworks would be willing to
provide support for that mode of operation.



On Thu, Sep 19, 2019 at 6:55 AM Mahmood Naderan <mahmood.nt at gmail.com> wrote:
>
> Thanks. Singularity seems to be interesting. I will try it.
>
> Regards,
> Mahmood
>
>
>
>
> On Thu, Sep 19, 2019 at 2:49 PM Christoph Brüning <christoph.bruening at uni-wuerzburg.de> wrote:
>>
>> Dear Mahmood,
>>
>> Docker is somewhat tricky, because it needs a daemon running and there
>> is no fine grained control over who is allowed to start and stop
>> containers. Also getting the container on the node can be unpleasant
>> (docker hub? private registry? build docker containers on the node
>> before running them?). I would recommend against it!
>>
>> However, there are projects like Singularity or Charliecloud designed to
>> bring the "bring your own environment" idea to HPC.
>>
>> We have Singularity installed, and some of our users use it. It seems to
>> work reasonably well, as I have heard no complaint except that the
>> available version is somewhat outdated...
>>
>> Best,
>> Christoph
>>
>>
>> On 19/09/2019 10.08, Mahmood Naderan wrote:
>> > Hi
>> > The question is not directly related to Slurm, but is actually related
>> > to the people in this community.
>> >
>> > For heterogeneous environments, where different operating systems,
>> > application and library versions are needed for HPC users, I would like
>> > to know it using docker/containers is better than yielding virtual machines?
>> >
>> > Actually, it is lighter than VM, however, I haven't seen a docker image
>> > for Matlab for example. If that is possible, can Slurm be used to
>> > schedule containers?
>> > If someone has any experience using docker in HPC clusters, please let
>> > me know.
>> >
>> >
>> > Regards,
>> > Mahmood
>> >
>> >
>>
>> --
>> Dr. Christoph Brüning
>> Universität Würzburg
>> Rechenzentrum
>> Am Hubland
>> D-97074 Würzburg
>> Tel.: +49 931 31-80499
>>


-- 
Tom Payerle
DIT-ACIGS/Mid-Atlantic Crossroads        payerle at umd.edu
5825 University Research Park               (301) 405-6135
University of Maryland
College Park, MD 20740-3831