[slurm-users] Heterogeneous HPC

[Michael Jennings]

On Friday, 20 September 2019, at 00:03:28 (+0430),
Mahmood Naderan wrote:

> For the replies. Matlab was an example. I would also like to create
> to containers for OpenFoam with different versions. Then a user can
> choose what he actually wants.

All modern container runtimes support the OCI standard container
format originally authored by Docker, Inc. and contributed to the Open
Container Initiative (OCI) as the starting point for their standard.
So your best bet would be to go to Docker Hub (hub.docker.com) and
search for the applications you're interested in, or (in the case of
commercial software) ask your vendor if they supply containers for
their packages and under what terms.

If you're comfortable with building as root, you can likely build your
own containers without too much trouble, but in order to build
containers without privilege, you'll need very recent Podman/Buildah
(or current Charliecloud plus Spokeo and umoci, if your Dockerfile is
supported by ch-grow).

> I would also like to know, if the technologies you mentioned can be
> deployed in multinode clusters. Currently, we use Rocks 7. Should I
> install singularity (or others) on all nodes or just the frontend?
> And then, can users use "srun" or "salloc" for interactively login
> to a node and run the container or not?

Most folks invoke the container runtime using srun, either in their
job script or as part of an interactive session.  There are several
examples in the Charliecloud docs, for example, here:
https://hpc.github.io/charliecloud/tutorial.html#your-first-single-node-multi-process-jobs

But yes, you will likely need the container runtime installed on every
node.  Most large HPC centers use Slurm, so you should have no problem
getting any or all of them to integrate well with your existing Slurm
installation. :-)

That said, I *do* recommend watching at least that last video before
you make your final decision on runtime.  With containers, as with any
technology, you're far more likely to get factual information from
folks who aren't trying to sell something! ;-)

Having personally deployed, tested, and evaluated over a dozen
different container solutions -- including every major HPC container
system as well as implementing a few of my own -- I can tell you with
absolute certainty that there's no single right answer to "What
container system should I use?"  There are several correct answers
depending on your use case and security & UX requirements.

Michael

-- 
Michael E. Jennings <mej at lanl.gov>
HPC Systems Team, Los Alamos National Laboratory
Bldg. 03-2327, Rm. 2341     W: +1 (505) 606-0605