[slurm-users] ntasks and cpus-per-task

[Mjelde, Matthew J]

Howdy,

Yes there is a difference between those two submissions.  You are correct that usually ntasks is for mpi and cpus-per-task is for multithreading, but let’s look at your commands.

For your first example, the “sbatch --ntasks 24 […]”, this will allocated a job with 24 tasks.  These tasks in this case are only 1 CPUs, but may be split across multiple nodes. So you get a total of 24 CPUs across multiple nodes.
For your second example, the “sbatch --ntasks 1 --cpus-per-task 24 [...]”, this will allocated a job with 1 task and 24 CPUs for that task.  So you will get a total of 24 CPUs on a single node.

So in other words, a task cannot be split across multiple nodes.  So using --cpus-per-task will ensure it gets allocated to the same node, while using --ntasks can and may allocate it to multiple nodes.

Hope this helps.
Matthew

From: slurm-users [mailto:slurm-users-bounces at lists.schedmd.com] On Behalf Of Miguel Gutiérrez Páez
Sent: Thursday, February 22, 2018 3:07 AM
To: Slurm User Community List <slurm-users at lists.schedmd.com>
Subject: Re: [slurm-users] ntasks and cpus-per-task

Hi,
That was just I thought, ntasks for mpi and cpus-per-task for multithreading.
So, for example, if every node has 24 cores, is there any difference between these commands?
sbatch --ntasks 24 [...]
sbatch --ntasks 1 --cpus-per-task 24 [...]
regards.

El jue., 22 feb. 2018 a las 9:52, Loris Bennett (<loris.bennett at fu-berlin.de<mailto:loris.bennett at fu-berlin.de>>) escribió:
Hi Chris,

Christopher Samuel <chris at csamuel.org<mailto:chris at csamuel.org>> writes:

> On 22/02/18 18:49, Miguel Gutiérrez Páez wrote:
>
>> What's the real meaning of ntasks? Has cpus-per-task and ntasks the
>> same meaning in sbatch and srun?
>
> --ntasks is for parallel distributed jobs, where you can run lots of
> independent processes that collaborate using some form of communication
> between the processes (usually MPI for HPC).
>
> So inside your batch script you would use "srun" to start up the tasks.
>
> However, unless you code is written to make use of that interface then
> it's not really going to help you, and so for any multithreaded
> application you need to use --cpus-per-task instead.

[snip (11 lines)]

But does it make any difference for a multithreaded program if I have

  #SBATCH --ntasks=4
  #SBATCH --nodes=1-1

rather than

  #SBATCH --ntasks=1
  #SBATCH --cpus-per-task=4

Up to now I have only thought of --cpus-per-task in connection with
hybrid MPI/OpenMP jobs, which we don't actually have.  Thus I tend to
tell users to think always in terms of tasks, regardless of whether
these are MPI processes or just threads.

One downside of my approach is that if the user forgets to specify
--nodes and --ntasks is greater than 1, non-MPI jobs can be assigned to
multiple nodes.

Cheers,

Loris

--
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universität Berlin         Email loris.bennett at fu-berlin.de<mailto:loris.bennett at fu-berlin.de>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20180222/1f26419e/attachment-0001.html>