[slurm-users] ntasks and cpus-per-task

[Christopher Benjamin Coffey]

Loris,

It’s simple, tell folks only to use -n for mpi jobs, and -c otherwise (default). 

It’s a big deal if folks use -n when it’s not an mpi program. This is because the non mpi program is launched n times (instead of once with internal threads) and will stomp over logs and output files (uncoordinated) leading to poor performance and incorrect results. 

Best,
Chris
> On Feb 22, 2018, at 1:52 AM, Loris Bennett <loris.bennett at fu-berlin.de> wrote:
> 
> Hi Chris,
> 
> Christopher Samuel <chris at csamuel.org> writes:
> 
>>> On 22/02/18 18:49, Miguel Gutiérrez Páez wrote:
>>> 
>>> What's the real meaning of ntasks? Has cpus-per-task and ntasks the
>>> same meaning in sbatch and srun?
>> 
>> --ntasks is for parallel distributed jobs, where you can run lots of
>> independent processes that collaborate using some form of communication
>> between the processes (usually MPI for HPC).
>> 
>> So inside your batch script you would use "srun" to start up the tasks.
>> 
>> However, unless you code is written to make use of that interface then
>> it's not really going to help you, and so for any multithreaded
>> application you need to use --cpus-per-task instead.
> 
> [snip (11 lines)]
> 
> But does it make any difference for a multithreaded program if I have
> 
>  #SBATCH --ntasks=4
>  #SBATCH --nodes=1-1
> 
> rather than
> 
>  #SBATCH --ntasks=1
>  #SBATCH --cpus-per-task=4
> 
> Up to now I have only thought of --cpus-per-task in connection with
> hybrid MPI/OpenMP jobs, which we don't actually have.  Thus I tend to
> tell users to think always in terms of tasks, regardless of whether
> these are MPI processes or just threads.
> 
> One downside of my approach is that if the user forgets to specify
> --nodes and --ntasks is greater than 1, non-MPI jobs can be assigned to
> multiple nodes.
> 
> Cheers,
> 
> Loris
> 
> -- 
> Dr. Loris Bennett (Mr.)
> ZEDAT, Freie Universität Berlin         Email loris.bennett at fu-berlin.de
>