[slurm-users] Multiple Program Runs using srun in one Slurm batch Job on one node

[Frank Lenaerts]

On Wed, Jun 15, 2022 at 02:20:56PM +0200, Guillaume De Nayer wrote:
> One collegue has to run 20,000 jobs on this machine. Every job starts
> his program with mpirun on 12 cores. The standard slurm behavior makes
> that the node, which runs this job is blocked (and 28 cores are idle).
> The small cluster has only 8 nodes, so only 8 jobs can run in parallel.

If your colleague also uses sbatch(1)'s --exclusive option, only one
job can run on a node...

> In order to solve this problem I'm trying to start some subtasks with
> srun inside a batch job (without mpirun for now):
> 
> #!/bin/bash
> #SBATCH --job-name=test_multi_prog_srun
> #SBATCH --nodes=1
> #SBATCH --partition=short
> #SBATCH --time=02:00:00
> #SBATCH --exclusive

> Regards,

Kind regards

> Guillaume

Frank