<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div>Hi all,</div>
<div><br>
</div>
<div>I'm trying to use heterogeneous jobs with the following slurm script:</div>
<div><br>
</div>
<pre>#!/usr/bin/env bash</pre>
<pre>#SBATCH --partition=cpu --time=01:00:00 --nodes=2 --ntasks-per-node=1 --cpus-per-task=2 --mem=8G</pre>
<pre>#SBATCH hetjob</pre>
<pre>#SBATCH --partition=gpu --time=01:00:00 --nodes=2 --ntasks-per-node=1 --cpus-per-task=2 --mem=8G --gres=gpu:1</pre>
<pre><br></pre>
<pre>srun \</pre>
<pre> --het-group=0 -K sh -c 'echo group 0 $(hostname) $SLURM_PROCID' : \</pre>
<pre> --het-group=1 -K sh -c 'echo group 1 $(hostname) $SLURM_PROCID'</pre>
<div><br>
</div>
<div>
<div><span style="font-size: 14.666667px;">It works when I manually run the commands via salloc, but it fails via sbatch with the following message:</span></div>
</div>
<div><br>
</div>
<pre>srun: error: Allocation failure of 2 nodes: job size of 2, already allocated 2 nodes to previous components.</pre>
<pre>srun: error: Allocation failure of 2 nodes: job size of 2, already allocated 2 nodes to previous components.</pre>
<div><br>
</div>
<div>Am I misunderstanding the sbatch documentation? Is it normal that sbatch and salloc behave differently?</div>
<div><br>
</div>
<div>Note: with salloc the job script runs on the slurmctld server whereas with sbatch it runs on the first node allocated to the batch. Slurm is in version 20.11.3.</div>
<div><br>
</div>
<div>Best regards,</div>
<div>Nicolas</div>
<div><span></span></div>
</body>
</html>