Hi Paul, Just my 2 cents: Are you running the latest Slurm version? BR Ole On 5/13/2026 6:48 PM, Paul Raines via slurm-users wrote:
In my cluster right now we have:
$ sinfo ... pubgpu-req up 7-00:00:00 7 mix cobra,l40s- [01,03],luisa,rtx-03,sevilla,shelob pubgpu-req up 7-00:00:00 5 idle fiber,glaurung,gothmog,l40s-02,leo
The following works fine:
$ srun -p pubgpu-req -A sysadm --nodelist=cobra,l40s-02 --gres=gpu:1 -N 1 \ --ntasks-per-node=1 --mem=1G --time=1:00:00 --cpus-per-task=4 --pty / bin/bash srun: tres_per_node => gres/gpu:1 cobra[0]:~$ exit exit
However just change the order of the nodelist and you get
$ srun -p pubgpu-req -A sysadm --nodelist=l40s-02,cobra --gres=gpu:1 -N 1 \ --ntasks-per-node=1 --mem=1G --time=1:00:00 --cpus-per-task=4 --pty / bin/bash srun: tres_per_node => gres/gpu:1 srun: error: Unable to create step for job 8255390: Requested node configuration is not available
More experimentation with this and it appears nodelist HAS to be in alphabetical order
$ srun -p pubgpu-req -A sysadm --nodelist=l40s-02,leo,fiber \ --gres=gpu:1 -N 1 --ntasks-per-node=1 --mem=1G --time=1:00:00 \ --cpus- per-task=4 --pty /bin/bash srun: tres_per_node => gres/gpu:1 srun: error: Unable to create step for job 8255401: Requested node configuration is not available
$ srun -p pubgpu-req -A sysadm --nodelist=fiber,l40s-02,leo \ --gres=gpu:1 -N 1 --ntasks-per-node=1 --mem=1G --time=1:00:00 \ --cpus- per-task=4 --pty /bin/bash srun: tres_per_node => gres/gpu:1 fiber[0]:~$
Surely there is no good reason for this?