<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
</head>
<body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">
We have a heterogeneous mix of nodes, most 32 core, but one group of 36 core, grouped into homogeneous partitions. We like to be able to specify multiple partitions so that a job can run on any homogeneous group. It would be nice if we could run on all such
nodes using 32 cores per node. To try to do this, I created a partition for the 36-core nodes (call them n2019) which specifies a max cpu # of 64
<blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class="">
<div class="">PartitionName=n2019 DefMemPerCPU=2631 Nodes=compute-4-[0-47]<br class="">
PartitionName=n2019_32 DefMemPerCPU=2631 Nodes=compute-4-[0-47] MaxCPUsPerNode=64 <br class="">
PartitionName=n2021 DefMemPerCPU=2960 Nodes=compute-7-[0-18]<br class="">
</div>
</blockquote>
<div class=""><br class="">
</div>
However, if I try to run a 128 task, 1 task per core job on n2019_32, the sbatch fails with
<blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class="">
<div class="">> sbatch --ntasks=128 --exclusive --partition=n2019_32 --ntasks-per-core=1 job.pbs</div>
</blockquote>
<div class="">
<blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class="">
<div class="">sbatch: error: Batch job submission failed: Requested node configuration is not available<br class="">
</div>
</blockquote>
<div class="">(please ignore the ".pbs" - it's a relic, and the job script works with slurm). The identical command but with "n2019" or "n2021" for the partition works (but the former uses 36 cores per node). If I specify multiple partitions it will only actually
run when the non-n2019 (same node set as n2019_32) nodes are available.</div>
<div class=""><br class="">
</div>
The job header includes only walltime, job name and stdout/stderr files, shell, and a job array range.</div>
<div class=""><br class="">
</div>
<div class="">I tried to add "-v" to the sbatch to see if that gives more useful info, but I couldn't get any more insight. Does anyone have any idea why it's rejecting my job?</div>
<div class=""><br class="">
</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>thanks,</div>
<div class=""><span class="Apple-tab-span" style="white-space:pre"></span>Noam</div>
</body>
</html>