<html><head><meta http-equiv="Content-Type" content="text/html; charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;" class="">On a cluster running Slurm 17.11.8 (cons_res) I can submit a job that requests e.g. 2 nodes with unique features on each:<div class=""><br class=""></div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class="">$ sbatch --nodes=2 --ntasks-per-node=1 --constraint="[256GB*1&192GB*1]" …</div></blockquote><br class=""><div class=""><br class=""></div><div class="">The job is submitted and runs as expected: on 1 node with feature "256GB" and 1 node with feature "192GB." A similar job on a cluster running 20.11.1 (cons_res OR cons_tres, tested with both) fails to submit:</div><div class=""><br class=""></div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class="">sbatch: error: Batch job submission failed: Requested node configuration is not available</div></blockquote><br class=""><div class=""><br class=""></div><div class="">I enabled debug5 output with NodeFeatures:</div><div class=""><br class=""></div><div class=""><br class=""></div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class="">[2020-12-16T08:53:19.024] debug: JobId=118 feature list: [512GB*1&768GB*1]</div><div class="">[2020-12-16T08:53:19.025] NODE_FEATURES: _log_feature_nodes: FEAT:512GB COUNT:1 PAREN:0 OP:XAND ACTIVE:r1n[00-47] AVAIL:r1n[00-47]</div><div class="">[2020-12-16T08:53:19.025] NODE_FEATURES: _log_feature_nodes: FEAT:768GB COUNT:1 PAREN:0 OP:END ACTIVE:r2l[00-31] AVAIL:r2l[00-31]</div><div class="">[2020-12-16T08:53:19.025] NODE_FEATURES: valid_feature_counts: feature:512GB feature_bitmap:r1n[00-47],r2l[00-31],r2x[00-10] work_bitmap:r1n[00-47],r2l[00-31],r2x[00-10] tmp_bitmap:r1n[00-47] count:1</div><div class="">[2020-12-16T08:53:19.025] NODE_FEATURES: valid_feature_counts: feature:768GB feature_bitmap:r1n[00-47],r2l[00-31],r2x[00-10] work_bitmap:r1n[00-47],r2l[00-31],r2x[00-10] tmp_bitmap:r2l[00-31] count:1</div><div class="">[2020-12-16T08:53:19.025] NODE_FEATURES: valid_feature_counts: NODES:r1n[00-47],r2l[00-31],r2x[00-10] HAS_XOR:T status:No error</div>[2020-12-16T08:53:19.025] select/cons_tres: _job_test: SELECT_TYPE: test 0 pass: test_only<div class="">[2020-12-16T08:53:19.026] debug2: job_allocate: setting JobId=118_* to "BadConstraints" due to a flaw in the job request (Requested node configuration is not available)</div><div class="">[2020-12-16T08:53:19.026] _slurm_rpc_submit_batch_job: Requested node configuration is not available</div></blockquote><div class=""><br class=""></div><div class=""><br class=""></div><div class="">My syntax agrees with the 20.11.1 documentation (online and man pages) so it seems correct — and it works fine in 17.11.8. Any ideas?</div><br class=""><br class=""><br class=""><div class="">
<div style="color: rgb(0, 0, 0); letter-spacing: normal; orphans: auto; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">::::::::::::::::::::::::::::::::::::::::::::::::::::::<br class=""> Jeffrey T. Frey, Ph.D.<br class=""> Systems Programmer V / Cluster Management<br class=""> Network & Systems Services / College of Engineering<br class=""> University of Delaware, Newark DE 19716<br class=""> Office: (302) 831-6034 Mobile: (302) 419-4976<br class="">::::::::::::::::::::::::::::::::::::::::::::::::::::::</div>
</div>
<br class=""></body></html>