<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"><!-- P {margin-top:0;margin-bottom:0;} --></style>
</head>
<body dir="ltr">
<div id="divtagdefaultwrapper" style="font-size:12pt;color:#000000;font-family:Calibri,Helvetica,sans-serif;" dir="ltr">
<p style="margin-top:0;margin-bottom:0"></p>
<div>Our site has been going through the process of upgrading SLURM on our primary cluster which was delivered to us with Slurm 16.05 with Bright Computing. We're currently at 17.02.13-2 and working to get to 17.11 and then 18.08. We've run into an issue
with 17.11 and switching effective GID on a sbatch/srun. I've only found one mention of this issue in the archive and no specific resolution:</div>
<div><br>
</div>
<div> https://groups.google.com/forum/#!topic/slurm-users/YZlTqBoMZ0o</div>
<div><br>
</div>
<div>Our site has a "many projects -> single user" mapping. So a given user is likely to be in 3+ projects, which map to corresponding SLURM accounts in sacctmgr. For each SLURM account, we create a corresponding POSSIX/UNIX group of the same name and setup
a directory on our GPFS storage appropriately owned by that group, with a disk quota.</div>
<div><br>
</div>
<div>We made the switch to Slurm from Torque a few year back. In Torque we where using "-W group_list=" option to allow the user to change their effective GID to one of their auxiliary groups on a per job basis. <span style="font-size: 12pt;">In 17.02 and
earlier, we've been using the --gid= option to similar effect, allowing users to switch their effective GID for a given job </span><span style="font-size: 12pt;">to one of their
</span>Auxiliary<span style="font-size: 12pt;"> groups that matches the project they are burning time for.</span></div>
<div><br>
</div>
<div>On 17.02 the following works fine:</div>
<div><br>
</div>
<div>[login1] $ id</div>
<div>uid=1000(user1) gid=1000(users) groups=1000(users),1001(test)</div>
<div><br>
</div>
<div>[login1] $ srun --account=general --pty /bin/bash -i</div>
<div>[compute1] $ id --group</div>
<div>1000</div>
<div><br>
</div>
<div>[login1] $ srun --account=test --gid=test --pty /bin/bash -i</div>
<div>[compute1] $ id --group</div>
<div>1001</div>
<div><br>
</div>
<div>On 17.11, using --gid gets an error:</div>
<div><br>
</div>
<div>[login1] $ srun --account=test --gid=test --pty /bin/bash -i</div>
<div>srun: error: --gid only permitted by root user</div>
<div><br>
</div>
<div>The only work around I've found that mimics the same behavior is to use "newgrp" or "sg" on the login node, to switch the auxiliary group to be the effective group during submit:</div>
<div><br>
</div>
<div>[login1] $ sg test 'srun --account=test --pty /bin/bash -i'</div>
<div>[compute1] $ id --group</div>
<div>1001</div>
<div><br>
</div>
<div>I've reviewed the slurm-users archive, bug notes, etc and understand the reason the change was made to disallow --uid/--gid except for root. What I am looking for is information/suggestions on the best way to mimic the 17.02 and earlier functionality
in a secure way.</div>
<div><br>
</div>
<div>I've already attempted to write a JobSubmit plugin and in the "<span>extern int job_submit" function overwrite <span>job_desc->group_id to use an alternate group ID based on <span>job_desc->account</span>. But that resulted in an error on the slurmd side:</span></span></div>
<div><span><span><br>
</span></span></div>
<div><span><span>
<div>[2019-07-16T13:26:20.073] error: job 95 credential created for gid 1001, expected 1000</div>
<div>[2019-07-16T13:26:20.073] error: Invalid job credential from 1000@172.20.2.2: Invalid job credential</div>
<div><br>
</div>
<div>I then attempted a spank plugin to add my own option, "--egid", to srun/sbatch and attempting to overwrite the GID that slurm picked up from the login node. I was able to bring in "--egid=test", resolve the group name to a GID number, but no matter <span style="font-size: 12pt;">which </span><span style="font-size: 12pt;">slurm_spank_*
function I tried, using the "setegid" or "setgid" system calls didn't hold. Once the actual slurmstepd process started, the effective GID in the user process was 1000 (the GID used when sbatch/srun was run) instead of 1001.</span></div>
<div><span style="font-size: 12pt;"><br>
</span></div>
<div><span style="font-size: 12pt;">I've been hoping there is something I missed either native to SLURM or in the JobSubmit/SPANK plugin that would let me have the ability to allow users to switch their effective GID on a per job basis to any of the groups
they belong too.</span><span style="font-size: 12pt;"></span></div>
<div><span style="font-size: 12pt;"><br>
</span></div>
<div><span style="font-size: 12pt;">Thanks,</span><br>
</div>
</span></span></div>
<div> -Brad Viviano</div>
<br>
<p></p>
<p style="margin-top:0;margin-bottom:0"><br>
</p>
<div id="Signature">
<meta content="text/html; charset=UTF-8">
<div dir="ltr" id="divtagdefaultwrapper" style="background-color:rgb(255,255,255)">
<div name="divtagdefaultwrapper" style="margin:0px">
<div style="color:rgb(0,0,0); font-family:Tahoma; font-size:13px"><font face="Tahoma" size="1"><span style="font-family:Arial,Helvetica,sans-serif; font-size:8pt">===================================================</span></font></div>
<div style="color:rgb(0,0,0); font-family:Tahoma; font-size:13px"><font face="Tahoma" size="1"><span style="font-family:Arial,Helvetica,sans-serif; font-size:8pt"></span></font><span style="font-size:13.3333px; color:rgb(31,56,100); font-family:Arial,sans-serif,serif,EmojiFont">Brad
Viviano</span><font face="Tahoma" size="1"><br>
</font></div>
<div style=""><font style=""><span class="gc-cs-link" title="Call with Google Voice" style=""><font style="">
<p class="x_MsoNormal" style="font-family:Calibri,sans-serif; color:rgb(33,33,33); font-size:11pt; margin:0in 0in 0.0001pt">
<span style="font-size:8pt; font-family:Arial,sans-serif,serif,EmojiFont; color:rgb(31,56,100)">Senior Systems Engineer</span></p>
<p class="x_MsoNormal" style="font-family:Calibri,sans-serif; color:rgb(33,33,33); font-size:11pt; margin:0in 0in 0.0001pt">
<br>
</p>
<p class="x_MsoNormal" style="font-family:Calibri,sans-serif; color:rgb(33,33,33); font-size:11pt; margin:0in 0in 0.0001pt">
<span style="font-size:8pt; font-family:Arial,sans-serif,serif,EmojiFont"><span style="color:rgb(31,73,125)"></span></span></p>
</font></span></font></div>
</div>
</div>
</div>
</div>
</body>
</html>