<html style="direction: ltr;">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<style type="text/css">body p { margin-bottom: 0cm; margin-top: 0pt; } </style>
</head>
<body bidimailui-charset-is-forced="true" style="direction: ltr;"
text="#000000" bgcolor="#FFFFFF">
<p>I think i found the problem, but I dont have any idea how to
solve.</p>
<p><br>
</p>
<p><b>scenario1</b>: pbis was never installed (adopt succeed)<br>
</p>
<p><br>
</p>
<p>login server$ srun -c1 --pty bash</p>
<p><br>
</p>
<p>compA$ ssh slurm-node</p>
<p>slurm-node$ nproc</p>
<p>1</p>
<p><br>
</p>
<p>auth.log on slurm-node:<br>
</p>
<p><br>
Mar 7 08:54:29 slurm-node pam_slurm_adopt[<b>7329</b>]:
Connection by user temp: user has only one job 5<br>
Mar 7 08:54:29 slurm-node pam_slurm_adopt[<b>7329</b>]: Process
7329 adopted into job 5<br>
Mar 7 08:54:29 slurm-node sshd[<b>7329</b>]: Accepted password
for temp from IP_HERE port 57052 ssh2<br>
Mar 7 08:54:29 slurm-node sshd[<b>7329</b>]:
pam_unix(sshd:session): session opened for user temp by (uid=0)</p>
<p><br>
</p>
<p>I bolded process number, it consistent.<br>
</p>
<p><br>
</p>
<p><b>sceanrio2</b>: pbis installed (adopt failed)<br>
</p>
<p><br>
</p>
<p>login server$ srun -c1 --pty bash
</p>
<p><br>
</p>
<p>compA$ ssh slurm-node</p>
<p>slurm-node$ nproc</p>
<p>2<br>
</p>
<p><br>
</p>
<p>*two is the total amount of cpus on slurm-node<br>
</p>
<p><br>
</p>
<p>auth.log on slurm-node:<br>
</p>
<p><br>
Mar 7 09:00:52 slurm-node pam_slurm_adopt[<b>1595</b>]:
Connection by user temp: user has only one job 8<br>
Mar 7 09:00:53 slurm-node pam_slurm_adopt[<b>1595</b>]: Process
1595 adopted into job 8<br>
Mar 7 09:00:53 slurm-node sshd[<b>1593</b>]: Accepted
keyboard-interactive/pam for temp from IP_HERE port 33218 ssh2<br>
Mar 7 09:00:53 slurm-node sshd[<b>1593</b>]:
pam_unix(sshd:session): session opened for user temp by (uid=0)<br>
</p>
<p><br>
</p>
<p>here the process number changed! the adoption is for one process
and eventually we successfully getting ssh access but with
different process number and context.</p>
<p><br>
</p>
<p>ps -ef |grep 1595</p>
<p><b><u>no output</u></b></p>
<p><br>
</p>
<p>ps -ef |grep 1593</p>
<p>root 1593 1093 0 14:57 ? 00:00:00 sshd: temp [priv]<br>
temp 1627 1593 0 14:57 ? 00:00:00 sshd: temp@pts/2<br>
<br>
</p>
<p><br>
</p>
<p><u>Notes:</u></p>
<p>sceanrio2 haven't changed when i tried:</p>
<p>a. stopping pbis service (lwsmd)</p>
<p>b. restore all pam.d files to scenario1 state</p>
<p>c. sudo apt purge pbis-open & reboot didnt help</p>
<p><br>
</p>
<p>My conclusion is that pbis changed something in the way linux pam
works but i can't figure out where<br>
</p>
<p><br>
</p>
<p><br>
</p>
<p>If anyone got an idea, will be glad to hear.</p>
<p><br>
</p>
<p><br>
</p>
<div class="moz-cite-prefix">On 2/24/19 9:22 AM, נדב טולדו wrote:<br>
</div>
<blockquote type="cite"
cite="mid:52f4919e-0613-b3fc-1300-4aa965c759c8@cs.technion.ac.il">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<style type="text/css">body p { margin-bottom: 0cm; margin-top: 0pt; } </style>
<div class="moz-cite-prefix">
<div>Thanks to both of you, I will try and let you know.<br>
<hr id="rwhMsgHdrDivider" style="border:0;border-top:1px solid
#B5C4DF;padding:0;margin:10px 0 5px 0;width:100%;"><span
style="margin: -1.3px 0 0 0 !important;"><font style="font:
13px Tahoma !important; color: #000000 !important;"
face="Tahoma" color="#000000"><b>From:</b> Prentice Bisbal</font></span><br>
<span style="margin: -1.3px 0 0 0 !important;"><font
style="font: 13px Tahoma !important; color: #000000
!important;" face="Tahoma" color="#000000"><b>Sent:</b>
Fri, Feb 22, 2019 6:16 PM IST</font></span><br>
<span style="margin: -1.3px 0 0 0 !important;"><font
style="font: 13px Tahoma !important; color: #000000
!important;" face="Tahoma" color="#000000"><b>To:</b> <a
class="moz-txt-link-abbreviated"
href="mailto:slurm-users@lists.schedmd.com"
moz-do-not-send="true">slurm-users@lists.schedmd.com</a></font></span><br>
<span style="margin: -1.3px 0 0 0 !important;"><font
style="font: 13px Tahoma !important; color: #000000
!important;" face="Tahoma" color="#000000"><b>Subject:</b>
[slurm-users] pam_slurm_adopt with pbis-open pam modules</font></span><br>
<br>
</div>
</div>
<blockquote type="cite"
cite="mid:85bde308-e5e8-1515-7bca-d0a067b88e26@pppl.gov"
style="border:none !important; margin-left:0px !important;
margin-right:0px !important; margin-top:0px !important;
padding-left:0px !important; padding-right:0px !important">
<p><br>
</p>
<div class="moz-cite-prefix">On 2/22/19 12:54 AM, Chris Samuel
wrote:<br>
</div>
<blockquote type="cite" cite="mid:3627066.BFLgRgZ6ST@quad"
style="border:none !important; margin-left:0px !important;
margin-right:0px !important; margin-top:0px !important;
padding-left:0px !important; padding-right:0px !important">
<pre class="moz-quote-pre" wrap="">On Thursday, 21 February 2019 8:20:36 AM PST נדב טולדו wrote:
</pre>
<blockquote type="cite" style="border:none !important;
margin-left:0px !important; margin-right:0px !important;
margin-top:0px !important; padding-left:0px !important;
padding-right:0px !important">
<pre class="moz-quote-pre" wrap="">Yeah I have, before i installed pbis and introduce lsass.so the slurm module
worked well Is there anyway to debug?
I am seeing in syslog that the slurm module is adopting into the job context
but then i am getting out of context somehow and have access to all
resources.
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">Yes, check the documentation and review your PAM configuration. As I
mentioned it sounds like you've got things in the wrong order there.
<a class="moz-txt-link-freetext" href="https://slurm.schedmd.com/pam_slurm_adopt.html#PAM_CONFIG" moz-do-not-send="true">https://slurm.schedmd.com/pam_slurm_adopt.html#PAM_CONFIG</a>
</pre>
</blockquote>
<p>I second this. PAM is extremely sensitive to the module order
by design. <br>
</p>
<p>Also, to debug, most PAM modules have a debug option you can
use to enable the logging of debug messages. If you check the
man pages for any pam modules, you'll see the debug options.
For pam_slurm_adopt, see <a class="moz-txt-link-freetext"
href="https://slurm.schedmd.com/pam_slurm_adopt.html"
moz-do-not-send="true">https://slurm.schedmd.com/pam_slurm_adopt.html</a>.
It looks like you can set a log_level setting: <br>
</p>
<p> </p>
<blockquote type="cite" style="border:none !important;
margin-left:0px !important; margin-right:0px !important;
margin-top:0px !important; padding-left:0px !important;
padding-right:0px !important">
<dl compact="compact">
<dt><b>log_level</b></dt>
<dd>See <a
href="https://slurm.schedmd.com/slurm.conf.html#OPT_SlurmdDebug"
moz-do-not-send="true"> SlurmdDebug</a> in slurm.conf
for available options. The default log_level is <b>info</b>.
</dd>
</dl>
</blockquote>
<p><br>
</p>
<p>So to set the debugging level for pam_slurm_adopt, all the
way up, you'd do something like this in your PAM file: <br>
</p>
<pre>account sufficient pam_slurm_adopt.so debug=debug5
</pre>
<p>If you can't tell what's going on just from that, I would see
how to enable debugging for all the PAM modules in the rest of
the stack, to get a better picture of what's going on
throughout the whole authentication process. When your done,
don't forget to turn off logging so you don't fill your log
files with unnecessary noise. <br>
</p>
<p><br>
</p>
<p>--</p>
<p>Prentice<br>
</p>
<p><br>
</p>
<br>
</blockquote>
</blockquote>
</body>
</html>