<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.msonormal0, li.msonormal0, div.msonormal0
{mso-style-name:msonormal;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-GB" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Ah! Thanks – that’s what I’d been missing.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">So the answer then is to either kludge the keys by making symlinks to the cluster and cluster.pub files warewulf makes (I tried this
already and I know it works), or to update to the v19.x release and the new style x11 forwarding.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Is the update to v19 fairly straightforward? Is it stable enough at this point to just give it a try?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Thanks<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US">Simon.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:EN-US"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span lang="EN-US" style="font-size:11.0pt;font-family:"Calibri",sans-serif"> slurm-users <slurm-users-bounces@lists.schedmd.com>
<b>On Behalf Of </b>Jeffrey T Frey<br>
<b>Sent:</b> 27 January 2020 14:39<br>
<b>To:</b> Slurm User Community List <slurm-users@lists.schedmd.com><br>
<b>Subject:</b> Re: [slurm-users] Srun not setting DISPLAY with --x11 for one account<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">The Slurm-native X11 plugin demands you use ~/.ssh/id_rsa{,.pub} keys. It's hard-coded into the plugin:<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<blockquote style="margin-left:30.0pt;margin-right:0cm">
<div>
<p class="MsoNormal">/*<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> * Ideally these would be selected at run time. Unfortunately,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> * only ssh-rsa and ssh-dss are supported by libssh2 at this time,<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> * and ssh-dss is deprecated.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"> */<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">static char *hostkey_priv = "/etc/ssh/ssh_host_rsa_key";<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">static char *hostkey_pub = "/etc/ssh/ssh_host_rsa_key.pub";<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">static char *priv_format = "%s/.ssh/id_rsa";<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal">static char *pub_format = "%s/.ssh/id_rsa.pub";<o:p></o:p></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><br>
<br>
<br>
<o:p></o:p></p>
<blockquote style="margin-top:5.0pt;margin-bottom:5.0pt">
<p class="MsoNormal">On Jan 27, 2020, at 09:34 , Simon Andrews <<a href="mailto:simon.andrews@babraham.ac.uk">simon.andrews@babraham.ac.uk</a>> wrote:<br>
<br>
I’ve managed to track down the difference between the accounts which work and those which don’t – but I still don’t understand the mechanism.<br>
<br>
The accounts which work all had their home directories used on an older system. The ones which fail were only ever used on the new system. The relevant difference seems to be the way their ssh keys are set up. On the old system a standard ssh-keygen was
run, creating ~/.ssh/id_rsa and ~/.ssh/id_rsa.pub files and putting the pub file into authorized_keys.<br>
<br>
On the new warewulf based system ssh-keygen was again run, but the default key file names was changed. We now have ~/.ssh/cluster and ~/.ssh/cluster.pub and there is a ~/.ssh/config file which contains:<br>
<br>
# Added by Warewulf 2019-12-10<br>
Host pebble*<br>
IdentityFile ~/.ssh/cluster<br>
StrictHostKeyChecking=no<br>
<br>
This all works fine, and I can ssh from the head node to the ‘pebble’ compute nodes just fine, however something in the code for the slurm x11 forwarder is specifically looking for id_rsa files (or is ignoring the config file), since the forwarding fails if
I don’t have these, and works as soon as I do.<br>
<br>
Any ideas where this might be happening so I can either file a bug for change whatever setting this needs?<br>
<br>
Simon.<br>
<br>
From: slurm-users <<a href="mailto:slurm-users-bounces@lists.schedmd.com">slurm-users-bounces@lists.schedmd.com</a>> On Behalf Of William Brown<br>
Sent: 24 January 2020 17:21<br>
To: Slurm User Community List <<a href="mailto:slurm-users@lists.schedmd.com">slurm-users@lists.schedmd.com</a>><br>
Subject: Re: [slurm-users] Srun not setting DISPLAY with --x11 for one account<br>
<br>
There are differences for X11 between Slurm versions so it may help to know which version you have.<br>
<br>
I tried some of your commands on our slurm 19.05.3-2 cluster, and interestingly on the session on the compute node I don't see the cookie for the login node: This was with MobaXterm:<br>
<br>
[user@prdubrvm005 ~]$ xauth list<br>
<a href="http://prdubrvm005.research.rcsi.com/unix:10">prdubrvm005.research.rcsi.com/unix:10</a> MIT-MAGIC-COOKIE-1 2efc5dd851736e3848193f65d038eca8<br>
[user@prdubrvm005 ~]$ srun --pty --x11 --preserve-env /bin/bash<br>
[user@prdubrhpc1-02 ~]$ xauth list<br>
prdubrhpc1-02.research.rcsi.com/unix:95 MIT-MAGIC-COOKIE-1 2efc5dd851736e3848193f65d038eca8<br>
[user@prdubrhpc1-02 ~]$ echo $DISPLAY<br>
localhost:95.0<br>
<br>
Any per-user problem would make me suspect the user having a different shell, or something in their login script. Can you make their .bashrc and .bash_profile just exit? Or look for hidden configuration files for <something> in their home directory?<br>
<br>
William<br>
<br>
<br>
<br>
On Fri, 24 Jan 2020 at 16:05, Simon Andrews <<a href="mailto:simon.andrews@babraham.ac.uk">simon.andrews@babraham.ac.uk</a>> wrote:<br>
I have a weird problem which I can’t get to the bottom of. <br>
<br>
We have a cluster which allows users to start interactive sessions which forward any X11 sessions they generated on the head node. This generally works fine, but on the account of one user it doesn’t work. The X11 connection to the head node is fine, but
it won’t transfer to the compute node.<br>
<br>
The symptoms are shown below:<br>
<br>
A good user gets this:<br>
<br>
[good@headnode ~]$ xauth list<br>
headnode.babraham.ac.uk/unix:12 MIT-MAGIC-COOKIE-1 f04a2bf9a921a3357e44373655add14a<br>
<br>
[good@headnode ~]$ echo $DISPLAY<br>
localhost:12.0<br>
<br>
[good@headnode ~]$ srun --pty -p interactive --x11 --preserve-env /bin/bash<br>
<br>
[good@compute ~]$ xauth list<br>
headnode.babraham.ac.uk/unix:12 MIT-MAGIC-COOKIE-1 f04a2bf9a921a3357e44373655add14a<br>
compute/unix:25 MIT-MAGIC-COOKIE-1 f04a2bf9a921a3357e44373655add14a<br>
<br>
[good@compute ~]$ echo $DISPLAY<br>
localhost:25.0<br>
<br>
So the cookie is copied from the head node and forwarded and the DISPLAY variable is updated.<br>
<br>
The bad user gets this:<br>
<br>
[bad@headnode ~]$ xauth list<br>
headnode.babraham.ac.uk/unix:10 MIT-MAGIC-COOKIE-1 c39a493a37132d308b37469d363d8692<br>
<br>
[bad@headnode ~]$ echo $DISPLAY<br>
localhost:10.0<br>
<br>
[bad@headnode ~]$ srun --pty -p interactive --x11 --preserve-env /bin/bash<br>
<br>
[bad@compute ~]$ xauth list<br>
headnode.babraham.ac.uk/unix:10 MIT-MAGIC-COOKIE-1 c39a493a37132d308b37469d363d8692<br>
<br>
[bad@compute ~]$ echo $DISPLAY<br>
localhost:10.0<br>
<br>
So the cookie isn’t copied and the DISPLAY isn’t updated. I can’t see any errors in the logs and I can’t see anything different about this account.<br>
<br>
If I do a straight forward ssh -Y from the head node to a compute node from the bad account then that works fine – it’s only whatever is specific about the way that srun forwards X which fails.<br>
<br>
Any ideas or suggestions for debugging would be appreciated as I’m running out of things to try!<br>
<br>
Simon.<br>
The Babraham Institute, Babraham Research Campus, Cambridge CB22 3AT Registered Charity No. 1053902.<br>
The information transmitted in this email is directed only to the addressee. If you received this in error, please contact the sender and delete this email from your system. The contents of this e-mail are the views of the sender and do not necessarily represent
the views of the Babraham Institute. Full conditions at: <a href="http://www.babraham.ac.uk">www.babraham.ac.uk</a><br>
The Babraham Institute, Babraham Research Campus, Cambridge CB22 3AT Registered Charity No. 1053902.<br>
The information transmitted in this email is directed only to the addressee. If you received this in error, please contact the sender and delete this email from your system. The contents of this e-mail are the views of the sender and do not necessarily represent
the views of the Babraham Institute. Full conditions at: <a href="http://www.babraham.ac.uk">www.babraham.ac.uk</a><o:p></o:p></p>
</blockquote>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal" style="text-autospace:none"><o:p></o:p><o:p></o:p><span style="font-size:8.0pt;
font-family:"Arial","sans-serif";color:#1F497D">The Babraham Institute, Babraham Research Campus, Cambridge CB22 3AT
</span><i><span style="font-size:
8.0pt;font-family:"Arial","sans-serif";color:#365F91;mso-themecolor:accent1;
mso-themeshade:191">Registered Charity No. 1053902.</span></i><span style="font-size:8.0pt;font-family:"Arial","sans-serif";color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-autospace:none"><span style="font-size:8.0pt;
font-family:"Arial","sans-serif";color:#1F497D">The information transmitted in this email is directed only to the addressee. If you received this in error, please contact the sender
and delete this email from your system. The contents of this e-mail are the views of the sender and do not necessarily represent the views of the Babraham Institute.
</span><span style="font-size:8.0pt;
font-family:"Arial","sans-serif";color:#365F91;mso-themecolor:accent1;
mso-themeshade:191">Full conditions at:
</span><a href="http://www.babraham.ac.uk/terms"><span style="font-size:8.0pt;font-family:"Arial","sans-serif";
color:#365F91;mso-themecolor:accent1;mso-themeshade:191">www.babraham.ac.uk</span></a><span style="font-size:8.0pt;font-family:"Arial","sans-serif";color:#365F91;
mso-themecolor:accent1;mso-themeshade:191"><o:p></o:p></span></p>
</body>
</html>