<div dir="ltr">Ok, thank you so much for that hint... I will try doing that and report back.<div><br></div><div>Thanks!</div><div>Lou</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Aug 16, 2019 at 11:05 AM Brian Andrus <<a href="mailto:toomuchit@gmail.com">toomuchit@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
  
    
  
  <div bgcolor="#FFFFFF">
    <p>Ah. I suspect your issue may be the cuda. 10.1 which does not
      create/register all the appropriate symlinks and "provides". <br>
      I ran into that trying to install tensorflow.</p>
    <p>If you can, downgrade to 10.0, which does a better job of
      installing itself.</p>
    <p>Brian<br>
    </p>
    <div class="gmail-m_-8669609114189166273moz-cite-prefix">On 8/16/2019 5:47 AM, Lou Nicotra
      wrote:<br>
    </div>
    <blockquote type="cite">
      
      <div dir="ltr">Brian, the package is being built and installed on
        the master server.  I am testing by removing all instances of
        V18 and installing the newly created V19 slurm rpms,  I get the
        error message on the slurm rpm install, all others (ctl, db, ...
        ) install fine.
        <div><br>
        </div>
        <div>After I get the error message, I remove all rpms from V19
          and reinstall V18 using the same procedure with no issues...
          And the system sees all nodes as it did before trying to
          install V19
          <div><br>
          </div>
          <div>The nvidia libraries are installed via the official
            Nvidia
            rpm... cuda-repo-rhel7-10-1-local-10.1.105-418.39-1.0-1.x86_64.rpm 
            supporting cuda10. Multi GPU server currently used by
            multiple users (DNN training) with no errors of any type
            while utilizing the nvidia libs/code.</div>
          <div><br>
          </div>
          <div>nvidia-smi command shows:  NVIDIA-SMI 418.39       Driver
            Version: 418.39       CUDA Version: 10.1</div>
        </div>
        <div><br>
        </div>
        <div>So, it is definitely something new to the V19 release... I
          have installed 18.08.0, .3, .4 and .8 on the same server and
          nodes since Sep of 2018 using the same procedures and never
          had any issues... Currently running 18.08.8</div>
        <div><br>
        </div>
        <div>Thanks.</div>
        <div>Lou</div>
      </div>
      <br>
      <div class="gmail_quote">
        <div dir="ltr" class="gmail_attr">On Thu, Aug 15, 2019 at 3:07
          PM Brian Andrus <<a href="mailto:toomuchit@gmail.com" target="_blank">toomuchit@gmail.com</a>> wrote:<br>
        </div>
        <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
          <div bgcolor="#FFFFFF">
            <p>Lou,</p>
            <p>Are you installing on the same machine you built?</p>
            <p>Are the nvidia libraries installed by RPM or a 'make
              install' on the box you compiled it on?</p>
            <p>Brian Andrus<br>
            </p>
            <div class="gmail-m_-8669609114189166273gmail-m_5122468330965931780moz-cite-prefix">On
              8/15/2019 7:53 AM, Lou Nicotra wrote:<br>
            </div>
            <blockquote type="cite">
              <div dir="ltr">I have tried running ldconfig manually as
                suggested with  slurm-19.05.1-2 and it fails the same
                way... 
                <div>error: Failed dependencies:<br>
                          libnvidia-ml.so.1()(64bit) is needed by
                  slurm-19.05.1-2.el7.centos.x86_64  <br>
                </div>
                <div><br>
                </div>
                <div>ldconfig -p shows:</div>
                <div>root@panther02 slurm# ldconfig -p|grep
                  libnvidia-ml.<br>
                          libnvidia-ml.so.1 (libc6,x86-64) =>
                  /usr/lib64/libnvidia-ml.so.1<br>
                          libnvidia-ml.so.1 (libc6) =>
                  /lib/libnvidia-ml.so.1<br>
                          libnvidia-ml.so (libc6,x86-64) =>
                  /usr/lib64/libnvidia-ml.so<br>
                          libnvidia-ml.so (libc6) =>
                  /lib/libnvidia-ml.so<br>
                </div>
                <div> <br>
                </div>
                <div>Just tried the latest release slurm-19.05.2 and it
                  fails in the same way... </div>
                <div>root@panther02 x86_64# rpm -Uvh
                  slurm-19.05.2-1.el7.centos.x86_64.rpm<br>
                  error: Failed dependencies:<br>
                          libnvidia-ml.so.1()(64bit) is needed by
                  slurm-19.05.2-1.el7.centos.x86_64<br>
                </div>
                <div><br>
                </div>
                <div>Reinstalled slurm-18.08.8 and it installs with no
                  issues... Just like slurm-18.08.03 and slurm-18.08.4
                  did...  All built on the same machine with rpmbuild
                  -ta command...</div>
                <div>root@panther02 slurm-18.08.8# rpm -Uvh
                  slurm-18.08.8-1.el7.centos.x86_64.rpm<br>
                  Preparing...                        
                   ################################# [100%]<br>
                  Updating / installing...<br>
                     1:slurm-18.08.8-1.el7.centos      
                  ################################# [100%]<br>
                </div>
                <div><br>
                </div>
                <div>Oh, well...</div>
                <div><br>
                </div>
                <div>Lou</div>
                <div><br>
                </div>
                <div><br>
                </div>
              </div>
              <br>
              <div class="gmail_quote">
                <div dir="ltr" class="gmail_attr">On Mon, Aug 12, 2019
                  at 1:32 AM Barbara Krašovec <<a href="mailto:barbara.krasovec@ijs.si" target="_blank">barbara.krasovec@ijs.si</a>>
                  wrote:<br>
                </div>
                <blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
                  <div bgcolor="#FFFFFF">
                    <p>What if you try to run ldconfig manually before
                      building the rpm?</p>
                    <p>Cheers,</p>
                    <p>Barbara<br>
                    </p>
                    <div class="gmail-m_-8669609114189166273gmail-m_5122468330965931780gmail-m_-2737365508863083922moz-cite-prefix">On
                      8/8/19 5:57 PM, Lou Nicotra wrote:<br>
                    </div>
                    <blockquote type="cite">
                      <div dir="ltr">I am running into an error while
                        trying to
                        install slurm-19.05.1-2.el7.centos.x86_64...
                        Error is as follows:
                        <div>root@panther02 x86_64# rpm -Uvh
                          slurm-19.05.1-2.el7.centos.x86_64.rpm<br>
                          error: Failed dependencies:<br>
                                  libnvidia-ml.so.1()(64bit) is needed
                          by slurm-19.05.1-2.el7.centos.x86_64</div>
                        <div><br>
                        </div>
                        <div>Packages are built using rpmbuild... And
                          complete with no errors...</div>
                        <div>+ cd /root/rpmbuild/BUILD<br>
                          + cd slurm-19.05.1-2<br>
                          + rm -rf
                          /root/rpmbuild/BUILDROOT/slurm-19.05.1-2.el7.centos.x86_64<br>
                          + exit 0<br>
                        </div>
                        <div><br>
                        </div>
                        <div>Investigation of the output while building
                          the rpm package shows that nvidia-ml is found:</div>
                        <div>checking for nvmlInit in -lnvidia-ml... yes<br>
                        </div>
                        <div>.</div>
                        <div>.</div>
                        <div>libtool: compile:  gcc -DHAVE_CONFIG_H -I.
                          -I../../../.. -I../../../../slurm
                          -I../../../.. -I../../../../src/common
                          -I/usr/local/cuda/include -I/usr/cuda/include
                          -DNUMA_VERSION1_COMPATIBILITY -O2 -g -pipe
                          -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
                          -fstack-protector-strong
                          --param=ssp-buffer-size=4
                          -grecord-gcc-switches -m64 -mtune=generic
                          -pthread -ggdb3 -Wall -g -O1
                          -fno-strict-aliasing -c gpu_nvml.c  -fPIC
                          -DPIC -o .libs/gpu_nvml.o<br>
                          libtool: compile:  gcc -DHAVE_CONFIG_H -I.
                          -I../../../.. -I../../../../slurm
                          -I../../../.. -I../../../../src/common
                          -I/usr/local/cuda/include -I/usr/cuda/include
                          -DNUMA_VERSION1_COMPATIBILITY -O2 -g -pipe
                          -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
                          -fstack-protector-strong
                          --param=ssp-buffer-size=4
                          -grecord-gcc-switches -m64 -mtune=generic
                          -pthread -ggdb3 -Wall -g -O1
                          -fno-strict-aliasing -c gpu_nvml.c -o
                          gpu_nvml.o >/dev/null 2>&1<br>
                          /bin/sh ../../../../libtool  --tag=CC  
                          --mode=link gcc  -DNUMA_VERSION1_COMPATIBILITY
                          -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2
                          -fexceptions -fstack-protector-strong
                          --param=ssp-buffer-size=4
                          -grecord-gcc-switches   -m64 -mtune=generic
                          -pthread -ggdb3 -Wall -g -O1
                          -fno-strict-aliasing -module -avoid-version
                          --export-dynamic -Wl,-z,relro   -o <a href="http://gpu_nvml.la" target="_blank">gpu_nvml.la</a>
                          -rpath /usr/lib64/slurm gpu_nvml.lo <font color="#ff0000">-lnvidia-ml</font><br>
                          libtool: link: gcc -shared  -fPIC -DPIC
                           .libs/gpu_nvml.o  <font color="#ff0000">
                            -lnvidia-ml  </font>-O2 -g
                          -fstack-protector-strong -grecord-gcc-switches
                          -m64 -mtune=generic -pthread -ggdb3 -g -O1
                          -Wl,-z -Wl,relro   -pthread -Wl,-soname
                          -Wl,gpu_nvml.so -o .libs/gpu_nvml.so<br>
                        </div>
                        <div><br clear="all">
                          <div>The Makefile in
                            /root/rpmbuild/BUILD/slurm-19.05.1-2/src 
                            includes: NVML_LIBS = -lnvidia-ml</div>
                          <div>but previous releases did not
                            (slurm-18.08.8) And I was able to compile
                            and install that release with no issues
                            after building it with rpmbuild...</div>
                          <div><br>
                          </div>
                          <div>My LD_LIBRARY_PATH is
/usr/lib64:/usr/lib:/usr/local/lib64:/usr/local/lib:/var/local/miniconda2/lib/:</div>
                          <div><br>
                          </div>
                          <div>Can anyone provide suggestions on working
                            out this issue?</div>
                          <div><br>
                          </div>
                          <div>Thanks.</div>
                           -- <br>
                          <div dir="ltr" class="gmail-m_-8669609114189166273gmail-m_5122468330965931780gmail-m_-2737365508863083922gmail_signature">
                            <div dir="ltr">
                              <div>
                                <div dir="ltr">
                                  <div>
                                    <div dir="ltr">
                                      <p class="MsoNormal"><span style="font-family:Arial,sans-serif;color:rgb(255,100,24)">LOU
                                          NICOTRA<span></span></span></p>
                                      <p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">IT
                                          Systems Engineer - SLT</span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
                                      <p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">Interactions
                                          LLC</span></p>
                                      <p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif">o:  </span><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)"><a href="tel:781-405-5114" target="_blank"><span style="color:rgb(17,85,204)">908-673-1833</span></a></span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
                                      <p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">m: <a href="tel:781-405-5114" target="_blank"><span style="color:rgb(17,85,204)">908-451-6983</span></a></span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
                                      <p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><u><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(17,85,204)"><a href="mailto:lnicotra@interactions.com" target="_blank">lnicotra@interactions.com</a></span></u><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
                                      <span style="font-size:9.5pt;line-height:107%;font-family:Arial,sans-serif;color:rgb(102,102,102)"><a href="http://www.interactions.com/" target="_blank"><span style="color:rgb(17,85,204)">www.interactions.com</span></a></span><br>
                                    </div>
                                  </div>
                                </div>
                              </div>
                            </div>
                          </div>
                        </div>
                      </div>
                      <br>
                      <font size="3" face="Times New Roman"> </font>
                      <p style="margin:0in 0in 8pt"><font size="3" face="Calibri">*******************************************************************************</font></p>
                      <font size="3" face="Times New Roman"> </font>
                      <p style="margin:0in 0in 8pt"><font size="3" face="Calibri">This e-mail and any of its
                          attachments may contain Interactions LLC
                          proprietary information, which is privileged,
                          confidential, or subject to copyright
                          belonging to the Interactions LLC. This e-mail
                          is intended solely for the use of the
                          individual or entity to which it is addressed.
                          If you are not the intended recipient of this
                          e-mail, you are hereby notified that any
                          dissemination, distribution, copying, or
                          action taken in relation to the contents of
                          and attachments to this e-mail is strictly
                          prohibited and may be unlawful. If you have
                          received this e-mail in error, please notify
                          the sender immediately and permanently delete
                          the original and any copy of this e-mail and
                          any printout. Thank You.  </font></p>
                      <font size="3" face="Times New Roman"> </font>
                      <p style="margin:0in 0in 8pt"><font face="Calibri"><font size="3">*******************************************************************************<span> 
                            </span></font></font></p>
                      <font size="3" face="Times New Roman"> </font> </blockquote>
                  </div>
                </blockquote>
              </div>
              <br clear="all">
              <div><br>
              </div>
              -- <br>
              <div dir="ltr" class="gmail-m_-8669609114189166273gmail-m_5122468330965931780gmail_signature">
                <div dir="ltr">
                  <div>
                    <div dir="ltr">
                      <div>
                        <div dir="ltr">
                          <p class="MsoNormal"><span style="font-family:Arial,sans-serif;color:rgb(255,100,24)">LOU
                              NICOTRA<span></span></span></p>
                          <p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">IT
                              Systems Engineer - SLT</span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
                          <p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">Interactions
                              LLC</span></p>
                          <p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif">o:  </span><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)"><a href="tel:781-405-5114" target="_blank"><span style="color:rgb(17,85,204)">908-673-1833</span></a></span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
                          <p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">m: <a href="tel:781-405-5114" target="_blank"><span style="color:rgb(17,85,204)">908-451-6983</span></a></span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
                          <p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><u><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(17,85,204)"><a href="mailto:lnicotra@interactions.com" target="_blank">lnicotra@interactions.com</a></span></u><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
                          <span style="font-size:9.5pt;line-height:107%;font-family:Arial,sans-serif;color:rgb(102,102,102)"><a href="http://www.interactions.com/" target="_blank"><span style="color:rgb(17,85,204)">www.interactions.com</span></a></span><br>
                        </div>
                      </div>
                    </div>
                  </div>
                </div>
              </div>
              <br>
              <font size="3" face="Times New Roman"> </font>
              <p style="margin:0in 0in 8pt"><font size="3" face="Calibri">*******************************************************************************</font></p>
              <font size="3" face="Times New Roman"> </font>
              <p style="margin:0in 0in 8pt"><font size="3" face="Calibri">This e-mail and any of its attachments
                  may contain Interactions LLC proprietary information,
                  which is privileged, confidential, or subject to
                  copyright belonging to the Interactions LLC. This
                  e-mail is intended solely for the use of the
                  individual or entity to which it is addressed. If you
                  are not the intended recipient of this e-mail, you are
                  hereby notified that any dissemination, distribution,
                  copying, or action taken in relation to the contents
                  of and attachments to this e-mail is strictly
                  prohibited and may be unlawful. If you have received
                  this e-mail in error, please notify the sender
                  immediately and permanently delete the original and
                  any copy of this e-mail and any printout. Thank You. 
                </font></p>
              <font size="3" face="Times New Roman"> </font>
              <p style="margin:0in 0in 8pt"><font face="Calibri"><font size="3">*******************************************************************************<span> 
                    </span></font></font></p>
              <font size="3" face="Times New Roman"> </font> </blockquote>
          </div>
        </blockquote>
      </div>
      <br clear="all">
      <div><br>
      </div>
      -- <br>
      <div dir="ltr" class="gmail-m_-8669609114189166273gmail_signature">
        <div dir="ltr">
          <div>
            <div dir="ltr">
              <div>
                <div dir="ltr">
                  <p class="MsoNormal"><span style="font-family:Arial,sans-serif;color:rgb(255,100,24)">LOU
                      NICOTRA<span></span></span></p>
                  <p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">IT
                      Systems Engineer -
                      SLT</span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
                  <p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">Interactions
                      LLC</span></p>
                  <p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif">o:  </span><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)"><a href="tel:781-405-5114" target="_blank"><span style="color:rgb(17,85,204)">908-673-1833</span></a></span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
                  <p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">m: <a href="tel:781-405-5114" target="_blank"><span style="color:rgb(17,85,204)">908-451-6983</span></a></span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
                  <p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><u><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(17,85,204)"><a href="mailto:lnicotra@interactions.com" target="_blank">lnicotra@interactions.com</a></span></u><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
                  <span style="font-size:9.5pt;line-height:107%;font-family:Arial,sans-serif;color:rgb(102,102,102)"><a href="http://www.interactions.com/" target="_blank"><span style="color:rgb(17,85,204)">www.interactions.com</span></a></span><br>
                </div>
              </div>
            </div>
          </div>
        </div>
      </div>
      <br>
      <font size="3" face="Times New Roman">
      </font>
      <p style="margin:0in 0in 8pt"><font size="3" face="Calibri">*******************************************************************************</font></p>
      <font size="3" face="Times New Roman">
      </font>
      <p style="margin:0in 0in 8pt"><font size="3" face="Calibri">This
          e-mail and any of its attachments may contain
          Interactions LLC proprietary information, which is privileged,
          confidential, or subject to copyright belonging to the
          Interactions
          LLC. This e-mail is intended solely for the use of the
          individual or
          entity to which it is addressed. If you are not the intended
          recipient of this
          e-mail, you are hereby notified that any dissemination,
          distribution, copying,
          or action taken in relation to the contents of and attachments
          to this e-mail
          is strictly prohibited and may be unlawful. If you have
          received this e-mail in
          error, please notify the sender immediately and permanently
          delete the original
          and any copy of this e-mail and any printout. Thank You.  </font></p>
      <font size="3" face="Times New Roman">
      </font>
      <p style="margin:0in 0in 8pt"><font face="Calibri"><font size="3">*******************************************************************************<span> 
            </span></font></font></p>
      <font size="3" face="Times New Roman">
      </font>
    </blockquote>
  </div>

</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><p class="MsoNormal"><span style="font-family:Arial,sans-serif;color:rgb(255,100,24)">LOU NICOTRA<span></span></span></p>

<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">IT Systems Engineer -
SLT</span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>

<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">Interactions LLC</span></p>

<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif">o:  </span><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)"><a href="tel:781-405-5114" target="_blank"><span style="color:rgb(17,85,204)">908-673-1833</span></a></span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>

<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">m: <a href="tel:781-405-5114" target="_blank"><span style="color:rgb(17,85,204)">908-451-6983</span></a></span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>

<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><u><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(17,85,204)"><a href="mailto:lnicotra@interactions.com" target="_blank">lnicotra@interactions.com</a></span></u><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>

<span style="font-size:9.5pt;line-height:107%;font-family:Arial,sans-serif;color:rgb(102,102,102)"><a href="http://www.interactions.com/" target="_blank"><span style="color:rgb(17,85,204)">www.interactions.com</span></a></span><br></div></div></div></div></div></div>

<br>
<font face="Times New Roman" size="3">

</font><p style="margin:0in 0in 8pt"><font face="Calibri" size="3">******************************<wbr>******************************<wbr>*******************</font></p><font face="Times New Roman" size="3">

</font><p style="margin:0in 0in 8pt"><font face="Calibri" size="3">This e-mail and any of its attachments may contain
Interactions LLC proprietary information, which is privileged,
confidential, or subject to copyright belonging to the Interactions
LLC. This e-mail is intended solely for the use of the individual or
entity to which it is addressed. If you are not the intended recipient of this
e-mail, you are hereby notified that any dissemination, distribution, copying,
or action taken in relation to the contents of and attachments to this e-mail
is strictly prohibited and may be unlawful. If you have received this e-mail in
error, please notify the sender immediately and permanently delete the original
and any copy of this e-mail and any printout. Thank You.  </font></p><font face="Times New Roman" size="3">

</font><p style="margin:0in 0in 8pt"><font face="Calibri"><font size="3">******************************<wbr>******************************<wbr>*******************<span>  </span></font></font></p><font face="Times New Roman" size="3">

</font>