<div dir="ltr">Ok, thank you so much for that hint... I will try doing that and report back.<div><br></div><div>Thanks!</div><div>Lou</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Aug 16, 2019 at 11:05 AM Brian Andrus <<a href="mailto:toomuchit@gmail.com">toomuchit@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Ah. I suspect your issue may be the cuda. 10.1 which does not
create/register all the appropriate symlinks and "provides". <br>
I ran into that trying to install tensorflow.</p>
<p>If you can, downgrade to 10.0, which does a better job of
installing itself.</p>
<p>Brian<br>
</p>
<div class="gmail-m_-8669609114189166273moz-cite-prefix">On 8/16/2019 5:47 AM, Lou Nicotra
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">Brian, the package is being built and installed on
the master server. I am testing by removing all instances of
V18 and installing the newly created V19 slurm rpms, I get the
error message on the slurm rpm install, all others (ctl, db, ...
) install fine.
<div><br>
</div>
<div>After I get the error message, I remove all rpms from V19
and reinstall V18 using the same procedure with no issues...
And the system sees all nodes as it did before trying to
install V19
<div><br>
</div>
<div>The nvidia libraries are installed via the official
Nvidia
rpm... cuda-repo-rhel7-10-1-local-10.1.105-418.39-1.0-1.x86_64.rpm
supporting cuda10. Multi GPU server currently used by
multiple users (DNN training) with no errors of any type
while utilizing the nvidia libs/code.</div>
<div><br>
</div>
<div>nvidia-smi command shows: NVIDIA-SMI 418.39 Driver
Version: 418.39 CUDA Version: 10.1</div>
</div>
<div><br>
</div>
<div>So, it is definitely something new to the V19 release... I
have installed 18.08.0, .3, .4 and .8 on the same server and
nodes since Sep of 2018 using the same procedures and never
had any issues... Currently running 18.08.8</div>
<div><br>
</div>
<div>Thanks.</div>
<div>Lou</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Aug 15, 2019 at 3:07
PM Brian Andrus <<a href="mailto:toomuchit@gmail.com" target="_blank">toomuchit@gmail.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>Lou,</p>
<p>Are you installing on the same machine you built?</p>
<p>Are the nvidia libraries installed by RPM or a 'make
install' on the box you compiled it on?</p>
<p>Brian Andrus<br>
</p>
<div class="gmail-m_-8669609114189166273gmail-m_5122468330965931780moz-cite-prefix">On
8/15/2019 7:53 AM, Lou Nicotra wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">I have tried running ldconfig manually as
suggested with slurm-19.05.1-2 and it fails the same
way...
<div>error: Failed dependencies:<br>
libnvidia-ml.so.1()(64bit) is needed by
slurm-19.05.1-2.el7.centos.x86_64 <br>
</div>
<div><br>
</div>
<div>ldconfig -p shows:</div>
<div>root@panther02 slurm# ldconfig -p|grep
libnvidia-ml.<br>
libnvidia-ml.so.1 (libc6,x86-64) =>
/usr/lib64/libnvidia-ml.so.1<br>
libnvidia-ml.so.1 (libc6) =>
/lib/libnvidia-ml.so.1<br>
libnvidia-ml.so (libc6,x86-64) =>
/usr/lib64/libnvidia-ml.so<br>
libnvidia-ml.so (libc6) =>
/lib/libnvidia-ml.so<br>
</div>
<div> <br>
</div>
<div>Just tried the latest release slurm-19.05.2 and it
fails in the same way... </div>
<div>root@panther02 x86_64# rpm -Uvh
slurm-19.05.2-1.el7.centos.x86_64.rpm<br>
error: Failed dependencies:<br>
libnvidia-ml.so.1()(64bit) is needed by
slurm-19.05.2-1.el7.centos.x86_64<br>
</div>
<div><br>
</div>
<div>Reinstalled slurm-18.08.8 and it installs with no
issues... Just like slurm-18.08.03 and slurm-18.08.4
did... All built on the same machine with rpmbuild
-ta command...</div>
<div>root@panther02 slurm-18.08.8# rpm -Uvh
slurm-18.08.8-1.el7.centos.x86_64.rpm<br>
Preparing...
################################# [100%]<br>
Updating / installing...<br>
1:slurm-18.08.8-1.el7.centos
################################# [100%]<br>
</div>
<div><br>
</div>
<div>Oh, well...</div>
<div><br>
</div>
<div>Lou</div>
<div><br>
</div>
<div><br>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Aug 12, 2019
at 1:32 AM Barbara Krašovec <<a href="mailto:barbara.krasovec@ijs.si" target="_blank">barbara.krasovec@ijs.si</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>What if you try to run ldconfig manually before
building the rpm?</p>
<p>Cheers,</p>
<p>Barbara<br>
</p>
<div class="gmail-m_-8669609114189166273gmail-m_5122468330965931780gmail-m_-2737365508863083922moz-cite-prefix">On
8/8/19 5:57 PM, Lou Nicotra wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">I am running into an error while
trying to
install slurm-19.05.1-2.el7.centos.x86_64...
Error is as follows:
<div>root@panther02 x86_64# rpm -Uvh
slurm-19.05.1-2.el7.centos.x86_64.rpm<br>
error: Failed dependencies:<br>
libnvidia-ml.so.1()(64bit) is needed
by slurm-19.05.1-2.el7.centos.x86_64</div>
<div><br>
</div>
<div>Packages are built using rpmbuild... And
complete with no errors...</div>
<div>+ cd /root/rpmbuild/BUILD<br>
+ cd slurm-19.05.1-2<br>
+ rm -rf
/root/rpmbuild/BUILDROOT/slurm-19.05.1-2.el7.centos.x86_64<br>
+ exit 0<br>
</div>
<div><br>
</div>
<div>Investigation of the output while building
the rpm package shows that nvidia-ml is found:</div>
<div>checking for nvmlInit in -lnvidia-ml... yes<br>
</div>
<div>.</div>
<div>.</div>
<div>libtool: compile: gcc -DHAVE_CONFIG_H -I.
-I../../../.. -I../../../../slurm
-I../../../.. -I../../../../src/common
-I/usr/local/cuda/include -I/usr/cuda/include
-DNUMA_VERSION1_COMPATIBILITY -O2 -g -pipe
-Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector-strong
--param=ssp-buffer-size=4
-grecord-gcc-switches -m64 -mtune=generic
-pthread -ggdb3 -Wall -g -O1
-fno-strict-aliasing -c gpu_nvml.c -fPIC
-DPIC -o .libs/gpu_nvml.o<br>
libtool: compile: gcc -DHAVE_CONFIG_H -I.
-I../../../.. -I../../../../slurm
-I../../../.. -I../../../../src/common
-I/usr/local/cuda/include -I/usr/cuda/include
-DNUMA_VERSION1_COMPATIBILITY -O2 -g -pipe
-Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions
-fstack-protector-strong
--param=ssp-buffer-size=4
-grecord-gcc-switches -m64 -mtune=generic
-pthread -ggdb3 -Wall -g -O1
-fno-strict-aliasing -c gpu_nvml.c -o
gpu_nvml.o >/dev/null 2>&1<br>
/bin/sh ../../../../libtool --tag=CC
--mode=link gcc -DNUMA_VERSION1_COMPATIBILITY
-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2
-fexceptions -fstack-protector-strong
--param=ssp-buffer-size=4
-grecord-gcc-switches -m64 -mtune=generic
-pthread -ggdb3 -Wall -g -O1
-fno-strict-aliasing -module -avoid-version
--export-dynamic -Wl,-z,relro -o <a href="http://gpu_nvml.la" target="_blank">gpu_nvml.la</a>
-rpath /usr/lib64/slurm gpu_nvml.lo <font color="#ff0000">-lnvidia-ml</font><br>
libtool: link: gcc -shared -fPIC -DPIC
.libs/gpu_nvml.o <font color="#ff0000">
-lnvidia-ml </font>-O2 -g
-fstack-protector-strong -grecord-gcc-switches
-m64 -mtune=generic -pthread -ggdb3 -g -O1
-Wl,-z -Wl,relro -pthread -Wl,-soname
-Wl,gpu_nvml.so -o .libs/gpu_nvml.so<br>
</div>
<div><br clear="all">
<div>The Makefile in
/root/rpmbuild/BUILD/slurm-19.05.1-2/src
includes: NVML_LIBS = -lnvidia-ml</div>
<div>but previous releases did not
(slurm-18.08.8) And I was able to compile
and install that release with no issues
after building it with rpmbuild...</div>
<div><br>
</div>
<div>My LD_LIBRARY_PATH is
/usr/lib64:/usr/lib:/usr/local/lib64:/usr/local/lib:/var/local/miniconda2/lib/:</div>
<div><br>
</div>
<div>Can anyone provide suggestions on working
out this issue?</div>
<div><br>
</div>
<div>Thanks.</div>
-- <br>
<div dir="ltr" class="gmail-m_-8669609114189166273gmail-m_5122468330965931780gmail-m_-2737365508863083922gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<p class="MsoNormal"><span style="font-family:Arial,sans-serif;color:rgb(255,100,24)">LOU
NICOTRA<span></span></span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">IT
Systems Engineer - SLT</span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">Interactions
LLC</span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif">o: </span><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)"><a href="tel:781-405-5114" target="_blank"><span style="color:rgb(17,85,204)">908-673-1833</span></a></span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">m: <a href="tel:781-405-5114" target="_blank"><span style="color:rgb(17,85,204)">908-451-6983</span></a></span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><u><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(17,85,204)"><a href="mailto:lnicotra@interactions.com" target="_blank">lnicotra@interactions.com</a></span></u><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
<span style="font-size:9.5pt;line-height:107%;font-family:Arial,sans-serif;color:rgb(102,102,102)"><a href="http://www.interactions.com/" target="_blank"><span style="color:rgb(17,85,204)">www.interactions.com</span></a></span><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<font size="3" face="Times New Roman"> </font>
<p style="margin:0in 0in 8pt"><font size="3" face="Calibri">*******************************************************************************</font></p>
<font size="3" face="Times New Roman"> </font>
<p style="margin:0in 0in 8pt"><font size="3" face="Calibri">This e-mail and any of its
attachments may contain Interactions LLC
proprietary information, which is privileged,
confidential, or subject to copyright
belonging to the Interactions LLC. This e-mail
is intended solely for the use of the
individual or entity to which it is addressed.
If you are not the intended recipient of this
e-mail, you are hereby notified that any
dissemination, distribution, copying, or
action taken in relation to the contents of
and attachments to this e-mail is strictly
prohibited and may be unlawful. If you have
received this e-mail in error, please notify
the sender immediately and permanently delete
the original and any copy of this e-mail and
any printout. Thank You. </font></p>
<font size="3" face="Times New Roman"> </font>
<p style="margin:0in 0in 8pt"><font face="Calibri"><font size="3">*******************************************************************************<span>
</span></font></font></p>
<font size="3" face="Times New Roman"> </font> </blockquote>
</div>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr" class="gmail-m_-8669609114189166273gmail-m_5122468330965931780gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<p class="MsoNormal"><span style="font-family:Arial,sans-serif;color:rgb(255,100,24)">LOU
NICOTRA<span></span></span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">IT
Systems Engineer - SLT</span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">Interactions
LLC</span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif">o: </span><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)"><a href="tel:781-405-5114" target="_blank"><span style="color:rgb(17,85,204)">908-673-1833</span></a></span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">m: <a href="tel:781-405-5114" target="_blank"><span style="color:rgb(17,85,204)">908-451-6983</span></a></span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><u><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(17,85,204)"><a href="mailto:lnicotra@interactions.com" target="_blank">lnicotra@interactions.com</a></span></u><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
<span style="font-size:9.5pt;line-height:107%;font-family:Arial,sans-serif;color:rgb(102,102,102)"><a href="http://www.interactions.com/" target="_blank"><span style="color:rgb(17,85,204)">www.interactions.com</span></a></span><br>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<font size="3" face="Times New Roman"> </font>
<p style="margin:0in 0in 8pt"><font size="3" face="Calibri">*******************************************************************************</font></p>
<font size="3" face="Times New Roman"> </font>
<p style="margin:0in 0in 8pt"><font size="3" face="Calibri">This e-mail and any of its attachments
may contain Interactions LLC proprietary information,
which is privileged, confidential, or subject to
copyright belonging to the Interactions LLC. This
e-mail is intended solely for the use of the
individual or entity to which it is addressed. If you
are not the intended recipient of this e-mail, you are
hereby notified that any dissemination, distribution,
copying, or action taken in relation to the contents
of and attachments to this e-mail is strictly
prohibited and may be unlawful. If you have received
this e-mail in error, please notify the sender
immediately and permanently delete the original and
any copy of this e-mail and any printout. Thank You.
</font></p>
<font size="3" face="Times New Roman"> </font>
<p style="margin:0in 0in 8pt"><font face="Calibri"><font size="3">*******************************************************************************<span>
</span></font></font></p>
<font size="3" face="Times New Roman"> </font> </blockquote>
</div>
</blockquote>
</div>
<br clear="all">
<div><br>
</div>
-- <br>
<div dir="ltr" class="gmail-m_-8669609114189166273gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<p class="MsoNormal"><span style="font-family:Arial,sans-serif;color:rgb(255,100,24)">LOU
NICOTRA<span></span></span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">IT
Systems Engineer -
SLT</span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">Interactions
LLC</span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif">o: </span><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)"><a href="tel:781-405-5114" target="_blank"><span style="color:rgb(17,85,204)">908-673-1833</span></a></span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">m: <a href="tel:781-405-5114" target="_blank"><span style="color:rgb(17,85,204)">908-451-6983</span></a></span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><u><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(17,85,204)"><a href="mailto:lnicotra@interactions.com" target="_blank">lnicotra@interactions.com</a></span></u><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
<span style="font-size:9.5pt;line-height:107%;font-family:Arial,sans-serif;color:rgb(102,102,102)"><a href="http://www.interactions.com/" target="_blank"><span style="color:rgb(17,85,204)">www.interactions.com</span></a></span><br>
</div>
</div>
</div>
</div>
</div>
</div>
<br>
<font size="3" face="Times New Roman">
</font>
<p style="margin:0in 0in 8pt"><font size="3" face="Calibri">*******************************************************************************</font></p>
<font size="3" face="Times New Roman">
</font>
<p style="margin:0in 0in 8pt"><font size="3" face="Calibri">This
e-mail and any of its attachments may contain
Interactions LLC proprietary information, which is privileged,
confidential, or subject to copyright belonging to the
Interactions
LLC. This e-mail is intended solely for the use of the
individual or
entity to which it is addressed. If you are not the intended
recipient of this
e-mail, you are hereby notified that any dissemination,
distribution, copying,
or action taken in relation to the contents of and attachments
to this e-mail
is strictly prohibited and may be unlawful. If you have
received this e-mail in
error, please notify the sender immediately and permanently
delete the original
and any copy of this e-mail and any printout. Thank You. </font></p>
<font size="3" face="Times New Roman">
</font>
<p style="margin:0in 0in 8pt"><font face="Calibri"><font size="3">*******************************************************************************<span>
</span></font></font></p>
<font size="3" face="Times New Roman">
</font>
</blockquote>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><p class="MsoNormal"><span style="font-family:Arial,sans-serif;color:rgb(255,100,24)">LOU NICOTRA<span></span></span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">IT Systems Engineer -
SLT</span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">Interactions LLC</span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif">o: </span><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)"><a href="tel:781-405-5114" target="_blank"><span style="color:rgb(17,85,204)">908-673-1833</span></a></span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(102,102,102)">m: <a href="tel:781-405-5114" target="_blank"><span style="color:rgb(17,85,204)">908-451-6983</span></a></span><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
<p style="margin-bottom:0.0001pt;line-height:normal;background-image:initial;background-position:initial;background-repeat:initial"><u><span style="font-size:9.5pt;font-family:Arial,sans-serif;color:rgb(17,85,204)"><a href="mailto:lnicotra@interactions.com" target="_blank">lnicotra@interactions.com</a></span></u><span style="font-size:9.5pt;font-family:Arial,sans-serif"></span></p>
<span style="font-size:9.5pt;line-height:107%;font-family:Arial,sans-serif;color:rgb(102,102,102)"><a href="http://www.interactions.com/" target="_blank"><span style="color:rgb(17,85,204)">www.interactions.com</span></a></span><br></div></div></div></div></div></div>
<br>
<font face="Times New Roman" size="3">
</font><p style="margin:0in 0in 8pt"><font face="Calibri" size="3">******************************<wbr>******************************<wbr>*******************</font></p><font face="Times New Roman" size="3">
</font><p style="margin:0in 0in 8pt"><font face="Calibri" size="3">This e-mail and any of its attachments may contain
Interactions LLC proprietary information, which is privileged,
confidential, or subject to copyright belonging to the Interactions
LLC. This e-mail is intended solely for the use of the individual or
entity to which it is addressed. If you are not the intended recipient of this
e-mail, you are hereby notified that any dissemination, distribution, copying,
or action taken in relation to the contents of and attachments to this e-mail
is strictly prohibited and may be unlawful. If you have received this e-mail in
error, please notify the sender immediately and permanently delete the original
and any copy of this e-mail and any printout. Thank You. </font></p><font face="Times New Roman" size="3">
</font><p style="margin:0in 0in 8pt"><font face="Calibri"><font size="3">******************************<wbr>******************************<wbr>*******************<span> </span></font></font></p><font face="Times New Roman" size="3">
</font>