[slurm-users] RPM build error - accounting_storage_mysql.so

William Brown william at signalbox.org.uk
Tue Nov 12 08:57:40 UTC 2019


Thanks to all for the useful suggestions.   I guess the main point is to
read the MariaDB updated instructions; I had looked at Release Notes but
what seems to have changed is the packaging, so that MariaDB-shared is now
required if using -lmariadb in the build.   I can see that it is likely
also possible to make it use a static library to avoid having to install
MariaDB-shared in target systems.   In fact I do only use SlurmDBD so the
MariaDB hooks are only needed in that package, not in slurmd at all.  I
expect if I read the Slurm build notes more carefully I can set options to
not install MySQL for slurmd while keeping for slurmdbd; it was installed
because the server on which I was building has the 'mysql_config' file,
which is enough to trigger building as if --with-mysql.

Ole, thanks for updating your pages - I have them bookmarked and have used
them for several years as an easy quick start.

As to why use MariaDB 10.x - mostly because the 5.x code path is not
maintained since Oracle bought MySQL, but also 10.2 and later include
backup tools, though I admit that I have yet to use them.

I cannot answer about Galera as I have not used it, but it does seem
strange that MariaDB having brought Galera into the main package only a few
versions ago have now split it again.

William

On Tue, 12 Nov 2019 at 08:14, Ole Holm Nielsen <Ole.H.Nielsen at fysik.dtu.dk>
wrote:

> Hi Daniel,
>
> Thanks for sharing your insights!  I have updated my Wiki page
>
> https://wiki.fysik.dtu.dk/niflheim/Slurm_installation#install-mariadb-database
> now.
>
> /Ole
>
> On 11/12/19 8:52 AM, Daniel Letai wrote:
> >
> > On 11/12/19 9:34 AM, Ole Holm Nielsen wrote:
> >> On 11/11/19 10:14 PM, Daniel Letai wrote:
> >>> Why would you need galera-4 as a build require?
> >>
> >> This is the MariaDB recommendation in
> >> https://mariadb.com/kb/en/library/yum/, see the section "Installing
> >> MariaDB Packages with YUM".  I have no clue why this would be needed.
> >
> > Yes, it's required for mariadb multimaster cluster. This has nothing to
> do
> > with the mariadb api required for linking against mariadb libs.
> >
> >
> > You don't even need the mariadb-server pkg for build purposes - it's
> only
> > required for deployment of slurmdbd.
> >
> > On a build machine, you should only require the client section:
> >
> https://mariadb.com/kb/en/library/yum/#installing-mariadb-clients-and-client-libraries-with-yum,
>
> > as well as the devel pkg.
> >
> >>
> >> /Ole
> >>
> >>
> >>> If it's required by any of the mariadb packages, it'll get pulled
> >>> automatically. If not, you don't need it on the build system.
> >>>
> >>>
> >>> On 11/11/19 10:56 PM, Ole Holm Nielsen wrote:
> >>>> Hi William,
> >>>>
> >>>> Interesting experiences with MariaDB 10.4!  I tried to collect the
> >>>> instructions from the MariaDB page, but I'm unsure about how to get
> >>>> the galera-4 RPM.
> >>>>
> >>>> Could you kindly review and correct my updated instructions?
> >>>>
> https://wiki.fysik.dtu.dk/niflheim/Slurm_installation#build-slurm-rpms
> >>>>
> >>>> That said, what are the main reasons for installing MariaDB 10 in
> >>>> stead of the 5.5 delivered by RedHat?  I'm not sure how well SchedMD
> >>>> has tested MariaDB 10 with Slurm?
> >>>>
> >>>> /Ole
> >>>>
> >>>>
> >>>> On 11-11-2019 21:23, William Brown wrote:
> >>>>> I have in fact found the answer by looking harder.
> >>>>>
> >>>>> The config.log clearly showed that the build of the test MySQL
> >>>>> program failed, which is why it was set to be excluded.
> >>>>>
> >>>>> It failed to link against '-lmariadb'.  It turns out that library is
> >>>>> no longer in MariaDB or MariaDB-devel, it is separately packaged in
> >>>>> MariaDB-shared.  That may of course be because I have built MariaDB
> >>>>> 10.4 from the mariadb.org site, because CentOS 7 only ships with
> the
> >>>>> extremely old version 5.5.
> >>>>>
> >>>>> Once I installed the missing package it built the RPMs just fine.
> >>>>> However it would be easier to use it linked to static MariaDB
> >>>>> libraries, as I now have to installed MariaDB-shared on every server
> >>>>> that will run slurmd, i.e. all compute nodes.  I expect that if I
> >>>>> looked harder at the build options there may be a way to do this,
> >>>>> perhaps with linker flags.
> >>>>>
> >>>>> For now, I can progress.
> >>>>>
> >>>>> Thanks
> >>>>>
> >>>>> William
> >>>>>
> >>>>> -----Original Message-----
> >>>>> From: slurm-users <slurm-users-bounces at lists.schedmd.com> On Behalf
> >>>>> Of Ole Holm Nielsen
> >>>>> Sent: 11 November 2019 20:02
> >>>>> To: slurm-users at lists.schedmd.com
> >>>>> Subject: Re: [slurm-users] RPM build error -
> accounting_storage_mysql.so
> >>>>>
> >>>>> Hi,
> >>>>>
> >>>>> Maybe my Slurm Wiki can help you build SLurm on CentOS/RHEL 7? See
> >>>>>
> https://wiki.fysik.dtu.dk/niflheim/Slurm_installation#build-slurm-rpms
> >>>>>
> >>>>> Note in particular:
> >>>>>> Important: Install the MariaDB (a replacement for MySQL) packages
> >>>>>> before you build Slurm RPMs (otherwise some libraries will be
> missing):
> >>>>>>
> >>>>>> yum install mariadb-server mariadb-devel
> >>>>>
> >>>>> /Ole
> >>>>>
> >>>>>
> >>>>> On 11-11-2019 15:22, William Brown wrote:
> >>>>>> Fabio
> >>>>>>
> >>>>>> Did you ever resolve the problem building
> accounting_storage_mysql.so?
> >>>>>>
> >>>>>> I have the exact same problem with CentOS 7.6, building Slurm
> 19.05.03.
> >>>>>> My command:
> >>>>>>
> >>>>>> rpmbuild -ta slurm-19.05.3-2.tar.bz2 | tee /var/tmp/slurm-build.log
> >>>>>>
> >>>>>> The directory with the plugin source is all there:
> >>>>>>
> /home/users/slurm/rpmbuild/BUILD/slurm-19.05.3-2/src/plugins/accountin
> >>>>>> g_storage/mysql, with a Makefile that is the same date/time as the
> >>>>>> other accounting_storage alternatives.
> >>>>>>
> >>>>>> In the log I can see:
> >>>>>>
> >>>>>> checking for mysql_config... /usr/bin/mysql_config
> >>>>>>
> >>>>>> Looking at the process of building the RPMs it looks as if it has
> >>>>>> skipped trying to create the missing library file, but then expects
> to
> >>>>>> find it in the RPM.
> >>>>>>
> >>>>>> This is what I see when it is building, it builds the
> >>>>>> accounting_storage .so files for _fileext, _none and _slurmdbd, but
> >>>>>> not for _mysql.  I do have MariaDB-devel 10.4.10 installed
> >>>>>>
> >>>>>> .
> >>>>>>
> >>>>>> .
> >>>>>>
> >>>>>> Making all in mysql
> >>>>>>
> >>>>>> make[5]: Entering directory
> >>>>>>
> `/home/users/slurm/rpmbuild/BUILD/slurm-19.05.3-2/src/plugins/accounting_storage/mysql'
>
> >>>>>>
> >>>>>>
> >>>>>> make[5]: Nothing to be done for `all'.
> >>>>>>
> >>>>>> make[5]: Leaving directory
> >>>>>>
> `/home/users/slurm/rpmbuild/BUILD/slurm-19.05.3-2/src/plugins/accounting_storage/mysql'
>
> >>>>>>
> >>>>>>
> >>>>>> .
> >>>>>>
> >>>>>> .
> >>>>>>
> >>>>>> Making install in mysql
> >>>>>>
> >>>>>> make[4]: Entering directory
> >>>>>>
> `/home/users/slurm/rpmbuild/BUILD/slurm-19.05.3-2/src/plugins/accounting_storage/mysql'
>
> >>>>>>
> >>>>>>
> >>>>>> make[5]: Entering directory
> >>>>>>
> `/home/users/slurm/rpmbuild/BUILD/slurm-19.05.3-2/src/plugins/accounting_storage/mysql'
>
> >>>>>>
> >>>>>>
> >>>>>> make[5]: Nothing to be done for `install-data-am'.
> >>>>>>
> >>>>>> make[5]: Leaving directory
> >>>>>>
> `/home/users/slurm/rpmbuild/BUILD/slurm-19.05.3-2/src/plugins/accounting_storage/mysql'
>
> >>>>>>
> >>>>>>
> >>>>>> make[4]: Leaving directory
> >>>>>>
> `/home/users/slurm/rpmbuild/BUILD/slurm-19.05.3-2/src/plugins/accounting_storage/mysql'
>
> >>>>>>
> >>>>>>
> >>>>>> The error stream shows the problem a little later:
> >>>>>>
> >>>>>> Processing files: slurm-19.05.3-2.el7.x86_64
> >>>>>>
> >>>>>> warning: File not found:
> >>>>>>
> /home/users/slurm/rpmbuild/BUILDROOT/slurm-19.05.3-2.el7.x86_64/home/a
> >>>>>> pps/slurm/lib64/slurm/accounting_storage_mysql.so
> >>>>>>
> >>>>>> .
> >>>>>>
> >>>>>> .
> >>>>>>
> >>>>>> Processing files: slurm-slurmdbd-19.05.3-2.el7.x86_64
> >>>>>>
> >>>>>> error: File not found:
> >>>>>>
> /home/users/slurm/rpmbuild/BUILDROOT/slurm-19.05.3-2.el7.x86_64/home/a
> >>>>>> pps/slurm/lib64/slurm/accounting_storage_mysql.so
> >>>>>>
> >>>>>> File not found:
> >>>>>>
> /home/users/slurm/rpmbuild/BUILDROOT/slurm-19.05.3-2.el7.x86_64/home/a
> >>>>>> pps/slurm/lib64/slurm/accounting_storage_mysql.so
> >>>>>>
> >>>>>> File not found:
> >>>>>>
> /home/users/slurm/rpmbuild/BUILDROOT/slurm-19.05.3-2.el7.x86_64/home/a
> >>>>>> pps/slurm/lib64/slurm/accounting_storage_mysql.so
> >>>>>>
> >>>>>> RPM build errors:
> >>>>>>
> >>>>>> I looked at the Makefile for the accounting_storage_mysql as
> compared
> >>>>>> to the Makefile for accounting_storage_slurmdbd to try to
> understand
> >>>>>> them.
> >>>>>>
> >>>>>> This is a selection from the Makefile for mysql:
> >>>>>>
> >>>>>> all: all-am
> >>>>>>
> >>>>>> all-am: Makefile $(LTLIBRARIES)
> >>>>>>
> >>>>>> LTLIBRARIES = $(pkglib_LTLIBRARIES)
> >>>>>>
> >>>>>> #pkglib_LTLIBRARIES = accounting_storage_mysql.la
> >>>>>> <http://accounting_storage_mysql.la>
> >>>>>>
> >>>>>> And this is for slurmdbd:
> >>>>>>
> >>>>>> all: all-am
> >>>>>>
> >>>>>> all-am: Makefile $(LTLIBRARIES)
> >>>>>>
> >>>>>> LTLIBRARIES = $(pkglib_LTLIBRARIES)
> >>>>>>
> >>>>>> pkglib_LTLIBRARIES = accounting_storage_slurmdbd.la
> >>>>>> <http://accounting_storage_slurmdbd.la>
> >>>>>>
> >>>>>> So I think that the problem is that the definition of
> >>>>>> pkglib_LTLIBRARIES is commented out in the accounting_storage_mysql
> >>>>>> Makefile, hence nothing to build.
> >>>>>>
> >>>>>> Is that intended?  Is it a consequence of something in my
> environment?
> >>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20191112/6e59aa09/attachment-0001.htm>


More information about the slurm-users mailing list