[slurm-users] RPM build error - accounting_storage_mysql.so

Tue Nov 12 08:11:30 UTC 2019

Hi Daniel,

Thanks for sharing your insights!  I have updated my Wiki page 
https://wiki.fysik.dtu.dk/niflheim/Slurm_installation#install-mariadb-database 
now.

/Ole

On 11/12/19 8:52 AM, Daniel Letai wrote:
> 
> On 11/12/19 9:34 AM, Ole Holm Nielsen wrote:
>> On 11/11/19 10:14 PM, Daniel Letai wrote:
>>> Why would you need galera-4 as a build require?
>>
>> This is the MariaDB recommendation in 
>> https://mariadb.com/kb/en/library/yum/, see the section "Installing 
>> MariaDB Packages with YUM".  I have no clue why this would be needed.
> 
> Yes, it's required for mariadb multimaster cluster. This has nothing to do 
> with the mariadb api required for linking against mariadb libs.
> 
> 
> You don't even need the mariadb-server pkg for build purposes - it's only 
> required for deployment of slurmdbd.
> 
> On a build machine, you should only require the client section: 
> https://mariadb.com/kb/en/library/yum/#installing-mariadb-clients-and-client-libraries-with-yum, 
> as well as the devel pkg.
> 
>>
>> /Ole
>>
>>
>>> If it's required by any of the mariadb packages, it'll get pulled 
>>> automatically. If not, you don't need it on the build system.
>>>
>>>
>>> On 11/11/19 10:56 PM, Ole Holm Nielsen wrote:
>>>> Hi William,
>>>>
>>>> Interesting experiences with MariaDB 10.4!  I tried to collect the 
>>>> instructions from the MariaDB page, but I'm unsure about how to get 
>>>> the galera-4 RPM.
>>>>
>>>> Could you kindly review and correct my updated instructions?
>>>> https://wiki.fysik.dtu.dk/niflheim/Slurm_installation#build-slurm-rpms
>>>>
>>>> That said, what are the main reasons for installing MariaDB 10 in 
>>>> stead of the 5.5 delivered by RedHat?  I'm not sure how well SchedMD 
>>>> has tested MariaDB 10 with Slurm?
>>>>
>>>> /Ole
>>>>
>>>>
>>>> On 11-11-2019 21:23, William Brown wrote:
>>>>> I have in fact found the answer by looking harder.
>>>>>
>>>>> The config.log clearly showed that the build of the test MySQL 
>>>>> program failed, which is why it was set to be excluded.
>>>>>
>>>>> It failed to link against '-lmariadb'.  It turns out that library is 
>>>>> no longer in MariaDB or MariaDB-devel, it is separately packaged in 
>>>>> MariaDB-shared.  That may of course be because I have built MariaDB 
>>>>> 10.4 from the mariadb.org site, because CentOS 7 only ships with the 
>>>>> extremely old version 5.5.
>>>>>
>>>>> Once I installed the missing package it built the RPMs just fine.  
>>>>> However it would be easier to use it linked to static MariaDB 
>>>>> libraries, as I now have to installed MariaDB-shared on every server 
>>>>> that will run slurmd, i.e. all compute nodes.  I expect that if I 
>>>>> looked harder at the build options there may be a way to do this, 
>>>>> perhaps with linker flags.
>>>>>
>>>>> For now, I can progress.
>>>>>
>>>>> Thanks
>>>>>
>>>>> William
>>>>>
>>>>> -----Original Message-----
>>>>> From: slurm-users <slurm-users-bounces at lists.schedmd.com> On Behalf 
>>>>> Of Ole Holm Nielsen
>>>>> Sent: 11 November 2019 20:02
>>>>> To: slurm-users at lists.schedmd.com
>>>>> Subject: Re: [slurm-users] RPM build error - accounting_storage_mysql.so
>>>>>
>>>>> Hi,
>>>>>
>>>>> Maybe my Slurm Wiki can help you build SLurm on CentOS/RHEL 7? See 
>>>>> https://wiki.fysik.dtu.dk/niflheim/Slurm_installation#build-slurm-rpms
>>>>>
>>>>> Note in particular:
>>>>>> Important: Install the MariaDB (a replacement for MySQL) packages 
>>>>>> before you build Slurm RPMs (otherwise some libraries will be missing):
>>>>>>
>>>>>> yum install mariadb-server mariadb-devel
>>>>>
>>>>> /Ole
>>>>>
>>>>>
>>>>> On 11-11-2019 15:22, William Brown wrote:
>>>>>> Fabio
>>>>>>
>>>>>> Did you ever resolve the problem building accounting_storage_mysql.so?
>>>>>>
>>>>>> I have the exact same problem with CentOS 7.6, building Slurm 19.05.03.
>>>>>> My command:
>>>>>>
>>>>>> rpmbuild -ta slurm-19.05.3-2.tar.bz2 | tee /var/tmp/slurm-build.log
>>>>>>
>>>>>> The directory with the plugin source is all there:
>>>>>> /home/users/slurm/rpmbuild/BUILD/slurm-19.05.3-2/src/plugins/accountin
>>>>>> g_storage/mysql, with a Makefile that is the same date/time as the
>>>>>> other accounting_storage alternatives.
>>>>>>
>>>>>> In the log I can see:
>>>>>>
>>>>>> checking for mysql_config... /usr/bin/mysql_config
>>>>>>
>>>>>> Looking at the process of building the RPMs it looks as if it has
>>>>>> skipped trying to create the missing library file, but then expects to
>>>>>> find it in the RPM.
>>>>>>
>>>>>> This is what I see when it is building, it builds the
>>>>>> accounting_storage .so files for _fileext, _none and _slurmdbd, but
>>>>>> not for _mysql.  I do have MariaDB-devel 10.4.10 installed
>>>>>>
>>>>>> .
>>>>>>
>>>>>> .
>>>>>>
>>>>>> Making all in mysql
>>>>>>
>>>>>> make[5]: Entering directory
>>>>>> `/home/users/slurm/rpmbuild/BUILD/slurm-19.05.3-2/src/plugins/accounting_storage/mysql' 
>>>>>>
>>>>>>
>>>>>> make[5]: Nothing to be done for `all'.
>>>>>>
>>>>>> make[5]: Leaving directory
>>>>>> `/home/users/slurm/rpmbuild/BUILD/slurm-19.05.3-2/src/plugins/accounting_storage/mysql' 
>>>>>>
>>>>>>
>>>>>> .
>>>>>>
>>>>>> .
>>>>>>
>>>>>> Making install in mysql
>>>>>>
>>>>>> make[4]: Entering directory
>>>>>> `/home/users/slurm/rpmbuild/BUILD/slurm-19.05.3-2/src/plugins/accounting_storage/mysql' 
>>>>>>
>>>>>>
>>>>>> make[5]: Entering directory
>>>>>> `/home/users/slurm/rpmbuild/BUILD/slurm-19.05.3-2/src/plugins/accounting_storage/mysql' 
>>>>>>
>>>>>>
>>>>>> make[5]: Nothing to be done for `install-data-am'.
>>>>>>
>>>>>> make[5]: Leaving directory
>>>>>> `/home/users/slurm/rpmbuild/BUILD/slurm-19.05.3-2/src/plugins/accounting_storage/mysql' 
>>>>>>
>>>>>>
>>>>>> make[4]: Leaving directory
>>>>>> `/home/users/slurm/rpmbuild/BUILD/slurm-19.05.3-2/src/plugins/accounting_storage/mysql' 
>>>>>>
>>>>>>
>>>>>> The error stream shows the problem a little later:
>>>>>>
>>>>>> Processing files: slurm-19.05.3-2.el7.x86_64
>>>>>>
>>>>>> warning: File not found:
>>>>>> /home/users/slurm/rpmbuild/BUILDROOT/slurm-19.05.3-2.el7.x86_64/home/a
>>>>>> pps/slurm/lib64/slurm/accounting_storage_mysql.so
>>>>>>
>>>>>> .
>>>>>>
>>>>>> .
>>>>>>
>>>>>> Processing files: slurm-slurmdbd-19.05.3-2.el7.x86_64
>>>>>>
>>>>>> error: File not found:
>>>>>> /home/users/slurm/rpmbuild/BUILDROOT/slurm-19.05.3-2.el7.x86_64/home/a
>>>>>> pps/slurm/lib64/slurm/accounting_storage_mysql.so
>>>>>>
>>>>>> File not found:
>>>>>> /home/users/slurm/rpmbuild/BUILDROOT/slurm-19.05.3-2.el7.x86_64/home/a
>>>>>> pps/slurm/lib64/slurm/accounting_storage_mysql.so
>>>>>>
>>>>>> File not found:
>>>>>> /home/users/slurm/rpmbuild/BUILDROOT/slurm-19.05.3-2.el7.x86_64/home/a
>>>>>> pps/slurm/lib64/slurm/accounting_storage_mysql.so
>>>>>>
>>>>>> RPM build errors:
>>>>>>
>>>>>> I looked at the Makefile for the accounting_storage_mysql as compared
>>>>>> to the Makefile for accounting_storage_slurmdbd to try to understand 
>>>>>> them.
>>>>>>
>>>>>> This is a selection from the Makefile for mysql:
>>>>>>
>>>>>> all: all-am
>>>>>>
>>>>>> all-am: Makefile $(LTLIBRARIES)
>>>>>>
>>>>>> LTLIBRARIES = $(pkglib_LTLIBRARIES)
>>>>>>
>>>>>> #pkglib_LTLIBRARIES = accounting_storage_mysql.la
>>>>>> <http://accounting_storage_mysql.la>
>>>>>>
>>>>>> And this is for slurmdbd:
>>>>>>
>>>>>> all: all-am
>>>>>>
>>>>>> all-am: Makefile $(LTLIBRARIES)
>>>>>>
>>>>>> LTLIBRARIES = $(pkglib_LTLIBRARIES)
>>>>>>
>>>>>> pkglib_LTLIBRARIES = accounting_storage_slurmdbd.la
>>>>>> <http://accounting_storage_slurmdbd.la>
>>>>>>
>>>>>> So I think that the problem is that the definition of
>>>>>> pkglib_LTLIBRARIES is commented out in the accounting_storage_mysql
>>>>>> Makefile, hence nothing to build.
>>>>>>
>>>>>> Is that intended?  Is it a consequence of something in my environment?
>>