<div dir="ltr"><div>Just for completition:</div><div>There has been a lock in the database when creating a table, you can see with </div><div><div id="gmail-content" class="gmail-ticket"><div><div id="gmail-changelog"><div class="gmail-change" id="gmail-trac-change-6-1573634732106612"><div class="gmail-comment gmail-searchable gmail-ticketdraft"><pre class="gmail-wiki">MariaDB [slurm_acct_db]> show full processlist;
+----+-------------+-----------+---------------+---------+------+---------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+
| Id | User | Host | db | Command | Time | State | Info | Progress |
+----+-------------+-----------+---------------+---------+------+---------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+
| 1 | system user | | NULL | Daemon | NULL | InnoDB purge coordinator | NULL | 0.000 |
| 3 | system user | | NULL | Daemon | NULL | InnoDB purge worker | NULL | 0.000 |
| 4 | system user | | NULL | Daemon | NULL | InnoDB purge worker | NULL | 0.000 |
| 2 | system user | | NULL | Daemon | NULL | InnoDB purge worker | NULL | 0.000 |
| 5 | system user | | NULL | Daemon | NULL | InnoDB shutdown handler | NULL | 0.000 |
| 11 | slurm | localhost | slurm_acct_db | Sleep | 943 | | NULL | 0.000 |
| 12 | slurm | localhost | slurm_acct_db | Sleep | 13 | | NULL | 0.000 |
| 20 | slurm | localhost | slurm_acct_db | Query | 307 | Waiting for table metadata lock | create table if not exists "mpi_ibk_event_table" (`time_start` bigint unsigned not null, `time_end` bigint unsigned default 0 not null, `node_name` tinytext default '' not null, `cluster_nodes` text not null default '', `reason` tinytext not null, `reason_uid` int unsigned default 0xfffffffe not null, `state` smallint unsigned default 0 not null, `tres` text not null default '', primary key (node_name(20), time_start)) engine='innodb' | 0.000 |
| 22 | root | localhost | slurm_acct_db | Query | 0 | init | show full processlist | 0.000 |
+----+-------------+-----------+---------------+---------+------+---------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+
<br>So this produced the second issue i think. The first issue is solved too, but it is not so clear why. My actual explanation is that I thought, systemctl restart mysql should restart the whole server (like postgres does ;)) but does not what it is thought to do. After a dedicated stop - start - procedure everything works like a charm. <br><br></pre><pre class="gmail-wiki">Thank you for your help!<br></pre></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Am Di., 12. Nov. 2019 um 04:45 Uhr schrieb Brian Andrus <<a href="mailto:toomuchit@gmail.com">toomuchit@gmail.com</a>>:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<p>That second one can happen as a race condition. It may be doing
an update or running a report or what-not when you ran your
command.</p>
<p>If the issue persists, restart mysql and slurmdbd.</p>
<p>Brian Andrus<br>
</p>
<div>On 11/11/2019 2:10 AM, Uwe Seher wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Hello!</div>
<div>I like zu use accounting via slurmdbd/mariadb and have some
problems with connection to the database.</div>
<div>When i try to connect via sacct or ascctmgr as a non-root
user the connection is completely refused:</div>
<div>
<pre style="margin-left:40px">sacctmgr: add cluster MPI_IBK
Adding Cluster(s)
Name = mpi_ibk
Would you like to commit changes? (You have 30 seconds to decide)
(N/y): y
Problem adding clusters: Access/permission denied
</pre>
<pre><font face="arial,sans-serif">I think this has something to do with the second problem, when trying to use sacctmgr as root. </font>
</pre>
<pre style="margin-left:40px">sacctmgr: add cluster name=mpi_ibk
Adding Cluster(s)
Name = mpi_ibk
Would you like to commit changes? (You have 30 seconds to decide)
(N/y): y
Database is busy or waiting for lock from other user.
</pre>
<pre><font face="arial,sans-serif">The first problem is caused by the lack of a configuration, as default only a user 'root' is configured in the database which can start some transactions. But for the second one i have no idea, the database is used only for slurm, i can log in with the configured user, all deamons are restarted and working fine.
The authentication inside slurm should work with the default munge service and i think this is also working in a kind of way, because the connection seems to be established. But i can not do any configuration, so no further logging is possible. Below are some further infomations.
</font></pre>
<pre><font face="arial,sans-serif">Thank you in advance for some hints concerning this issue.
</font></pre>
<pre><font face="arial,sans-serif">Regards
</font></pre>
<pre><font face="arial,sans-serif">Uwe Seher
</font></pre>
<pre><font face="arial,sans-serif">
</font></pre>
<pre><font face="arial,sans-serif">The accounting setup in slurm.conf is the following:
</font></pre>
<pre style="margin-left:40px"> # ACCOUNTING
JobAcctGatherType=jobacct_gather/linux
JobAcctGatherFrequency=30
# file
JobCompType=jobcomp/filetxt
JobCompLoc=/var/log/slurm_jobs.log
#AccountingStorageType=accounting_storage/filetxt
#AccountingStorageLoc=/var/log/slurm_acc.log
#slurmdb
AccountingStorageType=accounting_storage/slurmdbd
AccountingStorageHost=localhost
#AccountingStoragePass=*********
AccountingStorageUser=slurm
</pre>
<pre><font face="arial,sans-serif">sacctmgr show configuration shows this:
</font></pre>
<pre style="margin-left:40px"><font face="arial,sans-serif"><span style="font-family:monospace">sacctmgr show configuration</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">Configuration data as of 2019-11-11T10:58:04</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">AccountingStorageBackupHost = (null)</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">AccountingStorageHost = localhost</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">AccountingStorageLoc = N/A</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">AccountingStoragePass = (null)</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">AccountingStoragePort = 6819</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">AccountingStorageType = accounting_storage/slurmdbd</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">AccountingStorageUser = N/A</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">AuthType = auth/munge</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">MessageTimeout = 10 sec</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">PluginDir = /usr/lib64/slurm</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">PrivateData = none</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">SlurmUserId = slurm(400)</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">SLURM_CONF = /etc/slurm/slurm.conf</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">SLURM_VERSION = 17.11.13</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">TCPTimeout = 2 sec</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">TrackWCKey = 0</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace"></span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">SlurmDBD configuration:</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">ArchiveDir = /tmp</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">ArchiveEvents = No</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">ArchiveJobs = No</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">ArchiveResvs = No</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">ArchiveScript = (null)</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">ArchiveSteps = No</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">ArchiveSuspend = No</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">ArchiveTXN = No</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">ArchiveUsage = No</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">AuthInfo = (null)</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">AuthType = auth/munge</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">BOOT_TIME = 2019-11-11T09:29:01</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">CommitDelay = No</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">DbdAddr = localhost</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">DbdBackupHost = (null)</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">DbdHost = localhost</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">DbdPort = 6819</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">DebugFlags = (null)</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">DebugLevel = verbose</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">DebugLevelSyslog = quiet</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">DefaultQOS = (null)</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">LogFile = /var/log/slurmdbd.log</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">MaxQueryTimeRange = UNLIMITED</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">MessageTimeout = 10 secs</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">PidFile = /var/run/slurm/slurmdbd.pid</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">PluginDir = /usr/lib64/slurm</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">PrivateData = none</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">PurgeEventAfter = NONE</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">PurgeJobAfter = NONE</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">PurgeResvAfter = NONE</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">PurgeStepAfter = NONE</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">PurgeSuspendAfter = NONE</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">PurgeTXNAfter = NONE</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">PurgeUsageAfter = NONE</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">SLURMDBD_CONF = /etc/slurm/slurmdbd.conf</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">SLURMDBD_VERSION = 17.11.13</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">SlurmUser = slurm(400)</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">StorageBackupHost = (null)</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">StorageHost = localhost</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">StorageLoc = slurm_acct_db</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">StoragePort = 3306</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">StorageType = accounting_storage/mysql</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">StorageUser = slurm</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">TCPTimeout = 2 secs</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">TrackWCKey = No</span></font>
<font face="arial,sans-serif"><span style="font-family:monospace">TrackSlurmctldDown = No</span></font>
</pre>
<pre><font face="arial,sans-serif">
</font></pre>
</div>
</div>
</blockquote>
</div>
</blockquote></div>