[slurm-users] Adding Users to Slurm's Database

[Christoph Brüning]

Hi,

On 18/08/2020 17.56, Tina Friedrich wrote:
> Script. Not doing manual anything if it can at all be avoided, way to 
> error prone.
> 
> We have a cron job that does all of that. Checks if there are users or 
> groups in LDAP that aren't in SLURM yet, and adds them - that's adding 
> accounts, adding users, .... I think it also removed users/accounts if 
> necessary, and also handles user<->account associations, as those can 
> change (we allow users in multiple groups), and it makes a valient 
> attempt at changing default associations if they need changing, I believe.
> 
> I think it runs every 15 minutes.
> 
> The problem/pitfall that I can see is if a script is coded to, 
> basically, loop through all users running sacctmgr for each to check it 
> exists. We very definitely do not do that - the script gets all users, 
> accounts & associations from SLURM and the equivalent from LDAP and 
> then, basically, does a couple of list compares.
> 

we're doing pretty much the same.

We have a Python script that fetches users and groups from LDAP as well 
as current SLurmDB accounts and users
(single "sacctmgr list accounts withassoc" and "sacctmgr list "users")
and then does a bunch of list compares "offline".
It is run as a cron job every 20 minutes.

The script builds a "target" account tree from LDAP based on the 
following points:

- LDAP groups have an HPC entitlement flag
- Non-leaf accounts in Slurm's account tree are read from a config file. 
They have no users assigned and are used for FairShare propagation.
- Leaf accounts are generated from LDAP groups, Slurm account names are 
autogenerated from the group's GID
- LDAP groups have an additional field with an hierarchical string to 
denote their position within the university's hierarchy (like "08-05-02" 
meaning chemistry department -> physical chemistry -> some professor's 
workgroup). We use that to determine the parent of the generated 
account: Non-leaf accounts have a list of these organizational unit 
string prefixes that belong to it, and the longest, i.e. most specific, 
match wins.

- LDAP users also have an HPC entitlement flag and an additional field 
for the HPC-internal primary GID, only users with both fields reasonably 
set are further considered
- LDAP group membership equals Slurm account membership
- Primary GID equals default account

It then compares this to the what it got from SlurmDB and writes the 
changes back to SlurmDB.
This might fail when some entitlement flag or primary GID field gets 
erased, but I have not seen this happen yet. We do, of course, have 
daily backups of the MySQL DB.

The script does not touch any FairShare values for associations or QoS 
entitlements, those are set manually.

I have to mention that our script is heavily influenced by Ole Holm 
Nielsen's work. So, thanks a lot, Ole! :-)


Cheers,
Christoph

-- 
Dr. Christoph Brüning
Universität Würzburg
Rechenzentrum
Am Hubland
D-97074 Würzburg
Tel.: +49 931 31-80499