[slurm-users] remote license

[Davide DelVento]

Thanks Brian.

I am still perplexed. What is a database to install, administer,
patch, update, could break, be down, etc buying us? I see limited use
cases, e.g. a license server which does not provide the license
count/use in a parsable way, and that someone wants to use with
multiple SLURM installations (if it's on a single one, the local
license is perfect). Wouldn't it much, much easier for everybody if
one could specify a script (the bullet 1. you mentioned) inside SLURM,
and use the license server ITSELF as the authoritative source of
license count? Sure, it won't be perfect, e.g. race conditions in
license acquisition can still cause failures, but the database won't
be fixing that....
I must be missing something....

Alternatively, can one update the license count of local license with
a scontrol command, rather than changing the slurm.conf and
reconfigure? That could make what I say possible....

Thanks

On Fri, Sep 16, 2022 at 9:25 AM Brian Andrus <toomuchit at gmail.com> wrote:
>
> Davide,
>
> You have it pretty correct. While the database itself is not part of the
> slurm suite, slurmdbd (which would access the database) is.
>
> As far as writing something that keeps things updated, I'm sure many
> have done this. However, it would be unique to your installation. The
> specific number of licenses, naming them, what license server is being
> used, etc.
> All of that could easily be a few lines in a script that you have in a
> cron job or other trigger (eg prolog/epilog). You would just:
>
> 1) Read/parse current licenses/use (eg: if you are using flexlm, lmutil
> lmstat output)
> 2) Update the database (sacctmgr command)
>
> As you can see, that 1st step would be highly dependent on you and your
> environment. The 2nd step would be dependent on what things you are
> tracking within that.
>
> Brian Andrus
>
>
> On 9/16/2022 5:01 AM, Davide DelVento wrote:
> > So if I understand correctly, this "remote database" is something that
> > is neither part of slurm itself, nor part of the license server per
> > se, correct?
> >
> > Regarding the "if you got creative", has anybody on this list done
> > that already? I can't believe I'm the first one wanting this feature!
> > Matching the number in that database with the actual number the
> > license server knows would be extremely helpful! We use various
> > license servers (for various licensed software), so each one of them
> > would be useful. I can probably script/develop one of these myself,
> > but I am not sure I've got the time...
> >
> > Thanks!
> >
> > On Thu, Sep 15, 2022 at 6:04 PM Brian Andrus <toomuchit at gmail.com> wrote:
> >> So if you follow the links to: https://slurm.schedmd.com/licenses.html
> >> you should see the difference.
> >>
> >> Local licenses are just a counter that is setup in slurm.conf
> >> Remote liceneses are a counter in a database (the database is "remote"),
> >> so you can change/update it dynamically. So, you could change their
> >> allocation with a sacctmgr command. It is especially useful when you are
> >> managing multiple clusters that share licenses. You can allocate that a
> >> certain number are allowed by each cluster and change that if needed.
> >>
> >> If you got creative, you could keep the license count that is in the
> >> database updated to match the number free from flexlm to stop license
> >> starvation due to users outside slurm using them up so they really
> >> aren't available to slurm.
> >>
> >> Brian Andrus
> >>
> >>
> >> On 9/15/2022 3:34 PM, Davide DelVento wrote:
> >>> I am a bit confused by remote licenses.
> >>>
> >>> https://lists.schedmd.com/pipermail/slurm-users/2020-September/006049.html
> >>> (which is only 2 years old) claims that they are just a counter, so
> >>> like local licenses. Then why call them remote?
> >>>
> >>> Only a few days after, this
> >>> https://lists.schedmd.com/pipermail/slurm-users/2020-September/006081.html
> >>> appeared to imply (but not clearly stated) that the remote license are
> >>> not simply a counter, but then it's not clear how they are different.
> >>>
> >>> The current documentation (and attempts to run the "add resource"
> >>> command) says that one must use the license count, which seems to
> >>> imply they are just a simple counter (but then what do they need the
> >>> server for)?
> >>>
> >>> So what is what?
> >>>
> >>> In my cursory past experience with this, it seemed that it were
> >>> possible to query a license server (at least some of them) to get the
> >>> actual number of available licenses and schedule (or let jobs pending)
> >>> accordingly. Which would be very helpful for the not-too-uncommon
> >>> situation in which the same license server provides licenses for both
> >>> the HPC cluster and other non-slurm-controlled resources, such a
> >>> user's workstations. Was that impression wrong, or perhaps somebody
> >>> scripted it in some way? If the latter, does anybody know if those
> >>> scripts are publicly available anywhere?
> >>>
> >>> Thanks
> >>>
>