[slurm-users] Licenses: Remote vs Reservation

[Richard Ems]

Hi Brian,

Thanks for your answer and sorry for my late reply.
Yes, the cluster is not large, perhaps medium sized? What is considered for
Slurm a small/medium/large cluster?

I think I would go for the remote license method, to avoid what you
mentioned about keeping the slurm.conf consistent on all nodes. It works
fine if all nodes are up, but what when nodes are down and come back up
with an outdated slurm.conf? Syncing on boot the current slurm.conf could
perhaps be an option.

This should work as it does now with local licenses: jobs get submitted and
stay pending until the needed licenses become available. No need for
license tracking.

The reservation method can cause an sbatch command to be rejected, if that
> is what you are looking for.

How so? If all licenses on the reservation are in use, will then a new job
requesting one of these licenses get rejected and not stay pending in the
queue?
Then clearly no, that is not what we are looking for!

Many thanks,
Richard

On Wed, 30 Nov 2022 at 12:46, Brian Andrus <toomuchit at gmail.com> wrote:

> Richard,
>
> If you don't have a large cluster, doing the local license method is
> actually feasible. The biggest issue is the efforts to ensure all the
> nodes have the same slurm.conf and then the traffic when they are all
> re-queried to report their status on a reconfigure.
>
> That said, I have had good experience with the remote license method.
> the info is kept as part of the slurmdbd, so you will need that setup
> for it to work.
>
> I imagine one of the factors may be the purpose of the license tracking.
> Is it to cause jobs to queue if there are no licenses available or not
> be allowed to submit at all? The reservation method can cause an sbatch
> command to be rejected, if that is what you are looking for.
>
> Brian Andrus
>
> On 11/30/2022 6:29 AM, Richard Ems wrote:
> > Hi all,
> >
> > I have to change our set up to be able to update the total number of
> > available licenses due to users checking out licenses interactively.
> > We now use Local Licenses, and could just regularly update slurm.conf
> > and reconfigure, but I don't think that is the best solution.
> >
> > I see there are at least two other options:
> > 1. Remote Licenses as per https://slurm.schedmd.com/licenses.html
> > 2. The "older" method of creating an infinite license only reservation
> > and letting jobs request license from there
> >
> > What are the pros and cons of both methods?
> > What would you suggest based on your experience?
> >
> > Many thanks for any input,
> > Richard
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20221201/0b9a66da/attachment.htm>