[slurm-users] License management and invoking scontrol in the prolog

[Davide DelVento]

Thanks Brian for the suggestion, which I am now exploring.

The documentation is a bit cryptic for me, but exploring a few things
and checking https://funinit.wordpress.com/2018/06/07/how-to-use-job_submit_lua-with-slurm/
I suspect my slurm install (provided by cluster vendor) was not
compiled with the lua plugin installed. Do you know how to verify if
that is the case or if it's something else? I don't see a way to show
if the plugin is actually being "seen" by slurm, and I suspect it's
not.

Does anyone else have other suggestions or comment on either the
plugin or the prolog workaround?

Thanks!


On Tue, Aug 30, 2022 at 3:01 PM Brian Andrus <toomuchit at gmail.com> wrote:
>
> Not sure if you can do all the things you intend, but the job_submit
> script is precisely where you want to check submission options.
>
> https://slurm.schedmd.com/job_submit_plugins.html
>
> Brian Andrus
>
> On 8/30/2022 12:58 PM, Davide DelVento wrote:
> > Hi,
> >
> > I would like to soft-enforce license utilization only when the -L is
> > set. My idea: check in the prolog if the license was requested and
> > only if it were, set the environmental variables needed for the
> > license.
> >
> > I looked at all environmental variables set by slurm and did not find
> > any related to the license as I was hoping.
> >
> > As a workaround, I could check
> >
> > scontrol show job $SLURM_JOB_ID | grep License
> >
> > and that would work, but (as discussed in other messages in this list)
> > the documentation at https://slurm.schedmd.com/prolog_epilog.html say
> >
> >> Prolog and Epilog scripts should be designed to be as short as possible
> >> and should not call Slurm commands (e.g. squeue, scontrol, sacctmgr,
> >> etc). [...] Slurm commands in these scripts can potentially lead to performance
> >> issues and should not be used.
> > This is a bit of a concern, since the prolog would be invoked for
> > every job on the cluster, and it's a prolog (rather than the epilogue
> > like discussed in earlier messages).
> >
> > So two questions:
> >
> > 1) is there a better workaround to check in the prolog if the current
> > job requested a license and/or
> > 2) would this kind of use of scontrol be okay or is indeed a concern
> >
> > Thanks!
> >
>