[slurm-users] License management and invoking scontrol in the prolog

[Davide DelVento]

Thanks.

I did try a lua script as soon as I got your first email, but that
never worked (yes, I enabled it in slurm.conf and ran "scontrol
reconfigure" after). Slurm simply acted as if there was no job_submit script.

After various tests, all unsuccessful, today I found that link which I
mentioned saying that lua might not be compiled in, hence all my most
recent messages of this thread.

That file is indeed there, so that's good news that I don't need to recompile.
However I'm puzzled on what might be missing...


On Thu, Sep 1, 2022 at 6:33 PM Brian Andrus <toomuchit at gmail.com> wrote:
>
> lua is the language you can use with the job_submit plugin.
>
> I was showing a quick way to see that job_submit capability is indeed in
> there.
>
> You can see if lua support is there by looking for the job_submit_lua.so
> file is there.
> It would be part of the slurm rpm (not the slurm-slurmctl rpm)
>
> Usually it would be found at /usr/lib64/slurm/job_submit_lua.so
>
> If that is there, you should be good with trying out a job_submit lua
> script.
>
> Brian Andrus
>
> On 9/1/2022 1:24 PM, Davide DelVento wrote:
> > Thanks again, Brian, indeed that grep returns many hits, but none of
> > them includes lua, i.e.
> >
> >   strings `which slurmctld ` | grep -i job_submit | grep -i lua
> >
> > returns nothing. So I should use the C rather than the more convenient
> > lua interface, unless I recompile or am I missing something?
> >
> > On Thu, Sep 1, 2022 at 12:30 PM Brian Andrus <toomuchit at gmail.com> wrote:
> >> I would be surprised if it were compiled without the support. However,
> >> you could check and run something like:
> >>
> >> strings /sbin/slurmctld | grep job_submit
> >>
> >> (or where ever your slurmctld binary is). There should be quite a few
> >> lines with that in it.
> >>
> >> Brian Andrus
> >>
> >> On 9/1/2022 10:54 AM, Davide DelVento wrote:
> >>> Thanks Brian for the suggestion, which I am now exploring.
> >>>
> >>> The documentation is a bit cryptic for me, but exploring a few things
> >>> and checking https://funinit.wordpress.com/2018/06/07/how-to-use-job_submit_lua-with-slurm/
> >>> I suspect my slurm install (provided by cluster vendor) was not
> >>> compiled with the lua plugin installed. Do you know how to verify if
> >>> that is the case or if it's something else? I don't see a way to show
> >>> if the plugin is actually being "seen" by slurm, and I suspect it's
> >>> not.
> >>>
> >>> Does anyone else have other suggestions or comment on either the
> >>> plugin or the prolog workaround?
> >>>
> >>> Thanks!
> >>>
> >>>
> >>> On Tue, Aug 30, 2022 at 3:01 PM Brian Andrus <toomuchit at gmail.com> wrote:
> >>>> Not sure if you can do all the things you intend, but the job_submit
> >>>> script is precisely where you want to check submission options.
> >>>>
> >>>> https://slurm.schedmd.com/job_submit_plugins.html
> >>>>
> >>>> Brian Andrus
> >>>>
> >>>> On 8/30/2022 12:58 PM, Davide DelVento wrote:
> >>>>> Hi,
> >>>>>
> >>>>> I would like to soft-enforce license utilization only when the -L is
> >>>>> set. My idea: check in the prolog if the license was requested and
> >>>>> only if it were, set the environmental variables needed for the
> >>>>> license.
> >>>>>
> >>>>> I looked at all environmental variables set by slurm and did not find
> >>>>> any related to the license as I was hoping.
> >>>>>
> >>>>> As a workaround, I could check
> >>>>>
> >>>>> scontrol show job $SLURM_JOB_ID | grep License
> >>>>>
> >>>>> and that would work, but (as discussed in other messages in this list)
> >>>>> the documentation at https://slurm.schedmd.com/prolog_epilog.html say
> >>>>>
> >>>>>> Prolog and Epilog scripts should be designed to be as short as possible
> >>>>>> and should not call Slurm commands (e.g. squeue, scontrol, sacctmgr,
> >>>>>> etc). [...] Slurm commands in these scripts can potentially lead to performance
> >>>>>> issues and should not be used.
> >>>>> This is a bit of a concern, since the prolog would be invoked for
> >>>>> every job on the cluster, and it's a prolog (rather than the epilogue
> >>>>> like discussed in earlier messages).
> >>>>>
> >>>>> So two questions:
> >>>>>
> >>>>> 1) is there a better workaround to check in the prolog if the current
> >>>>> job requested a license and/or
> >>>>> 2) would this kind of use of scontrol be okay or is indeed a concern
> >>>>>
> >>>>> Thanks!
> >>>>>
>