[slurm-users] License management and invoking scontrol in the prolog

[Brian Andrus]

lua is the language you can use with the job_submit plugin.

I was showing a quick way to see that job_submit capability is indeed in 
there.

You can see if lua support is there by looking for the job_submit_lua.so 
file is there.
It would be part of the slurm rpm (not the slurm-slurmctl rpm)

Usually it would be found at /usr/lib64/slurm/job_submit_lua.so

If that is there, you should be good with trying out a job_submit lua 
script.

Brian Andrus

On 9/1/2022 1:24 PM, Davide DelVento wrote:
> Thanks again, Brian, indeed that grep returns many hits, but none of
> them includes lua, i.e.
>
>   strings `which slurmctld ` | grep -i job_submit | grep -i lua
>
> returns nothing. So I should use the C rather than the more convenient
> lua interface, unless I recompile or am I missing something?
>
> On Thu, Sep 1, 2022 at 12:30 PM Brian Andrus <toomuchit at gmail.com> wrote:
>> I would be surprised if it were compiled without the support. However,
>> you could check and run something like:
>>
>> strings /sbin/slurmctld | grep job_submit
>>
>> (or where ever your slurmctld binary is). There should be quite a few
>> lines with that in it.
>>
>> Brian Andrus
>>
>> On 9/1/2022 10:54 AM, Davide DelVento wrote:
>>> Thanks Brian for the suggestion, which I am now exploring.
>>>
>>> The documentation is a bit cryptic for me, but exploring a few things
>>> and checking https://funinit.wordpress.com/2018/06/07/how-to-use-job_submit_lua-with-slurm/
>>> I suspect my slurm install (provided by cluster vendor) was not
>>> compiled with the lua plugin installed. Do you know how to verify if
>>> that is the case or if it's something else? I don't see a way to show
>>> if the plugin is actually being "seen" by slurm, and I suspect it's
>>> not.
>>>
>>> Does anyone else have other suggestions or comment on either the
>>> plugin or the prolog workaround?
>>>
>>> Thanks!
>>>
>>>
>>> On Tue, Aug 30, 2022 at 3:01 PM Brian Andrus <toomuchit at gmail.com> wrote:
>>>> Not sure if you can do all the things you intend, but the job_submit
>>>> script is precisely where you want to check submission options.
>>>>
>>>> https://slurm.schedmd.com/job_submit_plugins.html
>>>>
>>>> Brian Andrus
>>>>
>>>> On 8/30/2022 12:58 PM, Davide DelVento wrote:
>>>>> Hi,
>>>>>
>>>>> I would like to soft-enforce license utilization only when the -L is
>>>>> set. My idea: check in the prolog if the license was requested and
>>>>> only if it were, set the environmental variables needed for the
>>>>> license.
>>>>>
>>>>> I looked at all environmental variables set by slurm and did not find
>>>>> any related to the license as I was hoping.
>>>>>
>>>>> As a workaround, I could check
>>>>>
>>>>> scontrol show job $SLURM_JOB_ID | grep License
>>>>>
>>>>> and that would work, but (as discussed in other messages in this list)
>>>>> the documentation at https://slurm.schedmd.com/prolog_epilog.html say
>>>>>
>>>>>> Prolog and Epilog scripts should be designed to be as short as possible
>>>>>> and should not call Slurm commands (e.g. squeue, scontrol, sacctmgr,
>>>>>> etc). [...] Slurm commands in these scripts can potentially lead to performance
>>>>>> issues and should not be used.
>>>>> This is a bit of a concern, since the prolog would be invoked for
>>>>> every job on the cluster, and it's a prolog (rather than the epilogue
>>>>> like discussed in earlier messages).
>>>>>
>>>>> So two questions:
>>>>>
>>>>> 1) is there a better workaround to check in the prolog if the current
>>>>> job requested a license and/or
>>>>> 2) would this kind of use of scontrol be okay or is indeed a concern
>>>>>
>>>>> Thanks!
>>>>>