[slurm-users] Re: How can we put limits on interactive jobs?

Hi all,

Thanks for the great suggestions! It seems that the Slurm job_submit.lua script is the most flexible way to check for interactive jobs, and change job parameters such as QOS, time_limit etc.

I've added this Lua function to our job_submit.lua script and it seems to work fine:

-- Check for interactive jobs -- Policy: Interactive jobs are limited to 4 hours function check_interactive_job (job_desc, submit_uid, log_prefix) if (job_desc.script == nil or job_desc.script == '') then local time_limit = 240 slurm.log_info("%s: user %s submitted an interactive job", log_prefix, userinfo) slurm.log_user("NOTICE: Job script is missing, assuming an interactive job") slurm.log_user(" Job timelimit is set to %d minutes", time_limit) job_desc.time_limit = time_limit end return slurm.SUCCESS end

The complete script is available at https://github.com/OleHolmNielsen/Slurm_tools/blob/master/plugins/job_submit...

Interestingly, Slurm by default (we're at 24.11.4) assigns job_desc.job_name="interactive" to interactive jobs submitted by salloc, from the manual page:

The default job name is the name of the "command" specified on the command line.

Users can of course override this with the --job-name parameter.

Best regards, Ole

On 4/25/25 11:37, ewan.roche@agroscope.admin.ch wrote:

Hello Ole, the way I identify interactive jobs is by checking that the script is empty in job_submit.lua.

If it's the case then they're assigned to an interactive QoS that limits the time and resources as well as only allowing one job per user.

     if job_desc.script == nil or job_desc.script == '' then

         slurm.log_info("slurm_job_submit: jobscript is missing, assuming interactive job")
         slurm.log_user("Launching an interactive job")

         if job_desc.partition == "gpu" then 
             job_desc.qos = "gpu_interactive"
         end

         if job_desc.partition == "cpu" then
             job_desc.qos = "cpu_interactive"
         end

         return slurm.SUCCESS

Thanks

Ewan

-----Original Message----- From: Ole Holm Nielsen via slurm-users slurm-users@lists.schedmd.com Sent: Freitag, 25. April 2025 11:15 To: slurm-users@schedmd.com Subject: [slurm-users] How can we put limits on interactive jobs?

We would like to put limits on interactive jobs (started by salloc) so that users don't leave unused interactive jobs behind on the cluster by mistake.

I can't offhand find any configurations that limit interactive jobs, such as enforcing a timelimit.

Perhaps this could be done in job_submit.lua, but I couldn't find any job_desc parameters in the source code which would indicate if a job is interactive or not.

Question: How do people limit interactive jobs, or identify orphaned jobs and kill them?

Thanks a lot, Ole

-- Ole Holm Nielsen PhD, Senior HPC Officer Department of Physics, Technical University of Denmark

-- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-leave@lists.schedmd.com