[slurm-users] Odd prolog Error?

[Brian Andrus]

 From the documentation:


*Parameter*

	

*Location*

	

*Invoked by*

	

*User*

	

*When executed*

Prolog (from slurm.conf)

	

Compute or front end node

	

slurmd daemon

	

SlurmdUser (normally user root)

	

First job or job step initiation on that node (by default); 
PrologFlags=Alloc will force the script to be executed at job allocation

So ensure:
1) /opt/slurm/prolog.sh exists on the node(s)
2) the slurmd user is able to execute it

I would connect to the node and try to run the command as the slurmd user.
Also, ensure the user exists on the node, however you are propagating 
the uids.

Brian ANdrus

On 4/11/2023 9:48 AM, Jason Simms wrote:
> Hello all,
>
> Regularly I'm seeing array jobs fail, and the only log info from the 
> compute node is this:
>
> [2023-04-11T11:41:12.336] error: /opt/slurm/prolog.sh: exited with 
> status 0x0100
> [2023-04-11T11:41:12.336] error: [job 26090] prolog failed status=1:0
> [2023-04-11T11:41:12.336] Job 26090 already killed, do not launch 
> batch job
>
> The contents of prolog.sh are incredibly simple:
>
> #!/bin/bash
> loginctl enable-linger $SLURM_JOB_USER
>
> I can't sort out what may be going on here. An example script from a 
> job that can result in this error is here:
>
> #!/bin/bash
> #SBATCH -t 2:00:00
> #SBATCH -n 1
> #SBATCH -N 1
> #SBATCH -p compute
> #SBATCH --array=1-100
> #SBATCH -o tempOut/MSO-%j-%a.log
>
> module load python3/python3
> python3 runVoltage.py $SLURM_ARRAY_TASK_ID
>
> Any insight would be welcome! This is really frustrating because it's 
> constantly causing nodes to drain.
>
> Warmest regards,
> Jason
>
> -- 
> *Jason L. Simms, Ph.D., M.P.H.*
> Manager of Research Computing
> Swarthmore College
> Information Technology Services
> (610) 328-8102
> Schedule a meeting: https://calendly.com/jlsimms
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20230411/f52511f9/attachment.htm>