[slurm-users] How to debug a prolog script?

[Davide DelVento]

Thanks a lot.

> > Does it need the execution permission? For root alone sufficient?
>
> slurmd runs as root, so it only need exec perms for root.

Perfect. That must have been then, since my script (like the example
one) did not have the execution permission on.

> I'm curious: What kind of disruption did it cause for your production
> jobs?

All jobs failed and went in pending/held with "launch failed requeued
held" status, all nodes where the jobs were scheduled went draining.

The logs only said "error: validate_node_specs: Prolog or job env
setup failure on node xxxx, draining the node". I guess if they said
"-bash: /path/to/prolog: Permission denied" I would have caught the
problem myself.

In hindsight it is obvious, but I don't think even the documentation
mentions that, does it? After all you can execute a file with a
non-executable with with "sh filename", so I made the incorrect
assumption that slurm would have invoked the prolog that way.

Thanks!