Good morning,
yesterday I came across a Slurm (sbatch) script that, after doing some stuff in the foreground, runs another executable in the background - and doesn't "wait" for it to finish - literally the last line of the script is
executable &
(and that executable is supposed to take several 10 seconds or more to finish)
How would Slurm handle this? Will the end of the script immediately trigger the job epilog, and what would happen to the leftover task? This certainly is discussed somewhere in the manual pages and other documentation but up to now I failed to find that place...
Thanks, Steffen
Good Morning;
This is not a slurm issue. This is a default shell script feature. If you want to wait to finish until all background processes, you should use wait command after all.
Regards;
C. Ahmet Mercan
On 26.07.2024 10:23, Steffen Grunewald via slurm-users wrote:
Good morning,
yesterday I came across a Slurm (sbatch) script that, after doing some stuff in the foreground, runs another executable in the background - and doesn't "wait" for it to finish - literally the last line of the script is
executable &
(and that executable is supposed to take several 10 seconds or more to finish)
How would Slurm handle this? Will the end of the script immediately trigger the job epilog, and what would happen to the leftover task? This certainly is discussed somewhere in the manual pages and other documentation but up to now I failed to find that place...
Thanks, Steffen
On Fri, 2024-07-26 at 10:42:45 +0300, Slurm users wrote:
Good Morning;
This is not a slurm issue. This is a default shell script feature. If you want to wait to finish until all background processes, you should use wait command after all.
Thank you - I already knew this in principle, and I also know that a login shell will complain at an attempt to exit when there are leftover background jobs. I was wondering though how Slurm's task control would react... Got to try myself, I guess...
Best, S
Generally speaking, when the batch script exits, slurm will clean up (ie kill) any stray processes. So, I would expect that executable to be killed at cleanup.
Brian Andrus
On 7/26/2024 2:45 AM, Steffen Grunewald via slurm-users wrote:
On Fri, 2024-07-26 at 10:42:45 +0300, Slurm users wrote:
Good Morning;
This is not a slurm issue. This is a default shell script feature. If you want to wait to finish until all background processes, you should use wait command after all.
Thank you - I already knew this in principle, and I also know that a login shell will complain at an attempt to exit when there are leftover background jobs. I was wondering though how Slurm's task control would react... Got to try myself, I guess...
Best, S
-
Brian Andrus -
mercan -
Steffen Grunewald