[slurm-users] Jobs that may still be running at X time?

[Ryan Novosielski]

Hi there,

Does anyone have a particularly clever way, either built-in or scripted, to find out which jobs will still be running at such-and-such time? I bet anyone who’s made the mistake of not entering a maintenance reservation soon enough knows the feeling.

I know that jobs /may/ end earlier, but we’re looking for the potential.

I originally started looking for jobs with an EndTime= that overlapped with the maintenance, using "scontrol show job” and excluding jobs that had a StartTime= that was after the maintenance. Today I had a minor panic as many many new jobs had an EndTime= that overlapped with the maintenance, but a StartTime= that was after we put in the maintenance reservation but before the maintenance period. Come to find they were still pending. Not sure why they have a StartTime= that is before the reservation, but it appears as if they won’t be running.

Anyway, I figure this is something people probably need to know often enough. Any tips?

--
#BlackLivesMatter
____
|| \\UTGERS,  	 |---------------------------*O*---------------------------
||_// the State	 |         Ryan Novosielski - novosirj at rutgers.edu
|| \\ University | Sr. Technologist - 973/972.0922 (2x0922) ~*~ RBHS Campus
||  \\    of NJ	 | Office of Advanced Research Computing - MSB C630, Newark
     `'