[slurm-users] Decreasing time limit of running jobs (notification)

Thu Jul 6 16:24:32 UTC 2023

Is the issue that the error in the time made it shorter than the time the job had already run, so it killed it immediately?

On Jul 6, 2023, at 12:04 PM, Jason Simms <jsimms1 at swarthmore.edu<mailto:jsimms1 at swarthmore.edu>> wrote:

No, not a bug, I would say. When the time limit is reached, that's it, job dies. I wouldn't be aware of a way to manage that. Once the time limit is reached, it wouldn't be a hard limit if you then had to notify the user and then... what? How long would you give them to extend the time? Wouldn't be much of a limit if a job can be extended, plus that would throw off the scheduler/estimator. I'd chalk it up to an unfortunate typo.

Jason

On Thu, Jul 6, 2023 at 11:54 AM Amjad Syed <amjadcsu at gmail.com<mailto:amjadcsu at gmail.com>> wrote:
Hello

We were trying to increase the time limit of a slurm running job

scontrol update job=<jobid> TimeLimit=16-00:00:00

But we accidentally got it to 16 hours

scontrol update job=<jobid> TimeLimit=16:00:00

This actually timeout and killed the running job and did not give any notification

Is this a bug, should not the user be warned that this job will be killled ?

Amjad

--
Jason L. Simms, Ph.D., M.P.H.
Manager of Research Computing
Swarthmore College
Information Technology Services
(610) 328-8102
Schedule a meeting: https://calendly.com/jlsimms

        U.S. NAVAL

        RESEARCH

        LABORATORY
Noam Bernstein, Ph.D.
Center for Materials Physics and Technology
U.S. Naval Research Laboratory
T +1 202 404 8628 F +1 202 404 7546
https://www.nrl.navy.mil

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20230706/f2ac04b1/attachment.htm>