[slurm-users] Slurm Job Error Output is Missing

[Maria Semple]

Hi all,

I have some jobs which write error messages to stderr, and I've noticed
that the stderr output is not being written to file. Here is a simple
reproduction case:

test.sh:
#!/bin/bash

echo "out"
echo "err" >&2
echo "err 2" 1>&2
>&2 echo "err 3"
echo "err 4" >/dev/stderr
echo "err 5" 1>/dev/stderr
echo "err 6" | tee /dev/stderr

When I run that script through srun, the output includes all the echoes:
out
err
err 2
err 3
err 4
err 5
err 6
err 6

If I run through sbatch without separating output and error into their own
files, I get only the last echo (err 6). If I run with separate output
files (e.g. sbatch --output slurm-%j.out --error slurm-%j.err), I get this:

slurm-<id>.out:
out
err 6

slurm-<id>.err:
err 6

I would expect to see all of the err lines in the slurm-<id>.err file. I've
tested this with Slurm 20.11.05 and Slurm 21.08.0 and the behaviour is the
same. Any assistance would be greatly appreciated!

-- 
Best Regards,
Maria
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20210913/1037230c/attachment.htm>