<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri",sans-serif;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=EN-GB link="#0563C1" vlink="#954F72"><div class=WordSection1><p class=MsoNormal><span style='mso-fareast-language:EN-US'>The srun man page says:<o:p></o:p></span></p><p class=MsoNormal><span style='mso-fareast-language:EN-US'><o:p> </o:p></span></p><p class=MsoNormal>When initiating remote processes <b>srun</b> will propagate the current working directory, unless <b>--chdir</b>=<<i>path</i>> is specified, in which case <i>path</i> will become the working directory for the remote processes.<o:p></o:p></p><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal>William<span style='mso-fareast-language:EN-US'><o:p></o:p></span></p><p class=MsoNormal><span style='mso-fareast-language:EN-US'><o:p> </o:p></span></p><p class=MsoNormal><b><span lang=EN-US>From:</span></b><span lang=EN-US> slurm-users <slurm-users-bounces@lists.schedmd.com> <b>On Behalf Of </b>Dean Schulze<br><b>Sent:</b> 21 January 2020 19:27<br><b>To:</b> Slurm User Community List <slurm-users@lists.schedmd.com><br><b>Subject:</b> [slurm-users] sbatch sending the working directory from the controller to the node<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>I run this sbatch script from the controller:<br><br>=======================<br>#!/bin/bash<br>#SBATCH --job-name=test_job<br>#SBATCH --mail-type=NONE # Mail events (NONE, BEGIN, END, FAIL, ALL)<br>#SBATCH --ntasks=1<br>#SBATCH --mem=1gb<br>#SBATCH --time=00:05:00 # Time limit hrs:min:sec<br>#SBATCH --output=test_job_%j.log # Standard output and error log<br><br>pwd; hostname; date<br>=======================<br><br><br>The node gets the directory that sbatch was executed from on the controller and tries to write the output file to that directory, which doesn't exist on the node. The node slurmd.log shows this error:<br><br>2020-01-21T11:25:36.389] [7.batch] error: Could not open stdout file /home/dean/src/slurm.example.scripts/serial_test_7.log: No such file or directory<br><br><br>If I change the sbatch script --output to a fully qualified directory that exists on the node<br><br> --output=/home/nodeuser/serial_test_%j.log<br><br>the output file is written to that directory, but it includes this error showing that the slurm node is trying to execute the job in the directory that sbatch was run from on the controller:<br><br>slurmstepd: error: couldn't chdir to `/home/dean/src/slurm.example.scripts': No such file or directory: going to /tmp instead<br><br><br>The sbatch docs say nothing about why the node gets the pwd from the controller. Why would slurm send a directory to a node that may not exist on the node and expect it to use it?<o:p></o:p></p><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>What's the right way to specify the --output directory in an sbatch script?<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'>Thanks.<br><br><br><o:p></o:p></p></div></div></div></body></html>