[slurm-users] Problem with sbatch

[Jeffrey Frey]

"-uid" is a perfectly valid sbatch flag:


       -u, --usage
              Display brief help message and exit.

       -i, --input=<filename pattern>
              Instruct Slurm to connect the batch script's standard input directly to the file name specified in  the
              "filename pattern".

              By default, "/dev/null" is open on the batch script's standard input and both standard output and stan‐
              dard error are directed to a file of the name "slurm-%j.out", where the "%j" is replaced with  the  job
              allocation number, as described below in the filename pattern section.


What you've specified with "-uid" is:


--usage --input=d


Since the "--usage" appears in the script header and not from the cli args, it's ignored.  That leaves just


--input=d


So your job script's STDIN is connected to a file named "d" in the working directory at time of submission:


JobId=596862 JobName=x.qs
     :
   StdErr=/home/1001/slurm-596862.out
   StdIn=/home/1001/d
   StdOut=/home/1001/slurm-596862.out
   Power=


> On Jul 9, 2019, at 11:13 AM, Daniel Torregrosa <daniel.torregrosa at insight-centre.org> wrote:
> 
> Slight correction, it does not look for a file named "d" in the home folder of the user in the (mistyped) -uid parameter, it looks for a file named "d" in the home folder of the user running sbatch. If this is not an expected behaviour, I will make a complete bug report.
> 
> 
> 
> On Tue, 9 Jul 2019 at 15:53, Daniel Torregrosa <daniel.torregrosa at insight-centre.org> wrote:
> Thanks a lot for your answers again!
> 
> @Marcus Thanks a lot for the clarification. 
> 
> About --uid, you are correct, I was mistyping it as -uid. But, the behaviour is slightly inconsistent:
> * If correctly typed (--uid) sbatch properly complains about needing to be root
> * If not present at all, or ignored (by adding a non-commented line before, like you said), everything goes fine
> * If incorrectly typed (-uid UID), it silently fails UNLESS /home/UID/d exists, then it is run as the requested user, i.e. if I add 
> 
> #SBATCH -uid test2
> 
> the log complains about /home/test2/d not existing. After creating that file as test2 (that is, the file /home/test2/d is -rw-r--r-- test2:test2), it executes the task as test (i.e. the output is, by default, in /home/test and owned by test). I guess this is a bug?
> 
> @Jeffrey Sorry, slurmdUser=sudo was a typo. Thanks a lot for the clarifications regarding the POSIX capabilities.
> 
> 
> On Tue, 9 Jul 2019 at 14:49, Jeffrey Frey <frey at udel.edu> wrote:
> > So, if I understand this correctly, for some reason, `srun` does not need root privileges on the computation node side, but `sbatch` does when scheduling. I was afraid doing so would mean users could do things such as apt install and such, but it does not seem the case.
> 
> The critical part here is slurmstepd running as one user (root, or slurm in your case) attempting to seteuid/setegid/setreuid/setregid the process.  If not running as root, you'll see in the slurmstepd source code that the seteuid/setegid/setreuid/setregid calls are skipped -- thus, all job tasks run as SlurmdUser.  On a multiuser system, SlurmdUser=root is necessary and safe.
> 
> 
> 
> > I am not going to be managing the actual cluster, only exploring possibilities. At this point I am mostly convinced slurmdUser=sudo is safe, so that is one less potential problem.
> 
> SlurmdUser must be set to a user name present on the machine.  Setting it to "sudo" merely has it run as a user named "sudo."
> 
> 
> > @Patrick: I do not know how to do that. I only know that I can make slurm sudoer and NOPASSWD, but slurm would still call to `chown` (not `sudo chown`). An alternative would be replacing `chown` with a small script that calls `sudo chown`, but that is likely to break a lot of stuff. I assume slurmd will also need other root-only commands to work.
> 
> The chown in question is a chown() system call in the slurmd/slurmstepd source code that's compiled into slurmd/slurmstepd.  Replacing /usr/bin/chown with a custom command that calls sudo would not help at all (and would probably create a security issue).
> 
> 
> > @Michael Indeed, the documentation/tutorials often mention that SlurmdUser should be root, but it is not clearly explained why anywhere (e.g. https://slurm.schedmd.com/quickstart_admin.html section Daemons). It seems that `srun whoami` returns the current user (and not root), so even when slurmdUser is root, users do not have privileges, so in principle there is no problem at all.
> 
> Again, the critical part here is slurmstepd's having the ability to change the running uid/gid of the processes it starts.  With SlurmdUser=root, it can do this.  With any other user name, it cannot, and every job step runs as SlurmdUser.
> 
> 
> 
> > @Jeffrey It is expected to be multi-user. As for your third option, I think you refer to something similar to what I wrote for Patrick. 
> 
> I was talking about POSIX capabilities, and the possibility that all the capabilities exist to grant to slurmstepd the ability to chown() like root can, etc.  That still doesn't help with the seteuid/setegid/setreuid/setregid calls because slurmstepd doesn't even make those calls when it's not running as root.
> 
> 
> 
> 
> ::::::::::::::::::::::::::::::::::::::::::::::::::::::
> Jeffrey T. Frey, Ph.D.
> Systems Programmer V / HPC Management
> Network & Systems Services / College of Engineering
> University of Delaware, Newark DE  19716
> Office: (302) 831-6034  Mobile: (302) 419-4976
> ::::::::::::::::::::::::::::::::::::::::::::::::::::::
> 
> 
> 
> 
> 


::::::::::::::::::::::::::::::::::::::::::::::::::::::
Jeffrey T. Frey, Ph.D.
Systems Programmer V / HPC Management
Network & Systems Services / College of Engineering
University of Delaware, Newark DE  19716
Office: (302) 831-6034  Mobile: (302) 419-4976
::::::::::::::::::::::::::::::::::::::::::::::::::::::




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20190709/f4b8f2ea/attachment-0001.htm>