[slurm-users] srun with &&, |, and > oh my!

[Williams, Gareth (IM&T, Black Mountain)]

Be brave and experiment! How far wrong can you go?

I will offer two related tips.

1) Since you want/need to use shell syntax, use srun to run a shell and pass the command as a string for that shell to interpret. Like: 
srun bash -c "cmd1 infile1 | cmd2 opt2 arg2 | cmd3 opt3 arg3 -- > outfile && cmd4 opt4 arg4"

2) put the compound command into a shell script and use srun to run the script. Like
srun bash myscript.sh
That may help you to have a more readable/maintainable command. 
The script could be static or dynamic (write it to a temporary file then use it straight away). The script could parse arguments if you like.

Happy experimenting.

Gareth

-----Original Message-----
From: slurm-users <slurm-users-bounces at lists.schedmd.com> On Behalf Of Chandler
Sent: Tuesday, 24 January 2023 1:23 PM
To: Slurm User Community List <slurm-users at lists.schedmd.com>
Subject: [slurm-users] srun with &&, |, and > oh my!

I want to run a command like: cmd1 infile1 | cmd2 opt2 arg2 | cmd3 opt3 arg3 -- > outfile && cmd4 opt4 arg4 Which runs fine at any prompt.

I'm afraid to just put `srun` at the beginning though, would it run the whole set of commands on the compute node?  I don't want to try it because it involves sending a bunch of data to stdout and then back into the other programs.  I think it might only run `cmd1 infile1` on the compute node and then the rest will try and run on the head node?