[slurm-users] How do you make --export=NONE the default behavior for our cluster?

[Ward Poelmans]

Hi,

We're using a cli filter for doing this. But it's more tricky then just `--export=NONE`. For a srun inside a sbatch, you want `--export=ALL` again because MPI will break otherwise.

We have this in our cli filter:

function slurm_cli_pre_submit(options, pack_offset)

         local default_env = "HOME,USER,TERM,PATH=/bin:/sbin"



         if options["type"] == "sbatch" or (options["type"] == "srun" and options["pty"] == "set") then

         -- for sbatch and interactive srun, set our minimal environment

             if options["export"] == nil then

                 options["export"] = default_env

                 slurm.log_debug("Setting export to %s", tostring(options["export"]))

             end

         elseif options["type"] == "srun" then

         -- in case of srun, and we're in our minimal environment, pass again everything (for mpi)

             if options["export"] == default_env then

                 options["export"] = "ALL"

                 slurm.log_debug("Setting export to %s", tostring(options["export"]))

             end

         end

end

So we set `--export=HOME,USER,TERM,PATH=/bin:/sbin` if the user has not specified anything else. For a srun inside a sbatch, we switch again to `--export=ALL`.

Ward

On 3/06/2022 20:39, Ransom, Geoffrey M. wrote:
> Hello
> 
>      We recently added new architectures to our compute and submit nodes and a PATH gets generated based on the type of machine our users log into. Unfortunately, this PATH is architecture dependent but it is getting copied with slurm jobs to the compute nodes which could be a different architecture as the submit node. Right now, running srun inside the sbatch script will fail with linking errors if you submit from a RHEL 8 box and land on a RHEL 7 box since it tries to run the RHEL 8 compiled version due to the PATH environment variable.
> 
> Adding “--export=NONE” to the job avoids the problem, but I’m not seeing a way to change this default behavior for the whole cluster.
> 
> How do you change/set default options for the cluster for all users so “--export=NONE” is the default behavior?
> 
> Is this change likely to break anything unexpected or cause problems?
> 
> Our users have NFS mounted home directories so if we removed the ssh non-admin user block they would get a similar/functional environment as they do on the submit nodes.
> 
> Thanks.
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4716 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20220604/c7f11906/attachment-0001.bin>