[slurm-users] How to use a pyhon virtualenv with srun?

Mark Hahn hahn at mcmaster.ca
Mon Nov 18 15:49:19 UTC 2019


>> Actually, you are not allowed to install things in your home on  computecanada,
>> this is why you need to install everything in a  virtualenv with pip
>> install. Also, you have to install each virtualenv  in $SLURM_TMDIR which is the
>> local drive of the node, because  everything else is slow, so I think I cannot
>> share homes.
>
> How are you prevented from installing things in your home directory?

Just for the record, this is a misapprehension of ComputeCanada policy:
there is no such requirement, no such rule against using home directories.

We have some variation of policy among sites, but nothing more extreme 
than home being RO (one site) or requiring jobs to be submitted from (cwd)
one of the non-home shared filesystems (another site).

For some workflows, it can indeed be convenient to perform a from-scratch
install on a node-local disk.  There is an obvious cost to doing this,
and indeed, the install would have to be performed via srun.

regards,
-- 
Mark Hahn | SHARCnet Sysadmin | hahn at sharcnet.ca | http://www.sharcnet.ca
           | McMaster RHPCS    | hahn at mcmaster.ca | 905 525 9140 x24687
           | Compute/Calcul Canada                | http://www.computecanada.ca




More information about the slurm-users mailing list