[slurm-users] heterogeneous jobs using packjob
Jeffrey Frey
frey at udel.edu
Tue Nov 13 08:03:54 MST 2018
See the documentation at
https://slurm.schedmd.com/heterogeneous_jobs.html#env_var
There are *_PACK_* environment variables in the job env that describe the heterogeneous allocation. The batch step of the job (corresponding to your script) executes on the first node of the first part of the allocation, which in your case is a node from the "high_mem" partition (hence the Slurm variables you're seeing). When using srun inside your batch script, the remote command's standard Slurm environment should be set (e.g. SLURM_NNODES, SLURM_JOB_NODELIST, SLURM_STEP_NUM_TASKS, etc.).
> On Nov 13, 2018, at 7:06 AM, Jing Gong <gongjing at kth.se> wrote:
>
> Hi,
>
> I can submit heterogeneous jobs using packjob likes
>
> ....
> #SBATCH -p high_mem
> #SBATCH -N 1
> #SBATCH --exclusive
> #SBATCH packjob
> #SBATCH -p log_mem
> #SBATCH -N 2
> #SBATCH --exclusive
> ....
> i.e. specify 1 fat node and two thin nodes for one jobs.
>
> If I use "squeue/scontrol" to check the job, it indeed distribute 3 nodes, However, if set SLURM environment variables in the job script, I always get the information of the first partition "high_mem".
>
> echo $SLURM_NNODES # 1
> echo $SLURM_JOB_NODELIST # p01k30
>
> How can I obtain the information of the two type nodes in a job script to run different applications on different nodes?
>
> Thanks a lot. /Jing
>
>
>
::::::::::::::::::::::::::::::::::::::::::::::::::::::
Jeffrey T. Frey, Ph.D.
Systems Programmer V / HPC Management
Network & Systems Services / College of Engineering
University of Delaware, Newark DE 19716
Office: (302) 831-6034 Mobile: (302) 419-4976
::::::::::::::::::::::::::::::::::::::::::::::::::::::
More information about the slurm-users
mailing list