[slurm-users] heterogeneous jobs using packjob

[Jeffrey Frey]

See the documentation at


	https://slurm.schedmd.com/heterogeneous_jobs.html#env_var


There are *_PACK_* environment variables in the job env that describe the heterogeneous allocation.  The batch step of the job (corresponding to your script) executes on the first node of the first part of the allocation, which in your case is a node from the "high_mem" partition (hence the Slurm variables you're seeing).  When using srun inside your batch script, the remote command's standard Slurm environment should be set (e.g. SLURM_NNODES, SLURM_JOB_NODELIST, SLURM_STEP_NUM_TASKS, etc.).





> On Nov 13, 2018, at 7:06 AM, Jing Gong <gongjing at kth.se> wrote:
> 
> Hi,
> 
> I can submit heterogeneous jobs  using packjob likes
> 
> ....
> #SBATCH -p high_mem
> #SBATCH -N 1
> #SBATCH --exclusive
> #SBATCH packjob
> #SBATCH -p log_mem
> #SBATCH  -N 2
> #SBATCH --exclusive
> ....
> i.e. specify 1 fat node and two thin nodes for one jobs. 
> 
> If I use "squeue/scontrol" to check the job, it indeed distribute 3 nodes, However, if set SLURM environment variables in the job script, I always get the information of the first partition "high_mem".
> 
> echo $SLURM_NNODES               # 1
> echo $SLURM_JOB_NODELIST   # p01k30
> 
> How can I obtain the information of the two type nodes in a job script to run different applications on different nodes?
> 
> Thanks a lot. /Jing 
> 
> 
> 


::::::::::::::::::::::::::::::::::::::::::::::::::::::
Jeffrey T. Frey, Ph.D.
Systems Programmer V / HPC Management
Network & Systems Services / College of Engineering
University of Delaware, Newark DE  19716
Office: (302) 831-6034  Mobile: (302) 419-4976
::::::::::::::::::::::::::::::::::::::::::::::::::::::