[slurm-users] Use all cores when submitting to heterogeneous nodes

[Richard Ems]

Hi all,

Thanks for your comments and suggestions.
Using --exclusive does not solve my issue because I need ntasks to get set
by Slurm to the MAX possible.
What I want is to request a fixed number of nodes with --nodes=N and
--ntasks=MAX , so that Slurm provides 2 nodes and sets then ntasks to the
max possible, depending on the nodes selected.
So for example if I request 2 nodes and I get them from the 40 cores
partition, then ntasks gets set to 80 by Slurm. If those nodes are from the
2nd partition with 48 cores each node, then Slurm should set ntasks to 96.
The request seems a bit odd, but I have a commercial software (Altair) that
is using "scontrol show ..." output to set the number of processes/tasks to
start. I cannot manipulate that in my script.
But probably I will have to stop telling Altair I am using Slurm so it does
not use "scontrol show ...", and create all parallel settings myself
using SLURM's variables as Troy suggested.

Thanks,
Richard

On Tue, 22 Mar 2022 at 12:22, Baer, Troy <troy at osc.edu> wrote:

> Requesting --exclusive and then using $SLURM_CPUS_ON_NODE to determine the
> number of the tasks or threads to use inside the job script would be my
> recommendation.
>
>         --Troy
>
> -----Original Message-----
> From: slurm-users <slurm-users-bounces at lists.schedmd.com> On Behalf Of
> Tina Friedrich
> Sent: Tuesday, March 22, 2022 10:43 AM
> To: slurm-users at lists.schedmd.com
> Subject: Re: [slurm-users] Use all cores when submitting to heterogeneous
> nodes
>
> Hi Richard,
>
> ...what's wrong with using '--exclusive'? I mean if you're wanting all
> cores on the node anyway, wouldn't asking for it exclusively be pretty
> much the same thing?
>
> Tina
>
> On 22/03/2022 14:29, Richard Ems wrote:
> > Hi all,
> >
> > I am looking for an option to use all cores when submitting to
> > heterogeneous nodes.
> > In this case I have 2 partitions:
> > part1:  #N1 nodes, each node has 40 cores
> > part2:  #N2 nodes, each node has 48 cores
> >
> > I want to submit to both partitions, requesting a number of nodes and
> > then set
> > --ntasks=40*#nodes
> >   or
> > --ntasks=48*#nodes
> > depending on which partition gets selected by Slurm.
> > Can this be done?
> >
> > An option similar to --ntasks=USE_ALL_CORES would be great.
> >
> > Many thanks,
> > Richard
> >
> > --
> > Richard Ems     /     aiduit     / r.ems at aiduit.com
> > <mailto:r.ems at aiduit.com>
>
> --
> Tina Friedrich, Advanced Research Computing Snr HPC Systems Administrator
>
> Research Computing and Support Services
> IT Services, University of Oxford
>
> https://urldefense.com/v3/__http://www.arc.ox.ac.uk__;!!KGKeukY!gMufcIC3i58UdzqvJu5NJgHD5TdADngBaj6rq48GMTu5RjsPPTccwzudcC6u$
>
> https://urldefense.com/v3/__http://www.it.ox.ac.uk__;!!KGKeukY!gMufcIC3i58UdzqvJu5NJgHD5TdADngBaj6rq48GMTu5RjsPPTccw4ucG3qS$
>
>

-- 
Richard Ems     /     aiduit     /     r.ems at aiduit.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.schedmd.com/pipermail/slurm-users/attachments/20220322/e9c24082/attachment.htm>