Hi Shooktija,

On Wed, Apr 17, 2024 at 7:45 AM Shooktija S N via slurm-users <slurm-users@lists.schedmd.com> wrote:
NodeName=server[1-3] RealMemory=128636 Sockets=1 CoresPerSocket=64 ThreadsPerCore=2 State=UNKNOWN Gres=gpu:1
PartitionName=mainPartition Nodes=ALL Default=YES MaxTime=INFINITE State=UP

I want to reserve a few cores and a few gigs of RAM for use only by the OS which cannot be accessed by jobs being managed by Slurm. What configuration do I need to do to achieve this?

You want to look at these parameters for the Node section of slurm.conf
https://slurm.schedmd.com/slurm.conf.html#OPT_CoreSpecCount
https://slurm.schedmd.com/slurm.conf.html#OPT_MemSpecLimit

 
Is it possible to reserve in a similar fashion a 'percent' of the GPU which Slurm cannot exceed so that the OS has some GPU resources?

Not that I know of
 
Is it possible to have these configs be different for each of the 3 nodes?

Yes. You will need to define the nodes using 3 separate Node definitions versus one definition for all 3
 
Best,

-Sean