<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-2022-jp">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
Dear Kota, </div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
Appreciate the feedback. I will read up the latest documentation when the time comes to configure. Thank you for your detailed email and will indeed read your blog. </div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
Regards</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
Tim</div>
<div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="Signature">
<div>
<div id="divtagdefaultwrapper" dir="ltr" style="font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<div id="divtagdefaultwrapper" style="font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255); font-family: Calibri, Arial, Helvetica, sans-serif;">
<p style="margin-top: 0px; margin-bottom: 0px;"></p>
</div>
</div>
</div>
</div>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> slurm-users <slurm-users-bounces@lists.schedmd.com> on behalf of Kota Tsuyuzaki <kota.tsuyuzaki.pc@hco.ntt.co.jp><br>
<b>Sent:</b> Wednesday, 21 April 2021 11:09<br>
<b>To:</b> 'Slurm User Community List' <slurm-users@lists.schedmd.com>; slurm-users@schedmd.com <slurm-users@schedmd.com><br>
<b>Subject:</b> Re: [slurm-users] SLURM A100</font>
<div> </div>
</div>
<div>
<p></p>
<div style="background-color:#FFEB9C; width:100%; border-style:solid; border-color:#9C6500; border-width:1pt; padding:2pt; font-size:10pt; line-height:12pt; font-family:'Calibri'; color:Black; text-align:left">
<span style="color:#9C6500">CAUTION:</span> This email originated outside the UCT network. Do not click any links or open attachments unless you know and trust the source.</div>
<br>
<p></p>
<div>Hello Tim,<br>
<br>
In the last year, I figured out the A100 MIG feature behavior with Slurm Workload Manager. At that time, it required non-default<br>
DEVFS mode in kernel config to constraint the MIG device via Slurm cgroup. After the setting, A100 MIG works well to me so I suppose<br>
it should NOT be blocking issue except you need to have the configuration.<br>
<br>
My testing NVIDIA driver version was at 450.51.06, and the mode was not default at that time but the NVIDIA documents said the DEVFS<br>
mode will be default in the future so that you should check the current newest docs if you mind the kernel setting.<br>
<br>
The procedure how we can configure the DEVFS mode to A100 was written to my blog post(*1). It's so sorry that was in Japanese but<br>
hopefully, the setting scripts and web links to NVIDIA official documents would be helpful for you. Perhaps, google translation too.<br>
<br>
1: <a href="https://medium.com/nttlabs/nvidia-a100-mig-as-linux-device-66220ca16698">https://medium.com/nttlabs/nvidia-a100-mig-as-linux-device-66220ca16698</a><br>
<br>
Best,<br>
<br>
<br>
--------------------------------------------<br>
$BO*:j!!9@B@(B (Kota Tsuyuzaki)<br>
kota.tsuyuzaki.pc@hco.ntt.co.jp<br>
NTT$B%=%U%H%&%'%"%$%N%Y!<%7%g%s%;%s%?(B<br>
$BJ,;6=hM}4pHW5;=Q%W%m%8%'%/%H(B<br>
0422-59-2837<br>
---------------------------------------------<br>
<br>
<br>
> -----Original Message-----<br>
> From: slurm-users <slurm-users-bounces@lists.schedmd.com> On Behalf Of Timothy Carr<br>
> Sent: Wednesday, April 21, 2021 4:14 PM<br>
> To: slurm-users@schedmd.com<br>
> Subject: [slurm-users] SLURM A100<br>
> <br>
> Dear Community,<br>
> <br>
> Trust everyone is well and keeping safe?<br>
> <br>
> We are considering the purchase of nodes with the Nvidia A100 GPUs and enabling the MIG feature which allows for the<br>
> creation of instance resource profiles. The creation of these profiles seems to be straightforward as per the<br>
> documentation. Have any of you had the opportunity to implement the A100 MIG with SLURM and have you found any<br>
> caveats you are willing to share?<br>
> <br>
> Kind Regards<br>
> <br>
> --<br>
> Tim<br>
> <br>
> <br>
> <br>
> <br>
> Disclaimer - University of Cape Town This email is subject to UCT policies and email disclaimer published on our website<br>
> at <a href="http://www.uct.ac.za/main/email-disclaimer">http://www.uct.ac.za/main/email-disclaimer</a> or obtainable from +27 21 650 9111. If this email is not related to the<br>
> business of UCT, it is sent by the sender in an individual capacity. Please report security incidents or abuse via<br>
> <a href="https://csirt.uct.ac.za/page/report-an-incident.php">https://csirt.uct.ac.za/page/report-an-incident.php</a>.<br>
<br>
<br>
</div>
</div>
Disclaimer - University of Cape Town This email is subject to UCT policies and email disclaimer published on our website at http://www.uct.ac.za/main/email-disclaimer or obtainable from +27 21 650 9111. If this email is not related to the business of UCT, it
is sent by the sender in an individual capacity. Please report security incidents or abuse via https://csirt.uct.ac.za/page/report-an-incident.php.
</body>
</html>