Hi All, At one point, when we were running slurm 22, we set up a few new partitions with `AllowAccounts` setting enabled. We limit access to these partitions only to a few accounts. We manually create these accounts and users using the `sacctmgr` command. At that time, everything was working. Only users in these accounts were able to run jobs in that partition. Few months ago we upgraded slurm to 23.02.7 but just a couple of days ago we noticed that anyone could submit jobs to these partitions. In addition to that, you can provide any value to the `--account` directive and slurm will accept and record it in the job accounting database. In our initial testing, you could only provide `--account` value that has been previously created. I'm still looking into this but I wanted to check if there was any change in version 23 that could have impacted this? I don't remember seeing anything in release notes. Thanks, Marko
Marko, We are running 23.02.6 and have a partition with a specific account set in AllowAccounts. We test that only that account can use that partition, and it works. I'll note that you'll need to set EnforcePartLimits=ALL in slurm.conf for it to work, and if you use the job_submit filter, make sure you don't make changes there that change the account (we do that, that's why I mention it). I've now tested against 23.02.7, and it also works there. Rob ________________________________ From: Marko Markoc via slurm-users <slurm-users@lists.schedmd.com> Sent: Monday, March 11, 2024 2:07 PM To: Slurm User Community List <slurm-users@lists.schedmd.com> Subject: [slurm-users] AllowAccounts partition setting Hi All, At one point, when we were running slurm 22, we set up a few new partitions with `AllowAccounts` setting enabled. We limit access to these partitions only to a few accounts. We manually create these accounts and users using the `sacctmgr` command. At that time, everything was working. Only users in these accounts were able to run jobs in that partition. Few months ago we upgraded slurm to 23.02.7 but just a couple of days ago we noticed that anyone could submit jobs to these partitions. In addition to that, you can provide any value to the `--account` directive and slurm will accept and record it in the job accounting database. In our initial testing, you could only provide `--account` value that has been previously created. I'm still looking into this but I wanted to check if there was any change in version 23 that could have impacted this? I don't remember seeing anything in release notes. Thanks, Marko
Hi Rob, Thank you for helping me with this! I've rebuilt our environment in slurm 22.05.9 on a VM and verified that slurm would prohibit a job run on a unpermitted partition JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON) 1 test test vagrant PD 0:00 1 (Job's account not known, so it can't use this partition (test allows lap_rc)) On version 23, the job goes straight into running state. I would appreciate it if anyone has any ideas on what could have changed in version 23. Thanks, Marko On Tue, Mar 12, 2024 at 8:26 AM Groner, Rob <rug262@psu.edu> wrote:
Marko,
We are running 23.02.6 and have a partition with a specific account set in AllowAccounts. We test that only that account can use that partition, and it works. I'll note that you'll need to set EnforcePartLimits=ALL in slurm.conf for it to work, and if you use the job_submit filter, make sure you don't make changes there that change the account (we do that, that's why I mention it).
I've now tested against 23.02.7, and it also works there.
Rob
------------------------------ *From:* Marko Markoc via slurm-users <slurm-users@lists.schedmd.com> *Sent:* Monday, March 11, 2024 2:07 PM *To:* Slurm User Community List <slurm-users@lists.schedmd.com> *Subject:* [slurm-users] AllowAccounts partition setting
Hi All,
At one point, when we were running slurm 22, we set up a few new partitions with `AllowAccounts` setting enabled. We limit access to these partitions only to a few accounts. We manually create these accounts and users using the `sacctmgr` command. At that time, everything was working. Only users in these accounts were able to run jobs in that partition. Few months ago we upgraded slurm to 23.02.7 but just a couple of days ago we noticed that anyone could submit jobs to these partitions. In addition to that, you can provide any value to the `--account` directive and slurm will accept and record it in the job accounting database. In our initial testing, you could only provide `--account` value that has been previously created.
I'm still looking into this but I wanted to check if there was any change in version 23 that could have impacted this? I don't remember seeing anything in release notes.
Thanks, Marko
So, have you resolved this issue from 23? I am encountering the same problem.
HI, Sorry for the late reply. I have not yet fixed this issue. I did even update a couple of minor versions and the problem was not resolved. So just replacing AllowAccounts with AllowGroups worked for you ? Thanks, Marko On Sat, Jun 29, 2024 at 1:30 AM daijiangkuicgo--- via slurm-users < slurm-users@lists.schedmd.com> wrote:
AllowGroups is ok.
-- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-leave@lists.schedmd.com
Hello Marco, all, As I mention in another mail : in version 23-* the allowgroups is not OK : there is a memory leak (Has anybody observed it ? could you check ?). Thanks Christine De : Marko Markoc via slurm-users <slurm-users@lists.schedmd.com> Envoyé : lundi 1 juillet 2024 18:35 À : daijiangkuicgo@gmail.com Cc : slurm-users@lists.schedmd.com Objet : [slurm-users] Re: AllowAccounts partition setting HI, Sorry for the late reply. I have not yet fixed this issue. I did even update a couple of minor versions and the problem was not resolved. So just replacing AllowAccounts with AllowGroups worked for you ? Thanks, Marko On Sat, Jun 29, 2024 at 1:30 AM daijiangkuicgo--- via slurm-users <slurm-users@lists.schedmd.com<mailto:slurm-users@lists.schedmd.com>> wrote: AllowGroups is ok. -- slurm-users mailing list -- slurm-users@lists.schedmd.com<mailto:slurm-users@lists.schedmd.com> To unsubscribe send an email to slurm-users-leave@lists.schedmd.com<mailto:slurm-users-leave@lists.schedmd.com>
Hi Christine, we don't use AllowGroups but have AllowAccounts, which is not working anymore as expected in version 23.02.x. We tried also with EnforcePartLimits. I can see that our slurmctld consumes each day approx. 1g more memory. Our current version is 23.02.7 which comes from ATOS/EVIDEN for our cluster. Best Regards, Carsten -- Carsten Beyer Abteilung Systeme Deutsches Klimarechenzentrum GmbH (DKRZ) Bundesstraße 45a * D-20146 Hamburg * Germany Phone: +49 40 460094-221 Fax: +49 40 460094-270 Email:beyer@dkrz.de URL:http://www.dkrz.de Geschäftsführer: Prof. Dr. Thomas Ludwig Sitz der Gesellschaft: Hamburg Amtsgericht Hamburg HRB 39784 Am 02.07.24 um 09:10 schrieb LEROY Christine 208562 via slurm-users:
Hello Marco, all,
As I mention in another mail : in version 23-* the allowgroups is not OK : there is a memory leak (Has anybody observed it ? could you check ?).
Thanks
Christine
*De :*Marko Markoc via slurm-users <slurm-users@lists.schedmd.com> *Envoyé :* lundi 1 juillet 2024 18:35 *À :* daijiangkuicgo@gmail.com *Cc :* slurm-users@lists.schedmd.com *Objet :* [slurm-users] Re: AllowAccounts partition setting
HI,
Sorry for the late reply. I have not yet fixed this issue. I did even update a couple of minor versions and the problem was not resolved. So just replacing AllowAccounts with AllowGroups worked for you ?
Thanks,
Marko
On Sat, Jun 29, 2024 at 1:30 AM daijiangkuicgo--- via slurm-users <slurm-users@lists.schedmd.com> wrote:
AllowGroups is ok.
-- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-leave@lists.schedmd.com
tested slurm-23.* version, AllowAccounts parameter does not work. Carsten Beyer via slurm-users <slurm-users@lists.schedmd.com> 于2024年7月2日周二 15:54写道：
Hi Christine,
we don't use AllowGroups but have AllowAccounts, which is not working anymore as expected in version 23.02.x. We tried also with EnforcePartLimits.
I can see that our slurmctld consumes each day approx. 1g more memory. Our current version is 23.02.7 which comes from ATOS/EVIDEN for our cluster.
Best Regards, Carsten
-- Carsten Beyer Abteilung Systeme
Deutsches Klimarechenzentrum GmbH (DKRZ) Bundesstraße 45a * D-20146 Hamburg * Germany
Phone: +49 40 460094-221 Fax: +49 40 460094-270 Email: beyer@dkrz.de URL: http://www.dkrz.de
Geschäftsführer: Prof. Dr. Thomas Ludwig Sitz der Gesellschaft: Hamburg Amtsgericht Hamburg HRB 39784
Am 02.07.24 um 09:10 schrieb LEROY Christine 208562 via slurm-users:
Hello Marco, all,
As I mention in another mail : in version 23-* the allowgroups is not OK : there is a memory leak (Has anybody observed it ? could you check ?).
Thanks
Christine
*De :* Marko Markoc via slurm-users <slurm-users@lists.schedmd.com> <slurm-users@lists.schedmd.com> *Envoyé :* lundi 1 juillet 2024 18:35 *À :* daijiangkuicgo@gmail.com *Cc :* slurm-users@lists.schedmd.com *Objet :* [slurm-users] Re: AllowAccounts partition setting
HI,
Sorry for the late reply. I have not yet fixed this issue. I did even update a couple of minor versions and the problem was not resolved. So just replacing AllowAccounts with AllowGroups worked for you ?
Thanks,
Marko
On Sat, Jun 29, 2024 at 1:30 AM daijiangkuicgo--- via slurm-users < slurm-users@lists.schedmd.com> wrote:
AllowGroups is ok.
-- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-leave@lists.schedmd.com
-- slurm-users mailing list -- slurm-users@lists.schedmd.com To unsubscribe send an email to slurm-users-leave@lists.schedmd.com
participants (6)
-
Carsten Beyer -
daijiangkuicgo＠gmail.com -
Groner, Rob -
LEROY Christine 208562 -
Marko Markoc -
shaobo liu