Multifactor fair-share with single account
Dear All, I have a question regarding the fair-share factor of the multifactor priority algorithm. My current understanding is that the fair-share makes sure that different *accounts* have a fair share of the computational power. But what if my organisation structure is flat and I have only one account where all my user reside. Is fair-share algorithm working in this situation -- does it take into account users (associations) from this single account, and tries to assing a fair-factor to each user? Or each user from this account have the same fair-factor at each iteration? And what if I have, say 3 accounts, but I do not wan't to calculate fair-share between accounts, but between all associations from all 3 accounts? In other words, is there a fair-share factor for users/associations instead of accounts? Kind regards -- Kamil Wilczek [https://keys.openpgp.org/] [D415917E84B8DA5A60E853B6E676ED061316B69B]
Hi Kamil, Kamil Wilczek <kmwil@mimuw.edu.pl> writes:
Dear All,
I have a question regarding the fair-share factor of the multifactor priority algorithm. My current understanding is that the fair-share makes sure that different *accounts* have a fair share of the computational power.
But what if my organisation structure is flat and I have only one account where all my user reside. Is fair-share algorithm working in this situation -- does it take into account users (associations) from this single account, and tries to assing a fair-factor to each user? Or each user from this account have the same fair-factor at each iteration?
And what if I have, say 3 accounts, but I do not wan't to calculate fair-share between accounts, but between all associations from all 3 accounts? In other words, is there a fair-share factor for users/associations instead of accounts?
Kind regards
We have a similar situation. We do in fact have an account for each research group and the groups are associated with institutes and departments, but we use FairShare=parent so that all users are given the same number of shares and thus treated equally by the fair-share mechanism. Cheers, Loris -- Dr. Loris Bennett (Herr/Mr) FUB-IT (ex-ZEDAT), Freie Universität Berlin
W dniu 4.01.2024 o 07:56, Loris Bennett pisze:
Hi Kamil,
Kamil Wilczek <kmwil@mimuw.edu.pl> writes:
Dear All,
I have a question regarding the fair-share factor of the multifactor priority algorithm. My current understanding is that the fair-share makes sure that different *accounts* have a fair share of the computational power.
But what if my organisation structure is flat and I have only one account where all my user reside. Is fair-share algorithm working in this situation -- does it take into account users (associations) from this single account, and tries to assing a fair-factor to each user? Or each user from this account have the same fair-factor at each iteration?
And what if I have, say 3 accounts, but I do not wan't to calculate fair-share between accounts, but between all associations from all 3 accounts? In other words, is there a fair-share factor for users/associations instead of accounts?
Kind regards
We have a similar situation. We do in fact have an account for each research group and the groups are associated with institutes and departments, but we use FairShare=parent so that all users are given the same number of shares and thus treated equally by the fair-share mechanism.
Hi Loris, but is the "FairShare=parent" still works for the Fair Tree, which is the default algorithm since 19.05? I can find this option only for the Classic Fair Share. And I'm trying to differentiate between users, so that they are not treated equally by the algorithm. Heavy users should have a lower factor. I think I could create an account for each user, but is that a common practice and not an overkill? I'm also trying to understand the Fair Tree, because there is a section when it says that users can have different factors if their common ancestor accounts have different factors. But what if they have only one single common ancestor? Would then association/users still be sorted by the fair-factor? Kind regards, --
Cheers,
Loris
-- Kamil Wilczek [https://keys.openpgp.org/] [D415917E84B8DA5A60E853B6E676ED061316B69B]
Hi Kamil, Kamil Wilczek <kmwil@mimuw.edu.pl> writes:
W dniu 4.01.2024 o 07:56, Loris Bennett pisze:
Hi Kamil, Kamil Wilczek <kmwil@mimuw.edu.pl> writes:
Dear All,
I have a question regarding the fair-share factor of the multifactor priority algorithm. My current understanding is that the fair-share makes sure that different *accounts* have a fair share of the computational power.
But what if my organisation structure is flat and I have only one account where all my user reside. Is fair-share algorithm working in this situation -- does it take into account users (associations) from this single account, and tries to assing a fair-factor to each user? Or each user from this account have the same fair-factor at each iteration?
And what if I have, say 3 accounts, but I do not wan't to calculate fair-share between accounts, but between all associations from all 3 accounts? In other words, is there a fair-share factor for users/associations instead of accounts?
Kind regards We have a similar situation. We do in fact have an account for each research group and the groups are associated with institutes and departments, but we use FairShare=parent so that all users are given the same number of shares and thus treated equally by the fair-share mechanism.
Hi Loris,
but is the "FairShare=parent" still works for the Fair Tree, which is the default algorithm since 19.05? I can find this option only for the Classic Fair Share.
We have used this for a long time and when the change from Classic Fair Share to Fair Tree Fairshare nothing changed for us.
And I'm trying to differentiate between users, so that they are not treated equally by the algorithm. Heavy users should have a lower factor.
I am not sure what you mean. Treating users equally and lowering the priority for heavy users do not contradict each other. All our users have the same number of normalised shares, but obviously, when a user consumes a large amount of resources, this is reflected in his or her usage and the corresponding fairshare value for the user will fall and thus so will the priority of his or her future jobs.
I think I could create an account for each user, but is that a common practice and not an overkill?
Because of what I wrote above, I don't think you need this.
I'm also trying to understand the Fair Tree, because there is a section when it says that users can have different factors if their common ancestor accounts have different factors. But what if they have only one single common ancestor? Would then association/users still be sorted by the fair-factor?
We don't use any specific features of Fair Tree Fairshare. We just have 'parent' all the way down, which results in every user having 1 raw share, regardless of the total number of users. HTH Loris -- Dr. Loris Bennett (Herr/Mr) FUB-IT (ex-ZEDAT), Freie Universität Berlin
On 1/4/24 02:41, Kamil Wilczek wrote:
W dniu 4.01.2024 o 07:56, Loris Bennett pisze:
Hi Kamil,
Kamil Wilczek <kmwil@mimuw.edu.pl> writes:
Dear All,
I have a question regarding the fair-share factor of the multifactor priority algorithm. My current understanding is that the fair-share makes sure that different *accounts* have a fair share of the computational power.
But what if my organisation structure is flat and I have only one account where all my user reside. Is fair-share algorithm working in this situation -- does it take into account users (associations) from this single account, and tries to assing a fair-factor to each user? Or each user from this account have the same fair-factor at each iteration?
And what if I have, say 3 accounts, but I do not wan't to calculate fair-share between accounts, but between all associations from all 3 accounts? In other words, is there a fair-share factor for users/associations instead of accounts?
Kind regards
We have a similar situation. We do in fact have an account for each research group and the groups are associated with institutes and departments, but we use FairShare=parent so that all users are given the same number of shares and thus treated equally by the fair-share mechanism.
Hi Loris,
but is the "FairShare=parent" still works for the Fair Tree, which is the default algorithm since 19.05? I can find this option only for the Classic Fair Share.
And I'm trying to differentiate between users, so that they are not treated equally by the algorithm. Heavy users should have a lower factor.
I think I could create an account for each user, but is that a common practice and not an overkill?
I'm also trying to understand the Fair Tree, because there is a section when it says that users can have different factors if their common ancestor accounts have different factors. But what if they have only one single common ancestor? Would then association/users still be sorted by the fair-factor?
Kind regards,
Kamil, fairshare=parent works for all algorithms, last I knew. It definitely works for Fair Tree. See https://slurm.schedmd.com/SUG14/fair_tree.pdf starting at page 80 for examples of usage. In page 84, the blue associations all have fairshare=parent set, so to fairshare it calculates it as if it looks like page 85. The example on pages 86-91 actually describe what's going on. Yes, Fair Tree considers sibling users/accounts even underneath another account. The algorithm is recursive and calculates fairshare amongst sibling users/accounts within each account. "sshare -l" will show you these calculations under "Level FS" within an account (keeping in mind that fairshare=parent will affect what looks like an account to Fair Tree). The presentation I linked to has a lot more details. https://slurm.schedmd.com/fair_tree.html is more succinct. Another way to think about this is that you are only looking at one subtree's children at a time. When one association wins, any of its children also win compared to any other associations. The algorithm recurses into that association (if there are children) to then determine which of those children win compared to others. So whether you have a flat structure or not, it compares siblings within an account. If you have 100 users directly parented by root (or 100 users underneath accounts with fairshare=parent all the way up to root), those 100 users will compete for fairshare like you want. Ryan
Hi, On 03.01.24 23:47, Kamil Wilczek wrote:
But what if my organisation structure is flat and I have only one account where all my user reside.
As do I.
Is fair-share algorithm working in this situation -- does it take into account users (associations) from this single account, and tries to assing a fair-factor to each user?
Yes.
And what if I have, say 3 accounts, but I do not wan't to calculate fair-share between accounts, but between all associations from all 3 accounts? In other words, is there a fair-share factor for users/associations instead of accounts?
FairShare=parent https://slurm.schedmd.com/sacctmgr.html I do not use this. MfG -- Markus Kötter, +49 681 870832434 30159 Hannover, Lange Laube 6 Helmholtz Center for Information Security
Thank you all for the help! I created a setup with a single account and multi-factor scheduling with three non-zero weights: job age, job size and fair-share. I'll monitor the fair-share when enough users will register on the cluster. Kind regards, -- Kamil Wilczek [https://keys.openpgp.org/] [D415917E84B8DA5A60E853B6E676ED061316B69B] W dniu 4.01.2024 o 11:56, Markus Kötter pisze:
Hi,
On 03.01.24 23:47, Kamil Wilczek wrote:
But what if my organisation structure is flat and I have only one account where all my user reside.
As do I.
Is fair-share algorithm working in this situation -- does it take into account users (associations) from this single account, and tries to assing a fair-factor to each user?
Yes.
And what if I have, say 3 accounts, but I do not wan't to calculate fair-share between accounts, but between all associations from all 3 accounts? In other words, is there a fair-share factor for users/associations instead of accounts?
FairShare=parent https://slurm.schedmd.com/sacctmgr.html
I do not use this.
MfG
participants (4)
-
Kamil Wilczek -
Loris Bennett -
Markus Kötter -
Ryan Cox