[slurm-users] snakemake and slurm in general

[David Laehnemann]

Yes, but only to an extent. The linked conversation ends with this:

>> Do you have any best practice about setting MaxJobCount to a proper
number?

> That depends upon your workload. You could probably set MaxJobCount
to at least 50000 with most systems (assuming you have at least a few
gigabytes of memory). Some sites run with a value of 1000000 or more.

So, it is configurable. But this has a limit. And if you have lots of
users on a system submitting lots of jobs, even a value of 1000000 can
get exhausted.

And in either case, this is not something that speaks against a
workflow management system giving you additional control over things.
So I'm not sure what exactly we are arguing about, right here...

cheers,
david



On Thu, 2023-02-23 at 17:41 +0100, Ole Holm Nielsen wrote:
> On 2/23/23 17:07, David Laehnemann wrote:
> > In addition, there are very clear limits to how many jobs slurm can
> > handle in its queue, see for example this discussion:
> > https://bugs.schedmd.com/show_bug.cgi?id=2366
> 
> My 2 cents: Slurm's job limits are configurable, see this Wiki page:
> https://wiki.fysik.dtu.dk/Niflheim_system/Slurm_configuration/#maxjobcount-limit
> 
> /Ole
>