[slurm-users] maximum size of array jobs

[Ole Holm Nielsen]

On 2/26/19 9:07 AM, Marcus Wagner wrote:
> Does anyone know, why per default the number of array elements is 
> limited to 1000?
> 
> We have one user, who would like to have 100k array elements!
> 
> What is more difficult for the scheduler, one array job with 100k 
> elements or 100k non-array jobs?
> 
> 
> Where did you set the limit? Do your users use array jobs at all?

Google is your friend :-)

https://slurm.schedmd.com/job_array.html

> A new configuration parameter has been added to control the maximum job array size: MaxArraySize. The smallest index that can be specified by a user is zero and the maximum index is MaxArraySize minus one. The default value of MaxArraySize is 1001. The maximum MaxArraySize supported in Slurm is 4000001. Be mindful about the value of MaxArraySize as job arrays offer an easy way for users to submit large numbers of jobs very quickly.

/Ole