[slurm-users] Slurm overhead

[Mahmood Naderan]

The CPUs are Opteron and don't have HT.
I also have found that for large run times (2-3 days) with high number
of threads, e.g 32 threads, the run time w/o slurm is nearly the same
with little differences. For shared memory runs, there is no problem
and difference is negligible. So, I will try to use latest ompi to dig
more.
I think the problem is not with slurm itself :)
Regards,
Mahmood




On Thu, Apr 26, 2018 at 12:56 PM, John Hearns <hearnsj at googlemail.com> wrote:
> Mahmood,  do you haave Hyperthreading enabled?
> That may be the root cause of your problem. If you have hyperhtreading, then
> when you start to run more than the number of PHYSICAL cores you
> will get over-subscription. Now, with certain workloads that is fine - that
> is what hyperhtreading is all about.
> However HPC workloads have traditionalyl not benifited from hyperhreading.
>
> I would suggest the following:
>
> a) share the result of  cat /proc/cpuinfo with is here so we can figure out
> f HT is enabled
> b) learn how to mimic HT being switched on or off by setting every odd
> numbered CPU core to 'offline'
>   This means you can 'play' with HT being on or off without a reboot
> c) reboot one of your servers and look at the BIOS settings
>     That is a good idea anyway - please tell us if HT is on or off. What is
> the Power Profile? Are C0 states disabled?
>
>
>
>