I third the suggestion to use OnDemand. FWIW, OnDemand uses VNC under the hood, so performance is identical to that, and the user experience is much, much better. Plain VNC is marginally easier for the administrator to set up: choose if you prefer doing a bit more administration work or (a little or a lot, depending on sophistication of your base) more user-support work.
To answer the original question, which most people have avoided.... The problem is that X11 is a protocol with a high number of latency-sensitive messages being exchanged, even for a simple action such as a single button click (let alone a menu). It was never really designed to run across complex networks as we do today. Every time you add a hop (a slurm one in which case) that latency increases and given the large number of messages involved, it easily becomes noticeable. Here slurm could be the last straw, but it's possible that the vast majority of latency is introduced by other network hops.
In our network setup, even just regular ssh with X-tunneling to the head node results in an unusable latency for X applications running on the head node itself (let alone on the compute nodes). OnDemand (web server on the head node, used also as a jump to the compute nodes since they are inaccessible from the outside) works just fine despite the additional network hoops.
If you are *really* stuck with plain X-tunneling, there isn't much you can do, other than a careful study on all the sources of latency and a careful, and tedious work attempting to limit them. Maybe some tracerouting/pinging can at least give you a first rough idea on what this would entail. You may be lucky and there is a single large source which you could easily mitigate, but in my experience it's always been a death by papercuts scenario.