Came here because I had the same issue on the latest Affinity Photo (1.10.4) on Windows 10, saw there was no solution here, and started investigating.
My machine is more modest, a laptop using 7th gen i5, Geforce 940MX, Intel Graphics HD 620, 16GB RAM.
Trying out Affinity Photo, Publisher and Designer for the first time. They were all set to use Geforce 940MX on my graphics panel, and on all three of the apps' settings I used this:
Renderer: Geforce 940MX
Retina rendering: Automatic (Best)
Hardware acceleration: Enable OpenCL compute acceleration
I opened up a simple 4-layer 25MB PSD saved for maximum compatibility on Affinity Photo. It lagged heavily for turning layers on and off or moving them around. Simply turning a layer on and off took 5+ seconds and would hang the application ("not responding" alert).
Then I opened the same file on both Photoshop and Affinity Designer, they were all working at the same time at this point. Photoshop was fast, Af Designer was even faster, super snappy, and that's when I thought there had to be something very wrong.
Tried changing Photo to use Intel HD Graphics on my system's graphics panel, tried changing renderers to Intel Graphics or WARP, tried changing retina rendering to faster, tried restarting, tried updating drivers, nothing helped much. I tried saving the file to Affinity Photo format, it didn't work either.
What fixed it for me was turning off "Hardware acceleration: Enable OpenCL compute acceleration" on Affinity Photo.
Independently of what graphics card I have configured to be used by Photo (either on my system's graphics panel or the inner "renderer" setting), when OpenCL compute acceleration is on, it prints for me the words "Intel HD Graphics 620" on the next line, which is kinda weird. Then when I move the layers back and forth quickly, I noted CPU usage went quickly to over a 100%. When that setting is off, everything works fine.
I wouldn't be so quick to blame Intel Graphics either, because with the exact same file and settings on the same computer I have very snappy performance from Affinity Designer. It also tells me OpenCL compute acceleration is using Intel Graphics. To me, this looks like a performance/optimization issue on Photo.
I know it's an oldish post, but this described my problem perfectly, and seeing as it posed no solution, I thought I'd share my experience.
