Well I tried to do some digging. I loaded up RenderDoc and did a few frame captures of me moving one of the artboards. I found that it rarely took over 5ms to render a single frame with 11ms being the worst (need 16.7ms or less to maintain a smooth 60 frames/sec). Nothing too crazy in the D3D calls that I could spot.
I did however notice that my CPU usage shot up dramatically when dragging an artboard or even a single object around. I don't have symbols for any of the Affinity products though so I can't really look into it any further. Best I could see from the VS 2019 debugger were some pretty hefty calls into libkernel.dll and libpersona.dll taking up to 2,800ms of total CPU time.
My experience is mostly in the game dev sector. I've done a few random things in OpenGL and Vulkan so I have a decent understanding of graphics pipelines. I don't see anything that stands out in the RenderDoc captures. Im going to say this is a bottleneck on the CPU side of things. Without symbols though, I can't dig any further and I just hope Serif can fix this eventually.
Im just throwing ideas out here but maybe it's something to do with a threading bottleneck (been there, done that). I know the "older" graphics APIs like OpenGL and DirectX 11 aren't super friendly when it comes to multi-threading. Might be worth it to move the Windows renderer to DirectX 12 or even Vulkan eventually.
That's all I got, wish I had more. I prefer Affinity products to the competition to I'll just deal with it and keep working.