The issue is as follows:
When using mouse, there's no issue.
When using the touch screen of my surface pro and holding either shift or ctrl for resizing, there's an extreme lag.
If I'm not holding shift or ctrl, it works fine.
I'm attaching a short clip of the issue, although I could not manage to record the position of the pen, so I'm not sure if it's clear enough. I do 3 movements: with finger, with surface pro pen and with surface pro pen while holding ctrl. For half of the time of the last, I am not touching the screen.
EDIT: Actually, you can see the last one speeds up right after I release the pen.
Affinity_Designer_2020-01-22_09-27-39.mp4