[Dragging Document with Huion Tablet is Incredibly Janky]

I have my Huion 1060P setup so the button on my pen acts as middle mouse, when in Affinity and ONLY Affinity Photo trying to drag the document with the pen is IMMENSELY JANK, it'll randomly grab and let go constantly and this causes me to accidentally scribble all over my document on top of my image jumping all over the screen. However, using middle click with my pen outside the app works perfectly.

Here's the system specs and info:
Processor    AMD Ryzen Threadripper 1920X 12-Core Processor    4.00 GHz
Installed RAM    32.0 GB (31.9 GB usable)
Device ID    036FD1CE-C1A0-4DEE-9776-516655452217
Product ID    00325-95800-00000-AAOEM
System type    64-bit operating system, x64-based processor
Pen and touch    Pen support

Edition    Windows 10 Home
Version    21H1
Installed on    ‎6/‎22/‎2020
OS build    19043.1288
Experience    Windows Feature Experience Pack 120.2212.3920.0
 

Edit: some reading shows that huion in general might be scuffed, the problem is that I know for a fact that middle mouse dragging with my pen worked fine in the past, but I guess they made some lame driver update that has completely ruined the functionality of the tablet.