It's been a while now, and after a few driver updates this is still an issue. It's definitely not an issue isolated to just Affinity Photo as I gave Krita a shot and had the same thing happen. However, I also gave Corel Paintshop pro a shot, and it did not have this issue. So for the time being I will switch to using Paintshop as it does not require me to switch my graphics driver settings for it to function properly.
If this ever gets resolved I would happily return, but just like I won't drive a car whose power steering goes dead every few seconds, I won't use a program that flickers my screen every few seconds.