@Chris B My monitors are not configured to Nvidia Surround. I am using 3 monitors however.
I played around with it some more, and it was only on the middle (main) monitor that will blackscreen. 2nd and 3rd monitor worked fine.
In a weird twist of events, I tried swapping the Display Port cables for my main and 2nd monitor (the third one is on HDMI) to check if it was a faulty port on GPU...
And that seems to have somehow fixed the problem for now. No blackscreens anywhere. Swapping the cables back to the first configuration reproduces the problem.
I also talked with an acquaintace, also on 3 monitors (2 displays, one display graphics tablet), and a GTX 1080, that had the same blackscreen behavior on Adobe Ai and PS. Also had to roll back driver. He later updated drivers again, where it had either been fixed on Adobe's or Nvidia's end. For reference, Ai and PS still work fine for me even when reproducing the problem.