I've been having this issue a lot lately on my M1 Pro/Monterey setup (2021 14" MacBook Pro). I just found this thread and tried turning off auto-brightness. I realized that auto-brightness has actually been annoying me anyway — I like to have full manual control — but I hadn't thought to look for the option to turn it off!
I've only just made this switch, so I haven't quite had enough time to know for sure that it fixes the problem. But it was happening so regularly with the file I'm currently working on that it should only take a minute or two of actions like switching layers, changing zoom level, etc. (the things that would cause the spinner before) to notice the problem, and so far so good!