Same here! My lag appears some time after using the App itself. Number of layers does not help as it seems the more you have, the bigger the lag. In my case the brushes work fine once activated, but UI is lagging 1-2 seconds between switching the tools. If I use pencil (N) and swap to move_node (A), it takes so long that sometimes I create extra nodes as I'm still in pencil mode, instead of nudging the nodes themselves. Same happens when I try to zoom and pane, it takes a sec or two for the tool to swap and activate itself. The software sometimes goes into freezing mode (not responding) and it takes 10-15 seconds before I can use anything again.
I tried playing with settings, memory and disk space, hardware acceleration, even Undo Limit number. The issue still persist. Also, making sure evey time I experience the lag that "Windows Ink/Pen" is disabled on my Xp-Pen tablet drivers (as I heard this sometimes causes issues too).
I use Win11, AMD Ryzen 9 (5900HS), 32 GB RAM & nVidia RTX 3060.