The bounding boxes for masks (the area allowed to paint inside) doesn't always increase / decrease automatically. Masking an image outside of an artboard bugs it out somehow.
Here's a file showcasing the issue. I don't know how to replicate it entirely, but the whole right side won't mask at all in this example:
Mask Issues.afphoto
Also, masks don't affect outside of the artboard and I think it's a strange behavior. Especially for cutting out objects. It's a lot of extra work for no reason. We shouldn't need to Rasterize > Trim and lose the photos quality as a workaround.