I am having a similar issue and looking forward to a reply from Serif.
Working on a photo book - at 5 pages, maybe 50 images, all linked.
When I first looked at the file size, it was at 1.84GB. Tried some of the previous solutions - ensured I wasn't using Save History, tried Save As - still 1.84GB.
Using the Resource Manager, I embedded all of the image files, then switched them back to linked and suddenly the file size was down to 3.2MB. Made a couple changes (because during this process, a bunch of the images suddenly appear like they have a white semi-transparent overlay on them and I wanted to make sure that wasn't the case), and the file size immediately climbed back up to several hundred MB.
Of course, Publisher quickly slows to a crawl - opening, saving, working on the file... when I did last year's photo book, I ended up doing 10 separate files of 10 pages each (and they were still each large and slow), which I'd rather not have to do!
M1 MacBook, Publisher 1.10.4