Affinity disk usage warning???

[Axhill]

I am working on a cycle route book. 94 pages A4 with a colour map on each page.
Size of the total file is 100.9 MB.

My Disk Usage Warning is at 32768 MB, yet I now get the warning:
"Affinity disk usage has risen above the warning limit - consider closing some documents".

No other Affinity Publisher document is open.

The hard disk used still has 3.37 TB of space left....

What is going on here?

Affinity Publisher 2 versie 2.2.1
macOS Sonoma 14.1.1 -  Mac mini M2
 

[Old Bruce]

4 minutes ago, Axhill said:

My Disk Usage Warning is at 32768 MB, yet I now get the warning:

 

4 minutes ago, Axhill said:

The hard disk used still has 3.37 TB of space left....

The Disk Warning is about the Hard drive on the Mac, the one with the OS on it. Not an external drive that holds the document you are working on.

[Axhill]

@Old Bruce.

SSD on my Mac: 278.49 GB free space

[Old Bruce]

You may have done a lot of editing on the document and filled up the Swap. There may be other applications open which are also using Swap.

I have the same settings as you, a little bit less free space on my hard-drive and I only see that warning after a long editing session on extremely large files, several GBs in size, not < 100 MBs. I tend to not have other applications open when doing work like that. If you have linked files in the Publisher Document it may be cacheing them on the hard-drive. How big are the colour maps on each page? As you scroll / move through the document those have to be read from the hard-drive or be stored in RAM. Try running only Publisher.

Finally, as you have 200 + GBs of free space you could increase the warning limit for Publisher if it really annoys you.

[Axhill]

I may have to rethink my workflow.
I have always been used to putting the folders containing images, design elements, etc. on an external HDD where the Publisher document is also stored and then work from the Mac.
With the current project, that may not be so convenient. This document has many elements (markers, POIs, place names etc) which is placed on the route map, along with many anchors and hyperlinks. The route maps have sizes ranging from 144 KB to 10.2 MB. 
I had already noticed that saving the file each time took much more time than I was used to in past years. I did not have a project like this before and fear that my method of working was not quite the right one.

[Axhill]

2 hours ago, Old Bruce said:

You may have done a lot of editing on the document and filled up the Swap. There may be other applications open which are also using Swap.

What does SWAP means? I do not know what this is.

[Old Bruce]

15 minutes ago, Axhill said:

What does SWAP means? I do not know what this is.

Swap is a word that means switch out (or change) one thing for another. I would like to swap my Kodak Instamatic for your Hassleblad 500 C. In this case it is your OS and Application switching out information (data) from RAM to a temporary file on the hard-drive.

[Axhill]

Still, I don't quite understand what is happening. Is it sometimes just a bug in Affinity Publisher 2?
The folder containing my entire projects - i.e. the Publisher documents and the images folder is 833 MB in size and has 240 components.
I have set the Disk Warning Limit to 52 GB (!!!) and still I get a warning??!?

I understand that I shouldn't worry and can just continue with the project, but I still find it very strange....

 

[Dan C]

Hi @Axhill,

The Disk Warning Limit is not related to the amount of free space available on your hard drive - this is shown when the Affinity document itself is using a higher value than your set amount of disk resources in order to load/display/manipulate the document.

Essentially, when opening and editing a document in Affinity, the file will be loaded into your memory (RAM), though this memory amount is limited on certain machines or documents can use more memory than is available to the system. Once this happens, the Affinity app will begin paging the overflow memory requirements to your disk, adding temporary data to your hard drive, which is removed once the app is closed.

The warning you're seeing should be shown once that memory paging overflow exceeds the value set on your hard drive and is simply meant as information for the user, as it doesn't negatively affect the file itself.

44 minutes ago, Axhill said:

The folder containing my entire projects - i.e. the Publisher documents and the images folder is 833 MB in size and has 240 components.

You can use Activity Monitor on macOS to determine how much of your memory and disk usage the Affinity app is using - as the size of the .afpub file itself isn't necessarily indicative of the memory/disk usage when editing the file:

Note I have edited the columns in my version of Activity Monitor to display more information in one screenshot, so you may see slightly different columns in your version by default.

45 minutes ago, Axhill said:

I have set the Disk Warning Limit to 52 GB (!!!) and still I get a warning??!?

I'd recommend restarting your mac, then relaunching the Affinity app and loading this document once again - as this should flush your memory cache and ensure that the app is only use the required amount of memory for editing this file.

Does this stop the warning from showing, or does Activity Monitor indicate this file is still using above 52GB of disk usage?

[Axhill]

Thank you for your explanation. I restarted the Mac and the activity monitor shows that Publisher 2 is now using 12.54 GB of RAM.

[thomaso]

16 hours ago, Axhill said:

What does SWAP means? I do not know what this is.

The currently used SWAP size gets displayed at the bottom of the Activity Monitor.app window. A right click on the column title bar lets you customize the displayed parameters. Also SWAP files appear in the hidden folder on your boot volume: "/private/var/vm/" (you can copy/paste this path without "" and enter it in the Finder -> Cmd-Shift-G to get the folder opened. (note, their displayed sizes differ)

Another temporary memory swapped by Affinity to the hard drive is the file "personabackstore.dat". Also this is in "/private/var/vm/", respectively a subfolder with a quite long name that may differ for various macs and users, with the app name as parent folder. For instance:

Although there are various reports in the forum about a huge file size of this "personabackstore.dat", in my setup its size always appears to be displayed as 0 kB.

Apart from your opened apps & their windows (e.g. browser tabs), the used disk space is also influenced by TimeMachine backup files and its hourly "snapshots" that get stored locally for 24 hours.

[ As long those temporary files get removed automatically after use they don't harm. Nevertheless it may happen that one appears to stick although it is outdated. You can notice that if your free disk space for your drive(s) don't return to 'normal' after a reboot with TimeMachine disabled since 24 hours. In this case it may be worth to investigate more to detect and manually remove the culprit. ]

[Axhill]

Thank you. I forgot the used disk space that activity monitor shows.

[Dan C]

No problem at all, many thanks for confirming that for me!

This is certainly on the higher end of RAM usage - but isn't necessarily unexpectedly high, as it will depend on the specific document, document properties, resources, etc and is not concerningly high in my opinion.

Has restarting your system also stopped this disk waring from appearing?

I'd like to request a copy of your Affinity document, in order to investigate this further here to confirm if the current RAM/disk usage is expected.

To ensure that I receive all of your linked/embedded files, fonts in use for your file etc - with this document open, please navigate to File > Save as Package, ensuring that both 'Include all fonts' and 'Include all Images' is enabled.

Create a new folder and then select this folder as the location to save the Package.

Once saved, please upload the full folder to the below link for me (you may wish to .ZIP the folder before uploading, such that you only need to upload one file total, but this isn't necessarily required)

https://www.dropbox.com/request/7wOPjxK6zzWR6RVdYXw0

Once uploaded to the above link, please reply here to let me know. Many thanks in advance

[Axhill]

2 hours ago, Dan C said:

Has restarting your system also stopped this disk waring from appearing?

After restarting the Mac and reopening the file, I got no disk warning.

2 hours ago, Dan C said:

I'd like to request a copy of your Affinity document, in order to investigate this further here to confirm if the current RAM/disk usage is expected.

Once I finish the project, I will send you the complete file.
I still have to create the hyperlinks and then I am ready (for now)

[Axhill]

@Dan C

I got the disk warning again, but I didn't pay attention to it anymore.

I will send the complete file to you one of these days.

Some comments:
I was surprised that it took so long to save the file after a change. A 136 MB file written from the Mac to its SSD always took over 3 minutes!
Maybe I had the wrong approach with this project. On every page with a map I added elements such as markings, place names, etc. Sometimes there were 15-20 elements on such a page!
I think I could have worked faster if I had first made each map separately with its additions and then saved that map as a JPG or PDF. Then I would have only had text and one image on each page.

 

Update:

File has just been sent

[Dan C]

My sincerest apologies for the delayed response here and many thanks for providing your file!

I can confirm that on both macOS and Windows, I'm seeing rather high memory usage for this file within Publisher - due to the number of linked images/documents, the memory usage is certainly expected to increase whilst editing this document, however I do believe that this seems to be higher than I'd expect and may indicate a memory leak with your specific file.

Therefore I have logged this document with our development team for further investigation, to try and reduce the apps memory footprint when editing this file - which in turn should reduce the disk usage and therefore stop this warning from being shown with your document.

Unfortunately in the meantime, I'm yet to find any workarounds or instant resolutions with your file - I did find that the memory usage appears to be slightly better when using embedded and not linked resources, though this significantly increased the Affinity Publisher filesize, without providing that much benefit in terms of reduced memory usage, so I wouldn't necessarily recommend this with your file.

I hope this clears things up & my apologies I don't have more direct steps to suggest at this time

[Axhill]

@Dan C

Since I finished the first part of the project - but there are several more parts to follow - I set up the whole project again anyway. I created all maps with their additions (markers, place names, directions, etc.) separately, filed them away as pdfs and in the new project imported these maps again. This drastically reduced the large number of linked parts. Probably because of this, the file is again just saved quickly and I also no longer get Disk Limit Warnings.

Conclusion: my procedure was not the right one. Reducing the number of linked components seems to be the solution to my problem.

Thanks for your effort.

 

Update:

Although the file can now be saved quickly, in Activity Monitor I see that Affinity Publisher 2 still requires 17.57 GB of internal memory.

 

[thomaso]

5 hours ago, Axhill said:

I created all maps with their additions (markers, place names, directions, etc.) separately, filed them away as pdfs and in the new project imported these maps again. This drastically reduced the large number of linked parts. Probably because of this, the file is again just saved quickly and I also no longer get Disk Limit Warnings.

Does it mean the placed PDF are embedded now + this causes the improvement?
Or do you rather mean the additions on the maps within APub have been causing the issue before?

[Axhill]

@thomaso

No, the whole document does not contain any embedded files.
The roadmap required for the document is a file I create with linked parts, but that is of no consequence because I export it again as a PDF.
So the project document only imports an image in the PDF format and it is thus linked. With about 100 maps (so 100 PDF images) I save about 500-600 linked parts and you can notice that!

By the way, I think as a trial I am going to add embedded parts to the whole document. Am curious to see how big the document file gets then and if it saves a lot of time downloading and saving.

[thomaso]

2 hours ago, Axhill said:

No, the whole document does not contain any embedded files.
The roadmap required for the document is a file I create with linked parts, but that is of no consequence because I export it again as a PDF.
So the project document only imports an image in the PDF format and it is thus linked. With about 100 maps (so 100 PDF images) I save about 500-600 linked parts and you can notice that!

So the difference is now that the previously 100 linked separate map files are now linked as just one single PDF ?

I am wondering because of @Dan C's experience …

18 hours ago, Dan C said:

I'm seeing rather high memory usage for this file within Publisher - due to the number of linked images/documents, (…)

(…) I did find that the memory usage appears to be slightly better when using embedded and not linked resources

… whether the high memory usage had been caused by the amount of linked layout data – or just by the number of single, separate files for this content?

– Or whether the additions (markers, place names, directions, etc.) placed within APub did matter? Are those still in the layout .afpub or are they linked, too, coming with the linked PDF ?

[ I'd rather expect that embedded resources would require more memory for the .afpub because all data get loaded when the .afpub gets opened – while I'd assume linked resources would be loaded only when their specific pages need to get displayed or edited. ]

[Axhill]

Apparently I don't make it very clear how I put the document together, therefore a further explanation.

The project or document (whatever you want to call it) is a book of bicycle routes. In addition to some introductory pages, the document now contains about a hundred pages of a bicycle map. On each page is text (with tips and explanations) plus some icons that point to a POI, give directions, etc. These icons are linked. Also on one such page is a bicycle map. In my first setup, this bike map was a PNG file (image) that contained the same icons that were also linked plus place names. So such a map only could contain as many as 10-15 linked items. This caused very slow saving and frequent Disk Limit Warning.

For the revised document, I have now created an image in PDF format for each page of the bike map with its linked parts.
Now every page except for the information that was already on it shows the bike map in PDF that is linked. So now the number of linked parts per page is about 6, so in the old document there were about 10-15 linked parts.

[thomaso]

32 minutes ago, Axhill said:

So now the number of linked parts per page is about 6, so in the old document there were about 10-15 linked parts.

Contrary to what I thought, it seems that it is not the amount of data (megapixels or vectors) but the number of different resource file paths that matters for the memory required.

This could mean that it may reduce the required memory even further if you just use one PDF file containing all maps on different PDF pages + link that one file as single resource + switch the various PDF pages within the book layout .afpub to get the relevant map displayed.

[ It reminds me of a workflow I used decades ago with one very long freehand page that contained a series of multiple designs in a regular grid: I linked that single resource within ID and its multiple picture frames to just display the relevant part of the resource. To display a specific area of the resource on the ID pages, I simply changed the coordinate of the linked file within its picture frame, e.g. by typing "+30 mm" in the transform panel to get the next detail of the linked resource displayed in a certain picture frame. ]

[Axhill]

Update:

It does not matter at all whether the parts are embedded or linked. The download time of both is the same.
File size embedded is 1.23 GB and with linked parts 776 MB.
Activity monitor gave a memory usage for Embedded 23.74 GB and for Linked 22.68 GB
With both documents I was now getting a Disdk Limit Warning.
So it all makes no difference...

[thomaso]

25 minutes ago, Axhill said:

File size embedded is 1.23 GB and with linked parts 776 MB.

On 11/28/2023 at 11:56 AM, Axhill said:

the Publisher documents and the images folder is 833 MB in size and has 240 components.

On 11/27/2023 at 5:19 PM, Axhill said:

Size of the total file is 100.9 MB.

Are you talking about (quite) different layout stages for this .afpub?

 

30 minutes ago, Axhill said:

With both documents I was now getting a Disdk Limit Warning.

16 hours ago, Axhill said:

the file is again just saved quickly and I also no longer get Disk Limit Warnings.

Does it mean the Disk Limit Warning for the .afpub version with linked resources initially (after your resource replacements as PDF) did not occur for the same (identical) document while it does in the meantime?

[Axhill]

 In the meantime, I had done some work on the document, but now all are the same in terms of content:

1. Document with only linked parts, size 135.6 MB
2. Document with PDF maps, the rest the same as under 1 (so everything linked), size 776 MB
3. Document as under 2, but now with all parts embedded, size 1.23 GB

Download speed
1. 1 m 41 s
2. 1 m 10 s
3. 1 m 13 s

Activity monitor  (memory(
1. 23.70 GB
2. 22.80 GB
3. 23.87 GB

when opening each file I got the disk limit warning

So conclusion: it does not seem to make any difference how you format or save the document.

Advantage linked parts: you can always change something in the individual part and it will be copied to your document.