Why does Publisher reload all linked image files in the background?

[mzed]

 

I am currently editing a relatively large file with about 100 large (200MB / image) linked image files. Publisher actually saves the image previews. But when loading the project file, Publisher still loads all the files in the background, which takes a corresponding amount of time, consumes an enormous amount of RAM (with 7GB!) and slows down my computer. Publisher is hardly usable during this time, as it slows down a lot. 

Is there a way to deactivate this behaviour?

[Pšenda]

Unfortunately, APublisher can't do something like Draft mode.
Therefore, I would only use image previews (small files with low resolution) when creating the document, and only when finalizing would I replace / copy the originals.

[carl123]

1 hour ago, mzed said:

I am currently editing a relatively large file with about 100 large (200MB / image) linked image files

Assuming the linked files are all within the same parent folder you could try renaming that folder.

Publisher should still run but will only show you the low-res version of the images

When needed rename the folder back to what it was before

 

[DWright]

I would also use the Resource Manager to check that all of your images are linked to the document.

[mzed]

Quote

I would also use the Resource Manager to check that all of your images are linked to the document.

 

Yes, I checked that, all the images are linked. The phenomenon is that the background loading only happens once when loading the project file. As described, this takes a long time and uses up a lot of RAM. When Publisher is finished, 2/3 of the RAM is released again and you can work normally and quickly. Actually, this would not be necessary, as the image previews are saved in the project file and are also displayed immediately. Reloading as needed would be better here.

[R C-R]

3 hours ago, mzed said:

Reloading as needed would be better here.

Reloading from disk is always going to take more time than just keeping as much as possible in RAM, so they probably decided to load once for performance reasons. Otherwise, it could take several seconds before you could see more than blurry low resolution previews after zooming/panning around in the file. 

[Pšenda]

32 minutes ago, R C-R said:

Otherwise, it could take several seconds before you could see more than blurry low resolution previews after zooming/panning around in the file. 

However, there is no need to load all pages that may not be displayed in the end. OP suggests that they not be loaded immediately after opening the document, when loading consumes all the power of the processor and disk, and thus slows down the work, but when the pages are displayed, or in the background with a lower priority. This is (should be) the standard procedure and use of multithreading.

[R C-R]

30 minutes ago, Pšenda said:

However, there is no need to load all pages that may not be displayed in the end.

How could the app predict which image files (not pages) the user would want to be displayed or when? Also, what exactly do you mean by "in the end"?

[Pšenda]

1 hour ago, R C-R said:

Also, what exactly do you mean by "in the end"?

The document has 10,000 pages, which, when opened, gradually load and consume the power of the computer. The user views page 1, edits it, and closes the document. Thus, the next 9999 pages loaded completely unnecessarily (in the end, they were not used or displayed), and only by consuming computing power did they limit the work on that page 1.

 

1 hour ago, R C-R said:

which image files (not pages)

I guess I don't understand the question well enough - how could the contents of image files be displayed without displaying the image data contained in them, which are an integral part of the mentioned pages, but which should not be displayed ?! Strange.

 

1 hour ago, R C-R said:

How could the app predict

Is there something incomprehensible about that?

1 hour ago, Pšenda said:

but when the pages are displayed, or in the background with a lower priority.

Of course, if the user jumps to page 9999 after viewing page 1, he will have to wait for its display (its content will not be loaded yet). Which he probably would have to even with the current solution, where pages 2-9998 are loaded (so far unnecessarily) immediately after opening the document.

[R C-R]

3 hours ago, Pšenda said:

The document has 10,000 pages, which, when opened, gradually load and consume the power of the computer. The user views page 1, edits it, and closes the document. Thus, the next 9999 pages loaded completely unnecessarily (in the end, they were not used or displayed), and only by consuming computing power did they limit the work on that page 1.

But who is to say that the next time the document is opened, the user will still want to edit or even view page 1, or any other specific page? I can't speak for anybody else but I frequently want to open a multi-page Affinity document & review several pages/spreads one or a few at a time before I zoom into any one page or do any edits on any of them.

3 hours ago, Pšenda said:

I guess I don't understand the question well enough - how could the contents of image files be displayed without displaying the image data contained in them, which are an integral part of the mentioned pages, but which should not be displayed ?! Strange.

Obviously, the image files have to be loaded before they can be displayed, but there could be 1, 2, or any arbitrary number of them on whatever page(s) the workspace view is currently displaying. Remember that the view can be zoomed out to show many pages or zoomed in to show just a part of one page. This can be done smoothly with out long stalls because the image data is already in memory. Also, we have the option of multiple views of the document open simultaneously via the View > Views menu option.

[carl123]

Lazy loading is a technique used on the Internet to help with these sorts of image loading issues.

Maybe Publisher could have a preference setting to switch on Lazy Loading for those that would prefer it.

Possibly adding it as an option to the existing Retina Rendering choices in Preferences > Performance

https://web.dev/lazy-loading/

[mzed]

11 hours ago, R C-R said:

How could the app predict which image files (not pages) the user would want to be displayed or when? Also, what exactly do you mean by "in the end"?

I think it would be very useful to be able to control the behaviour at any time and specifically via options. There are different phases in the layout of documents. If the images are already set and you are only designing text, you do not necessarily need a high resolution of the image view when zooming.
There are also other reasons. For example, my images are not stored on my fast internal SSD but on a NAS. And depending on the computer system and the connection, access to the data storage medium puts the system under different loads. 

[Pšenda]

6 hours ago, carl123 said:

Lazy Loading

Just to the name - I think Affinity meets it perfectly, because I do not know any other application for processing texts and images, which is so incredibly slow and lazy to display documents after its opening 🙂

[Old Bruce]

On 6/13/2022 at 12:56 AM, mzed said:

I am currently editing a relatively large file with about 100 large (200MB / image) linked image files.

100 x 200MB = 20 GB. That is a lot of memory required. Having to read from the networked drive will take a long time as you note later on. And to be honest it is not a good idea to work off of a networked drive as the many threads about corrupted documents attest to.

Myself I would not work this way. I would generate 100 low resolution greyscale TIFF* files and place them. Now when I want to I rename the folder with the low resolution files to "Old Pic Files" and then I would make a directory with the name "Pic Files" on my fast SSD internal drive (so as to have the exact same file path) and move (copy) the high resolution TIFFs into there. Automatic swap of High for Low will follow when I next open the Publisher Document which will now be ready for export.

You can use .afphoto files if you want instead of TIFFs just make sure that the copies are tiny in file size, resize them proportionally Flatten them so they are small. You want files in the less than a megabyte in file size. They are just place holders and will be replace by the 200MB images.

The trick is to use files with of the same type* for Preview and Print resolution. .aphoto for .aphoto, .TIFF for .TIFF, .JPEG for .JPEG etc. All this is best thought about long before you start working on the document.

 

* And most importantly file extension, .JPEG for .JPEG not .JPEG for .jpg Otherwise the automatic linking will fail.

[mzed]

2 hours ago, Old Bruce said:

100 x 200MB = 20 GB. That is a lot of memory required.

NAS and 20 GB would not be a problem if Publisher would let me decide whether, how and when I want to reload. Renaming folders and using low-resolution images are time-consuming practices from other eras and should no longer be necessary in modern software.

[R C-R]

1 hour ago, mzed said:

NAS and 20 GB would not be a problem if Publisher would let me decide whether, how and when I want to reload.

Actually, the Affinity apps already do not load everything into memory at once. This is part of the reason it is not a good idea to risk file corruption by using a NAS or remote server while working on documents in Affinity. These external sources sometimes can't update what gets passed back to them fast enough as part of the memory management built into the apps, so when they do need to get brought back into memory they are loading incorrect data.

But if you want to force the apps to load less into memory (& accept the performance hit) you can open Preferences > Performance & lower the RAM limit. You probably will not like what that does to performance or to the risks associated with using remote storage like a NAS. This also will probably degrade the OS level memory management's ability to provide the best overall performance by keeping things in RAM for as long as possible.

[mzed]

10 hours ago, R C-R said:

This is part of the reason it is not a good idea to risk file corruption by using a NAS or remote server while working on documents in Affinity.

I don't think that happens, as long as it's developed cleanly. There are usually enough safeguards for this on the OS side alone (buffers, prioritisation, integrity checks, etc.). I think it is no longer unusual to work with network storage. 

In relation to my concern, limiting the RAM consumption hardly makes sense. The only problem is that when the project file is opened, it is unnecessarily reloaded. As it is, Publisher does this in order to re-render all images to the document resolution, which temporarily requires RAM, performance and time. I would like to be able to control this behaviour myself, nothing more 🙂

[R C-R]

9 hours ago, mzed said:

I think it is no longer unusual to work with network storage. 

For some apps it isn't but a quick search through this forum will demonstrate it definitely is an issue for the Affinity apps due to how they use serialization & Mipmaps, among other things that most apps do not use.

The official advice is "use at your own risk."