Memory consumption

[blw]

I just noticed that Publisher had sucked down a full 33GB of memory - I have a 40G system. I only noticed because some of the stats on the system seemed to be going kind of nuts. The system was definitely behaving and if that's normal, so be it. My main project has MANY images, mostly large ones, so it's not 100% surprising to eat a lot of RAM. But 33G is a lot...

Quitting and reloading the same file yields a memory footprint of slightly under 4GB, so I'd say it's virtually certain that something was leaking memory within the process. Judging by the magnitude and what I've been doing with the program, I'd guess it would have to be related to images. Anything I can do to better identify this?

[mac_heibu]

Perform a „Save As …“ to reorganise the file, and compare the file size  of the „Saved“ and „Saved As“ versions.

Additionally: For speed (and other)  reasons Affinity files are quite heavy. But: Why RAM is there to use it. Otherwise it isn’t necessary. (Are you on a Mac?)

[blw]

The file is 1.7GB in size before save as and 1.62GB in size after. Not materially different, I'd say.

> Are you on a Mac?

I did post this in the "beta on Mac" forum, but yes I'm on an iMac 17,1 with 40GB memory.

> For speed (and other)  reasons Affinity files are quite heavy

I get it. I did say - in my original note: "the system was definitely behaving and if that's normal, so be it."

> Why RAM is there to use it. Otherwise it isn’t necessary.

I know that. I've worked on Unix VM systems - professionally - so I have a fairly decent idea of what's going on. I neglected to mention that without the large Publisher footprint, the rest of my system readily grows to ~28-30 GB. So Publisher probably is displacing pages that would be productively going to other uses, albeit perhaps at lower priority. MacOS didn't go off and compress a bunch of stuff, so it would seem that the memory pressure is light, but present.

[fde101]

On 10/10/2018 at 6:13 PM, blw said:

I did post this in the "beta on Mac" forum, but yes I'm on an iMac 17,1 with 40GB memory.

There have been a few cases of people accidentally posting in the wrong place - mac_heibu probably just wanted to confirm.

[Chris B]

Hi blw,

The developers have always said we optimise the app to use any available resources so a CPU running at 100% or using all available RAM is a good thing. We should release the RAM when something else needs it or you quit the app. I'd say this is perfectly normal unless you notice any ill effects such as overheating or lag/freezing etc.

It seems clear to me that you are aware your machine is running fine even with such high RAM usage but you have made a fair observation. Please let us know if you do start to notice any issues with it running so high.

 

[blwfish]

I have more information on this. (I'm the same person who reported it originally, I'm having email trouble.)

Using a large file with MANY images in it, Publisher is consuming about 750MB "at idle." When I load the file, it goes up to about 1.7GB - that's to be expected. If I then scroll through the book, I can see the memory footprint get bigger and bigger as I cross more pages. On my 40MB desktop system, it was not really a problem to use 33GB. However, scrolling through 55 pages on my 16GB MacBook Pro DEFINITELY IS A PROBLEM as it blows out Publisher's footprint to around 17-18GB. That of course is 10 pounds in a five pound sack, so there's flour all over the floor.

If I close the file, memory footprint goes back to about 750-800MB within a few seconds. If I then load the file again and scroll, it repeats the behavior. For grins I loaded the file and then let it sit and do nothing. Of course, the footprint stays static in that case. This behavior is entirely repeatable or at least it was whilst I was on the plane this afternoon. If I let this continue, it will eventually lead to lots of spinning beachballs (that's how I noticed). I've been working around it by frequently saving, closing and reopening. Rather than scrolling to the work site within the book, I Document->Goto Page which seems to avoid the problem. This of course is only a workaround and it's only as good as long as I refrain from scrolling, which as you can imagine is not easy.

This system is a Macbook Pro 14,2 with 16GB memory. It too is running Mojave.

[Chris B]

Hey blwfish,

That definitely sounds like something we should look at. Can you upload the file you mentioned to this private Dropbox link. I've got plenty of big files as you can imagine but I want to make sure we're on the same page.

[blwfish]

Happy to do that, although it will probably have to wait until I get to a faster connection. I just tried on my hotel connection and it's crawling along at a bit less than 200KB/sec, which would take a really long time.

[blw]

I uploaded a rather large file to the link above. It's not in the same state that it was in when I hit the problem last week, but it's still doing the same thing now. (I brought it back to my 40GB machine, which makes it a lot easier to dodge the problem.)

[Chris B]

On 10/27/2018 at 4:46 AM, blw said:

I uploaded a rather large file to the link above. It's not in the same state that it was in when I hit the problem last week, but it's still doing the same thing now. (I brought it back to my 40GB machine, which makes it a lot easier to dodge the problem.)

Thanks for the file. I'm seeing a few issues with it. It crashes instantly on the last page when I view it. I'm not really getting chance to record its resource usage at the moment. I'll likely submit this to the developers to see what we can improve. I'll stick with it for now though. Thanks  

[blw]

Well that's curious. It does not crash on me going to the last page - on either of my systems.

Try this: pop up the Activity Monitor and go to the Memory tab, then open the file in Publisher. You should see ~730MB to start, then jump immediately to around 2GB. Document->Goto page 1, and it will be about the same. Then scroll through it, watching the memory use in Activity Monitor. By the time you get to page 50 or thereabouts, it should be huge, and at least on my systems, it won't have crashed...

[Chris B]

Sorry I should have said the crash only happens when I open the file on Windows. Dev are already looking at that.

I've got it open on my Mac at the moment and I'm just monitoring it and taking notes.

[blw]

In the intervening couple of weeks, I've upgraded to .162 and things appear to be quite different now. Although scrolling from newly loaded file does grow the memory size, it now grows only so far and then stops growing. I can't get my file to exceed 22GB now, even on my 40GB system - even though the file is now materially larger than it was back when I reported the bug. I've been scrolling back and forth all day and it's still sitting at about 21GB. Previously this would just go much further (over 33GB as previously noted). Something's been fixed, although I don't see a reference to such a thing in the release notes.

[Chris B]

I can't see a direct reference to any fix but I imagine optimisation happens throughout the development cycle. We've always said that if there are resources available we want to use them but not to the point where it causes the user any issues.

I'm glad you feel like this has been improved but please monitor it and resurrect your thread should you encounter any issues again.

[carl123]

13 hours ago, blw said:

I can't get my file to exceed 22GB now, even on my 40GB system

In Preferences, Performance what is your Ram Usage Limit set to? Maybe something changed there?

[blw]

On 11/13/2018 at 3:20 AM, carl123 said:

In Preferences, Performance what is your Ram Usage Limit set to? Maybe something changed there?

It's set to 40GB, the same size as physical RAM on the system. I'll assume that it has always been set this way as I didn't know it was there.

[blw]

On 11/13/2018 at 2:56 AM, Chris B said:

I can't see a direct reference to any fix but I imagine optimisation happens throughout the development cycle. We've always said that if there are resources available we want to use them but not to the point where it causes the user any issues.

I'm glad you feel like this has been improved but please monitor it and resurrect your thread should you encounter any issues again.

I did look in the release notes for a fix that I thought corresponded, but I agree that I don't see anything that specifically fixes it. However, I'm pretty certain that there is *something* different, as noted I had Publisher up to 33GB on a reasonably routine basis using the 40GB system. Perhaps the fix was really done somewhere else and this is a pleasant side effect.

[blw]

One more update on this matter. I'm now on 206, and the behavior has improved still more. I was able to load the same large file, scroll happily through it up, down and backwards for the better part of an hour as I cleaned up some messy image outlines. This on my 16GB laptop. Memory allocated to Publisher never exceeded about 8GB until I had to do a full export. The full export still did not blow the size up past about 17.2GB, and while that did drive memory pressure far into the yellow zone and occasionally for brief moments into the red zone (as shown by Activity Monitor's Memory display), I was able to generate an entire PDF/x4 output file at full resolution. I think this is a bit bigger than Publisher really wants to do on this size system (my images are mostly 36- or 45-mp and there are lots of them), but given that I'm on the road and have only the laptop, this was more than satisfactory. And of course, closing the file after generating the PDF put memory right back down to the ~1.5GB level. I think this is very well resolved. I have been watching the various release notes and I haven't seen anything that corresponds to this, but the issue clearly has received developer attention and now works as well as can reasonably be expected given that I was putting six pounds in a five pound sack.

[MauricioC]

I have a screaming fast AMD Ryzen 3600 series 6 core processor at 3.6GHz with a Radeon rx 500 series running 32GB Corsair Vengeance RGB PRO  (2x16GB) DDR4 3200MHz computer running Windows 10, everything on this machine is ridiculously fast.

Except Publisher. 

It takes Publisher about 10 to 20 minutes on this system to export a PDF. Here is a screenshot of Publisher around 15 minutes after saving a PDF file, emphasis on the word after- the process is no longer running, the computer should be pretty much idle, but ALL the available RAM is being used. The solid state disc is at 100% usage.

The next set of screenshots is the same file being run on Clip Studio Paint. The first is while the program is actually importing all 187 pages. Next image is Clip Studio Paint rendering an EPUB from those 187 pages modifying each image to fit the profile setup, so there is a lot of rendering going on. When the computer idles, the memory goes down to around twice the file size, pretty normal stuff. 

We've got a problem.

It's similar performance with Affinity Photo, but the latest releases have sometimes fixed the problem, sometimes it seems to come back. I've run Affinity on 4 different machines and bought this latest machine thinking that there is no way Affinity can possibly fill up all the memory on a machine like this. 

Why would any program need 10x the memory of the file itself? Why doesn't the program release the memory it is not using.