Massive RAM consumption/leak?

[benwiggy]

I have a fairly innocuous document in Affinity Publisher on Mac Ventura. It's 78 pages, around 55 of which have a single PDF placed on them. (Content is mono, text and a few lines.) The remaining pages have one text box each. (It's the kind of document I would have run on an Mac with only 1Gb RAM, 20 years ago!)

I'm currently using 9.5Gb of memory, according to Activity Monitor! That seems excessive for such a simple document, and if I quit and re-open, it goes down to  5.44 Gb.

If I close the document, AP is still holding onto over 5Gb of memory. ("Real Mem" is also over 5Gb.) I've waited a while to see if it will go down, but no. If I open a smaller (2pp) document, it goes down to 2Gb.

I've got 32 Gb of RAM, and the RAM Usage Limit in Settings is set to use all of it. Is AP just being extravagant with the available resources, or is it a bit leaky?

Lucky I'm not doing something with layers and transparency and all the bells and whistles!

[Pšenda]

25 minutes ago, benwiggy said:

If I open a smaller (2pp) document, it goes down to 2Gb.

The standard work with available memory, which is mostly done at the OS level and the application can't even affect it (and doesn't need to), is that unnecessary memory is not freed - it doesn't waste time and energy. It is released only on a new request, when the memory allocation management must be done anyway. In general, the available memory is there to store some data, even if it is not needed right now.
In this situation - after closing a large document and holding a large part of memory, try to start some other application with large memory requirements (for example, open a very large document in Word, or an Internet browser with many pages, etc. Even if due to your large memory (32 GB) this can be a bit of a problem as the memory requirements will have to be really high.) Then it will be interesting to see if the allocated/held memory gets freed.

[benwiggy]

Other people have reported a number of memory leaks in Affinity apps, so I think this does warrant a look. A linked PDF that is less than 1Mb in size should not produce a document 10,000 times that size! 

[Bryan Rieger]

I’ve yet to see Serif comment on (or even acknowledge) any potential memory leak posts which doesn’t fill me with confidence. The Affinity 2.x apps memory usage behaviour doesn’t resemble any other application I use (macOS) in that they tend to hold on to memory, grabbing more with every new document), and never fully releasing it unless the app itself is closed (or crashes, which is sadly not uncommon in v2). Whether this is by design or not has never been addressed by Serif, and having another random user ramble on about ‘how memory works’ is getting old as only the Affinity apps exhibit this behaviour.

[thomaso]

49 minutes ago, benwiggy said:

Other people have reported a number of memory leaks in Affinity apps

Are you talking about reports concerning Affinity only – or general RAM handling in macOS / or in particular in Ventura?
https://forums.macrumors.com/threads/the-terrible-memory-leak-issues-still-exist-on-ventura-13-4-with-workaround.2390354/

52 minutes ago, benwiggy said:

A linked PDF that is less than 1Mb in size should not produce a document 10,000 times that size! 

Are talking about RAM – or about the .afpub document file size on disk after saving it?

[v_kyr]

8 minutes ago, thomaso said:

Are talking about RAM – or about the .afpub document file size on disk after saving it?

As the thread title is "Massive RAM consumption/leak?" it should be clear!

[thomaso]

3 minutes ago, v_kyr said:

As the thread title is "Massive RAM consumption/leak?" it should be clear!

Yes, it should be – but is it, too? That's why I am asking. The OP mentioned "document" size, not RAM, memory or cache file size.

A placed PDF increasing the .afpub document size would be a different problem than RAM consumption – while then RAM usage may be 'just' a consequence of such a document's file size.

[v_kyr]

6 minutes ago, thomaso said:

The OP mentioned "document" size, not RAM, memory or cache file size. ...

Read his initial posting about Activity Monitor and shown RAM usage ... etc.

[thomaso]

7 minutes ago, v_kyr said:

Read his initial posting about Activity Monitor and shown RAM usage ... etc.

Yes. The initial post mentions the term "document" five times, each of them refers to an .afpub document.

That is what seems unclear about the use of the term "document" in OP's second post (to which I referred in my question).

[v_kyr]

4 minutes ago, thomaso said:

That is what seems unclear about the use of the term "document" in OP's second post (to which I referred in my question).

The OP will probably better can tell you here than I.

[Pšenda]

46 minutes ago, v_kyr said:

As the thread title is "Massive RAM consumption/leak?" it should be clear!

The wording "produce a document" doesn't sound like a RAM topic.

[thomaso]

Just now, v_kyr said:

The OP will probably better can tell you here than I.

That's why I asked the OP, not you ;•)

Background / possible hint: If you place a PDF which contains uncurved vector text only into an Affinity document it gets placed with blend mode "Passthrough" while its content gets created as preview + saved with this document as pixel image. Then, depending on the amount of text, font size and PDF pages, 'just' text may result in a massively larger .afpub file size then its linked (or embedded) PDF.

[v_kyr]

20 minutes ago, Pšenda said:

The wording "produce a document" doesn't sound like a RAM topic.

The OP's first post is about RAM usage and memory leaks, in the second one he expands that to the document sizings. Where the later has also to do with memory usage here, since every doc you create occupies memory during the apps runtime and if a document size increases massively during altering the document, that indirectly also means that the in app by the doc used memory increased here accordingly. The whole should be visable/detectable via memory monitors like Activity Monitor ... etc.

[Bryan Rieger]

34 minutes ago, LondonSquirrel said:

By itself that is not a memory leak. For example, if I open the assets panel, those are loaded into memory. If I close the document, those assets seem to be kept in memory for faster use next time (if I use them). That is a valid design choice and is not a memory leak.

But welcome to programming in the 2020s, where we now need 20 times as much memory to achieve what we were doing 20 years ago. I still have CorelDraw 9 in a VM. It flies. In general I blame bloated GUI libraries and OOP.

Yeah, to be fair I've always said 'suspected', 'potential', or 'smells like a' memory leak. As you said, there are many design decisions at play which could contribute to these observed behaviours, including loading assets, caching strategies, font loading/rendering, linked documents, pass-through operations, etc. If this behaviour is by design it may have some unintended consequences that may need to be addressed as in my experience I've noticed it negatively impacting my system performance.

As for programming today, I hear ya. It's one of the main reasons why I chose to leave the industry a couple of years ago—far too many leaky abstractions piled on each other, or if you like, it's 'turtles all the way down'.

[v_kyr]

4 minutes ago, LondonSquirrel said:

By itself that is not a memory leak. For example, if I open the assets panel, those are loaded into memory.

Make that also styles + font panels etc. - But if those resources are not shared right between docs (a One to Many usage), so that every new created doc again occupies memory for holding up the already once loaded resources, then it will get a waste of runtime memory. Further if allocate memory and release bugs are in the routines for all that, then those can have mem leaks too.

13 minutes ago, LondonSquirrel said:

But welcome to programming in the 2020s, where we now need 20 times as much memory to achieve what we were doing 20 years ago.

  Sadly true, but let's also not forget that complexity and techniques used in app development has steadily increased always too over time, in contrast to old former times.

16 minutes ago, LondonSquirrel said:

In general I blame bloated GUI libraries and OOP.

It always depends, for GUI libraries & OOP about their software technical design/model and of course the quality of their implementation. Without OOP building complex GUI libs and the usage of many GUI widgets (...and related features) wouldn't be that easily, from a reusability point of view and also implementation wise to build those.

[Bryan Rieger]

8 minutes ago, LondonSquirrel said:

A trend to bloat. "Memory is cheap", so they say.

Disk space also, so they say, yet many consumers still opt for lower storage and memory specs to save money, while companies continue to bloat the size of their applications (Universal builds for Apple Silicon/Intel are terrible in this regards).

[Bryan Rieger]

37 minutes ago, v_kyr said:

It always depends, for GUI libraries & OOP about their software technical design/model and of course the quality of their implementation. Without OOP building complex GUI libs and the usage of many GUI widgets (...and related features) wouldn't be that easily, from a reusability point of view and also implementation wise to build those.

Theoretically there's nothing wrong with GUI libs and OOP, but reality often shows us otherwise. Operating systems typically provide a comprehensive set of UI elements, design patterns, and UX guidelines that are often overlooked (or discarded entirely) by designers and developers today, choosing instead to 'innovate' in areas that are outside of their core product, such as new UI components and patterns that differ enough from the underlying system as to only serve to confuse users who are usually most familiar with the operating system they use everyday. There are absolutely use cases for new interface elements in many products, but rarely is there a use case to create (and maintain) an entirely new UI stack/abstraction which often never feels right on any platform (ahem, Serif).

As for OOP, great idea, but in the hands of an 'architecture astronaut' it can quickly turn into a mass of largely meaningless abstractions only understood by the author themselves (at least that's the hope), which in turn forces those folks using said libraries to come up with extensive workarounds, which leads to ongoing maintenance headaches, and ultimately a level of technical debt which inhibits any further meaningful progress on the product.

[Pšenda]

1 minute ago, Bryan Rieger said:

only understood by the author themselves

And it is the better case 🙂

[v_kyr]

10 minutes ago, LondonSquirrel said:

But... we were creating complex GUIs pre 2000. Maybe they weren't as spoinky as they are now. Maybe they took more time to implement. But it was not impossible.

Well but it often was a nightmare too here then.

As an real-life example, I once upon a time ported the whole Sun XView OpenLook libs and stuff (~30 MB of compressed source code) over to NeXTstep (NeXT Computers). The Sun XView system, even it was implemented in C and thus for/with Sun SPARC RISC-CPU workstaions, was itself modeled to offer OOP capabilities. But the NeXT of course had a Motorolla 68040 CPU. I had to revisit all the Sun C code and had some hundreds of compile & runtime errors under NeXTstep which really drived me crazy, since on a Sun SPARC all compiled here fine without any issues & runtime errors. To shorten things here, the overall problem was that the Sun C source code was full of C pointer dereferencing errors (hundreds of them) which the NeXTstep Gnu C based compilation was only at runtime picky about and so very lately just during runtime crashed XView example apps then. I really wondered that the whole source code that way did run fine under Sun SPARCs and began to recherche where the main and problematic differences were. The reason it all worked fine in Suns was, that their Sun C compiler implementation for SPARC CPUs doesn't care about not right (faulty) setup C dereferencing pointers, it handled that on it's own during compilation and so fixed faulty pointer code during compilation. So on a Sun SPARC you could left out a usually needed pointer dereferencing and the Sun compiler & CPU will straighten out this. Of course that behavior wasn't the case with NeXT's C compiler and the Motorola CPU and so I had to correct all those hundreds of pointer handling problems manually. - It took me nearly a half year to adapt all Sun XView code the right fixed way for NeXTstep.

[v_kyr]

43 minutes ago, Bryan Rieger said:

Disk space also, so they say, yet many consumers still opt for lower storage and memory specs to save money, while companies continue to bloat the size of their applications (Universal builds for Apple Silicon/Intel are terrible in this regards).

JIP universal builds of apps do occupy a bunch of unnecessary disk space for often unused additional architectures. And having both architectures in apps for Silicon hardware is only needed for things which haven't been ported over to Mx SocS completely and thus where the Intel code will be then run under Rosetta 2.

For universal build apps which are completely ported and adapted over to Silicon hardware, I personally do strip out the unneeded intel architecture libraries and binaries in order to shrink app sizes and free up some disk space! - And on Intel based Macs I instead do strip out the Arm architectures, in order to shrink app sizes there.

 

NOTE: that there are also plain macOS apps which deal/call internally macOS lipo and thus can do the same here, aka stripping out unneeded architectures.

 

 

[Bryan Rieger]

Stripping unused architectures using lipo isn't something the average user (of graphic design and photography applications) is going to do, even with a python script. Of course some Universal apps still have Intel components and plug-ins that haven't yet been ported and thus require Rosetta, but apps such as the Affinity suite, that have been fully ported to Apple Silicon and don't require Rosetta really should offer architecture specific builds for their users, especially when the size of the application reaches into the multiple GBs. Being respectful of a users' choice (for whatever reasons) to purchase a system with 128GB or 256GB of storage space will only pay dividends down the road. It's their storage space, not yours.

[v_kyr]

38 minutes ago, Bryan Rieger said:

Theoretically there's nothing wrong with GUI libs and OOP, but reality often shows us otherwise. Operating systems typically provide a comprehensive set of UI elements, design patterns, and UX guidelines that are often overlooked (or discarded entirely) by designers and developers today, choosing instead to 'innovate' in areas that are outside of their core product, such as new UI components and patterns that differ enough from the underlying system as to only serve to confuse users who are usually most familiar with the operating system they use everyday. There are absolutely use cases for new interface elements in many products, but rarely is there a use case to create (and maintain) an entirely new UI stack/abstraction which often never feels right on any platform (ahem, Serif).

That depends on the respective individual developers!  And I blame more the nowadays frontend & design based people for all this. Nowadays everybody is going to build his own iOS & desktop apps and the like, especially designers and the graphics people are proone to this, as they often do more think to implement something overall more cool & modern and new looking. Common old school developers follow more the respective OS UI style guidelines here and also do reuse more the common by an OS provided libs and frameworks for such tasks.

48 minutes ago, Bryan Rieger said:

As for OOP, great idea, but in the hands of an 'architecture astronaut' it can quickly turn into a mass of largely meaningless abstractions only understood by the author themselves (at least that's the hope), which in turn forces those folks using said libraries to come up with extensive workarounds, which leads to ongoing maintenance headaches, and ultimately a level of technical debt which inhibits any further meaningful progress on the product.

Good software engineers/architects do usually follow more what they once learned and what is approved in this OOP regard ...

• for OOP based modeling
• approved Design Patterns
• reusing stable frameworks/libs/services
• and they also refactor all if needed
• ...

[v_kyr]

9 minutes ago, Bryan Rieger said:

Stripping unused architectures using lipo isn't something the average user (of graphic design and photography applications) is going to do, even with a python script.

The Py script is my way here, average and inexperienced users can use instead tools like ARMless + lipo etc.

11 minutes ago, Bryan Rieger said:

really should offer architecture specific builds for their users, especially when the application size reaches into the multiple GBs

Some sources do so, but of course not all, the majority of commercial software app providers does only provide fat binaries (multi-architecture apps) so they are more sure their tryout/download of apps can run on either (a wider) system base. - Furthermore, it also takes more effort to build and test independent architecture builds (... especially for small companies with few employees, or single fighting people).

[R C-R]

7 hours ago, Bryan Rieger said:

Of course some Universal apps still have Intel components and plug-ins that haven't yet been ported and thus require Rosetta, but apps such as the Affinity suite, that have been fully ported to Apple Silicon and don't require Rosetta really should offer architecture specific builds for their users, especially when the size of the application reaches into the multiple GBs.

But at least for AP, aren't there some third party plugins that won't work on M series Macs unless AP is running using Rosetta 2?