APub book export regularly crashes on Mac Book Pro M2

[Gianni Becattini]

I also have a recurrent problem in exporting books, the system crashes (with the dump window). It seems that the problem is independent from the export setting.

My problem is that now it crashes ALWAYS with TWO different (rather big) documents, which constitute my entire job.

The problem now arises only on M2 MacBook Pro, while, until now, it survives on the iMac i9. They have different memory sizes: 16 on the M2 and 64 on the i9. Could this explain the different behavior?

I am very concerned, because if also the i9 starts crashing I am stuck with the work.

Any idea? Thanks 

[v_kyr]

1 hour ago, Gianni Becattini said:

My problem is that now it crashes ALWAYS with TWO different (rather big) documents, which constitute my entire job.

The problem now arises only on M2 MacBook Pro, while, until now, it survives on the iMac i9. They have different memory sizes: 16 on the M2 and 64 on the i9. Could this explain the different behavior?

Maybe, how huge are the documents size wise on disk and how huge is your MBP M2 SSD (...is there enough free disk space on)?  And how much system memory does APub use/occupies on the iMac, view/inspect that via the macOS Activity Monitor, when the books are loaded into APub there then?

Other than that, there might also still be some Arm architecture (Apple Silicon) related bugs in conjunction with APub under macOS Arm based APIs etc..

[MikeTO]

On the M2 that crashes, try exporting all of the chapters one by one on their own with otherwise identical settings. Do any of them crash?

Try opening the book on the M2 without the linked resources and then exporting - does it still crash? If not, the problem may be with the linked resources. Check that all of them are fully downloaded from iCloud if any are on iCloud.

Good luck

[Gianni Becattini]

I could do the check.

On the i9, if I just open the book (without opening the chapters) the APub process uses 848M. If I open all the chapters, it raises to 28.11 Gb.

During the export process, it slowly increases up to 21 Gb. When finished, it maintains 17 G allocated until quit.

On the M2 the test is not significant for the crash (at about 10 Gb used).

Both have plenty of disk space available (600 on i9, 400 on M2)

[Gianni Becattini]

I exported each single chapter without problems.

[Lee_T]

Hi Gianni Becattini,

Could you please send the documents to this location for us to test with? https://www.dropbox.com/request/RKOwNhEgqLiVsxxvQUu4 (private link).

It may be worth trying to disable hardware acceleration in Affinity Publisher > Settings > Peformance, additionally if you could send us any crash logs the application has recorded using the following guide, that would be most helpful.

Lee

 

 

[Gianni Becattini]

sorry, saw just now, tomorrow I do

[Gianni Becattini]

Sorry, I am late, but uploading all seems not practical. The project with the images is about 200 G. Could it be enough the book and the afpub files?

Thanks

[Gianni Becattini]

Sorry, I understand that I didn't give you enough information, but I believe I can confirm you that the problem is not only repetitive, but relatively independent from specific files. In my case EVERY BIG PROJECT behaves this way (I am working on FOUR). The available memory really seems an ISSUE (all the projects compile correctly on the 64G i9 and each project regularly crashes on the 16G M2).

Probably it is my specific approach with aphoto files always used directly, but renouncing to this feature would be very time-consuming when you have hundredths of photos with frequent changes/corrections.

[walt.farrell]

9 hours ago, Gianni Becattini said:

Probably it is my specific approach with aphoto files always used directly

That could certainly contribute to memory problems. Exporting your .afphoto files as an image format (PNG, TIFF, or JPG) would be better for memory management, and when you perform the export, Publisher can detect the changed file and update it automatically in your project.

[Gianni Becattini]

Thanks for replying, Walt, and best wishes!

Changing file format for images is for me not real; it would require weeks and this is for me just an hobby. But I hope that in time all these bugs will be corrected. Having the aphoto file directly included avoids lot of errors and makes maintenance/correction very fast. I insist that a trace file of what Publisher does could be a big help for both the users and the developers; in the worst case, you could try to decode where the catastrophe happened.

In the meantime, I wish a long life to my 64G Mac...

Thanks for all the help you gave me in 2023 and a wonderful 2024 to you and everybody here!

Gianni

[Medical Officer Bones]

18 hours ago, Gianni Becattini said:

In my case EVERY BIG PROJECT behaves this way (I am working on FOUR). The available memory really seems an ISSUE (all the projects compile correctly on the 64G i9 and each project regularly crashes on the 16G M2).

...I have to ask: are you really that surprised that your 64GB i9 is capable of exporting your ~30GB project versus the 16GB M2 Mac that crashes?

Even though Apple's new memory architecture can deal somewhat with RAM shortages using its unified memory blah blah and its M2 RAM efficiency, the fact remains your M2 Mac's core RAM is seriously overtaxed by your project's memory requirements.

I would actually have been amazed if Affinity would be able to finish that export on the M2 without any hitch or slowdowns!

Your primary problem is very very simple: 16GB RAM doesn't cut it for the work that you need to do with it. That is the straight-forward answer to the issues that you experience.

You'd need at least a 32GB M2 Mac. I speak from experience: I work with 3D files that require 64GB or more RAM. I tested with M2 Studio Macs that have 32GB installed and my files bring those 32GB M2 Macs to their knees at work. The software crashes even when I attempt to load those files. The entire MacOS crashes willy-nilly, and I had to reboot those machines several times while testing. I

In short: it's not an Affinity bug as far as I can tell. It's simply the lack of memory in your M2 Mac: you need at least 32GB to work comfortably within the context of your work requirements. Apple might sing the magical wonders of its new M1/M2 memory architecture and its efficiency, but in real life severe lack of RAM means something's gotta give, either in performance or in stability, or both.

Your solution would be to upgrade your M2 Mac to 32GB or more RAM. I would take it on the safe side and get a 64GB M2 Mac, because your OS, other software running, and the screen video ram gobble up parts of that 32GB as well.

Unfortunately --Apple being Apple-- that means sending in your Mac for a hefty RAM upgrade price or getting a new one with more memory. Nowadays end-users cannot upgrade the RAM of their Macs anymore. 😞

[Gianni Becattini]

Yes, you're right... but since many years, the virtual memory should swaps to disk and make the machine even infinitely slow... but never crash. One trillion years ago I wrote an operating system for a Z280 processor, a Z80 with Virtual Memory Manager, something that most people never heard... OK, sometimes it crashed, but I was not Apple and the Z280 was not an M2.

The application should not be aware of the available memory, it's an OS duty to make it transparent (besides speed).

I could bet that the crash is not due to memory but to something else. Future releases of APub or a future purchase of a fully-memorized M2 will tell us the truth...

However I fully agree with you, thanks

[Medical Officer Bones]

Z80! Those were the times. I learned basic Z80 assembly language on my Amstrad CPC 6128! But quickly switched to C.

[v_kyr]

20 minutes ago, Medical Officer Bones said:

Even though Apple's new memory architecture can deal somewhat with RAM shortages using its unified memory blah blah and its M2 RAM efficiency, the fact remains your M2 Mac's core RAM is seriously overtaxed by your project's memory requirements.

I would actually have been amazed if Affinity would be able to finish that export on the M2 without any hitch or slowdowns!

Your primary problem is very very simple: 16GB RAM doesn't cut it for the work that you need to do with it. That is the straight-forward answer to the issues that you experience.

...

Let's add to that, that there's a reason I always recommend to people here in the forum, who are looking for buying some Apple Silicon Mx SoC based hardware, to not be that short-sighted and buying those limited entry configuration (8/16 GB RAM, 256/512 GB SSD) devices, just in order to save some money on the cheaper side. They won't be happy with these sooner or later, especially if they try to do something non-trivial, aka more memory hungry things, with those machines then.

Having nothing RAM- and storage space wise in reserve here nowadays is always pretty bad, also in terms of growing future software demands, as you can't upgrade those glued in RAM & SSD components nowadays anymore! - So don't listen to these people (the chatterboxes) who then tell you that you can do everything super well on an 8GB RAM Mx hardware based computer, same as they did always years ago with their old computers. 🙉

 

[MikeTO]

2 hours ago, Medical Officer Bones said:

Your primary problem is very very simple: 16GB RAM doesn't cut it for the work that you need to do with it. That is the straight-forward answer to the issues that you experience.

You'd need at least a 32GB M2 Mac.

I'm using a 16GB M1 Pro and find it sufficient for books of several hundred pages with several hundred high res photos. I intended to buy it with 32GB but messed up. After ordering I was worried that it would be an issue because memory was a bottleneck on my Intel MBP with 16GB, but it's been fine. Even if I force Publisher to load all of my book's 14GB of linked photos into memory, macOS handles the memory swaps without me noticing.

I will order 32GB next time for other reasons and I wish I'd done so this time because as @v_kyr wrote, it's good to have something in reserve.

On 12/11/2023 at 5:18 PM, Gianni Becattini said:

I exported each single chapter without problems.

I think this is key and we've seen this with other Book export problems that were fixed in the recent 2.x updates. There were some simple books that exported as separate chapters but not as a single book.

BTW, did you disable hardware acceleration (metal) as suggested by Lee?

2 hours ago, Gianni Becattini said:

I wrote an operating system for a Z280 processor, a Z80 with Virtual Memory Manager, something that most people never heard...

I thought I knew all of the 1980s and 90s processors but I've never heard of the Z280!

[R C-R]

9 minutes ago, MikeTO said:

I'm using a 16GB M1 Pro and find it sufficient for books of several hundred pages with several hundred high res photos. I intended to buy it with 32GB but messed up. After ordering I was worried that it would be an issue because memory was a bottleneck on my Intel MBP with 16GB, but it's been fine. Even if I force Publisher to load all of my book's 14GB of linked photos into memory, macOS handles the memory swaps without me noticing.

That is exactly what I would expect to happen. There is no reason for the app to crash. It should at worst just start getting sluggish if the OS has to constantly swap data out of RAM & back in again.

Besides, even if the Mac is maxed out at 32GB, there quite possibly eventually will be times when Affinity and/or other apps have several huge files open, far more than could all fit into even 32GB with enough room left for wired memory the OS needs to function.

[walt.farrell]

35 minutes ago, MikeTO said:

I'm using a 16GB M1 Pro and find it sufficient for books of several hundred pages with several hundred high res photos.

Have you tried it with .afphoto files instead of image files, as @Gianni Becattini is doing?

[MikeTO]

23 minutes ago, walt.farrell said:

Have you tried it with .afphoto files instead of image files, as @Gianni Becattini is doing?

I don't really use the afphoto file format so I don't have enough files to perform much of a test. While I avoid placing documents in my afpub files due to potential performance reasons, I hadn't really considered whether they'd consume more memory.

As we page through our Publisher documents and come across linked images and documents that we haven't viewed yet this session, Publisher needs to retrieve the image or document and render it at a higher resolution than the low resolution preview stored in the afpub file. If it's just a photo in the afphoto container, i.e., a single pixel layer, the memory required should be about the same as if it were TIFF or PNG, but if the afphoto has a bunch of layers and adjustment layers then Affinity will need to render all of that and it will take more memory than a flat image.