Win Publisher and Book Size issues

[Jeffjn]

I’d like to ask a basic question about the size of book it is possible to produce in Affinity Publisher.

If the Programmers/Support staff read this forum – is there an official limit to the number of pages or illustrations that can be handled? What's the biggest project you have tested?

To other users, what the largest book you have created?

I’m asking this because I have a problem. Currently I’ve created a 680 page book with 500 illustrations.

https://www.amazon.co.uk/Pinnacle-Studio-Revealed-Jeff-Naylor/dp/0993487122

Because I was on a deadline, rather than struggle with issues within Publisher I decided to split the book up. Previous versions of the book have been created successfully in PagePlus 9 (and earlier), all as a single file. The last version was actually bigger than the current version as I’ve pruned it a bit

In AP I’ve ended up with 4 files.

The Intro and TOC consists of the start of the book and a TOC that has been created by manually merging the TOCs from Parts 1 and 2.

Part 1 is 340 pages with 280 illustrations. The Embedded version of this file is 758Mb.

Part 2 is 336 pages with 260 illustrations. The Embedded version of this file 436Mb.

The Index is created in Part 1 by exporting the index from part 2 and manually re-sorting the entries.

4 PDFs are exported and merged in Adobe Acrobat DC.

Since the book has been published, I’ve spent days trying to create a merged single file from single chapters so I can update the book easily in the future. I’ve moved to the Beta because it has ability to Collect assets and Add pages from File and I was advised it treated Linked files differently.

However, whatever I do I end up with the same problems I encountered in version 1.7, which are:

Sometimes the larger file I’m adding to just refuses to finish saving or loading – AP hangs at a CPU usage of 15% (30% on my laptop) and after a very long time stops responding.

Sometimes I get an error message that tells me the maximum number of files are open and AP can’t proceed.

Often when working with a big file, closing it and trying to open another smaller one, even the small file refuses to open until I've restarted AP so it doesn't have a menory overhead - generally once I'm seeing AP using over 3Gb

I’ve tried working with files that are all Embedded and files that are all Linked. I’ve tried compiling the sections a chapter at a time. I’ve added more memory so now I’m working with 32Gb. I’ve tried using my laptop instead.

I realise that it’s quite possible that the manner in which I first created the publication may have resulted in some corruption – I mostly copied and pasted text from PP9 to AP, a chapter at a time, re-added the illustrations, and re-wrote each chapter as required. Other chapters were created from scratch with text from Word, and screenshots added via Affinity Designer so that they could be labelled. However, I don't really want to start creating it from scratch again if there is an inherent issue with AP.

If anyone has any experience of the above issues I’d be very pleased to hear them. I’d also like some advice as to how I can escalate this the Serif, as by moving from PP9 to AP I’ve ended up with a much more unwieldy production process.

TIA

Jeff

[Jeffjn]

Hope it's OK to bump this topic - given the winter break it may not have been seen by Serif. In the meantime I've tried a few other things, to no avail.

[walt.farrell]

What format are your illustrations, Jeff?

And are you Embedding or Linking them?

[Pšenda]

2 minutes ago, walt.farrell said:

And are you Embedding or Linking them?

On 12/31/2019 at 11:58 AM, Jeffjn said:

I’ve tried working with files that are all Embedded and files that are all Linked.

 

[Jeffjn]

I've worked with both Imbedded and Linked files - it happens with all Linked illustrations (the pub files sizes I've listed above are reduced significantly) and with them all Embbeded. Most of the illustrations are Affinity Designer file, with some are pngs or jpegs.

[v_kyr]

Well sounds like the sort of things people already fighted some +/- 24 years ago with. - In order to have less trouble, also in terms of approved working functionality and possible huge book size handling (here especially memory usage too) etc., my advice would be to use instead the right tools for such projects.

• some thoughts about book publishing

Everything else will give you sooner or later much headache, time consuming frickling around with shortcomings and forced workarounds etc. and the end result often won't look at all like initially planned or hoped.

 

[Jeffjn]

5 hours ago, v_kyr said:

Well sounds like the sort of things people already fighted some +/- 24 years ago with. - In order to have less trouble, also in terms of approved working functionality and possible huge book size handling (here especially memory usage too) etc., my advice would be to use instead the right tools for such projects.

• some thoughts about book publishing

Everything else will give you sooner or later much headache, time consuming frickling around with shortcomings and forced workarounds etc. and the end result often won't look at all like initially planned or hoped.

 

Thanks for the link, which I've read, but it looks like you didn't read the details of my post (or just wanted to point me to some negative feedback). I've been succesfully publishing books of the same or even larger size using Serif's Page Plus 9 (and earlier 32 bit versions) for about five years now. So, for the book I'm publishing (no footnotes, no bibliography, no maths formula), Serif Page Plus is exactly the right tool for the job . I've waited until Publisher had Float with text (and Designer had arrowheads) and for Publisher to be out of beta. The only thing I'm mssing is proper cross-referencing and I can work round that for now.

So I don't need another product - I could always go back to Page Plus. What I need is to know is if it is a design limitation, a bug or something corrupt within my own APub project that is causing me the inconveinince of having to split the project into two parts. If I can get answers to my questions I can decide how to proceed - which were:

If the Programmers/Support staff read this forum – is there an official limit to the number of pages or illustrations that can be handled? What's the biggest project you have tested?

To other users, what the largest book you have created?

[v_kyr]

I've saw your previous description and AFAI remember (though used PP in the past long time ago just for tryouts one or two times) PP had and offeres quite much more capabilities than the initial APub version now. Add to this that APub is pretty fresh and thus hasn't gone throw several development circles yet (also in terms of uncaught bugs), I won't expect it to be on par here.

Related to APub and possible book sizes, memory consumption, robustness, pitfalls etc., I can't tell since I don't and won't use it for such book publishing purposes yet. Further I fear that it's actual initial all in one file limited concept, instead of book part dividing/structuring mode, it's less optimal for larger book publications with a lot of images, references and so on. - In other words, I doubt it is actually very well suited for bigger book publications and if I would actually use it, than probably more for manageable smaller size brochures and the like.

So somebody else has to dive here in to tell you, if it might do the job for you now and yet as PP did before!

 

[Old Bruce]

59 minutes ago, Jeffjn said:

what the largest book you have created?

Did a text only test that ran about 500 pages and another with very few (under 20) illustrations that ran 800 pages. The largest size is going to be dependant on a number of factors:

Number and size (in bytes) of illustrations and whether they are resized by Publisher or are placed at 100%. The latter is preferred I do know that it is not always practical.

Shear volume of text.

Are there effects applied in Publisher to the images and or text.

Then there is the CPU, RAM and graphics card(s) plus disk speed (and you need to have a fair amount of empty space on the system drive for swapping memory) to take into consideration.

A couple of people can have very different results with what may sound like very similar sized projects on the same machine. 

You just have to try. Planning is key.

[Jeffjn]

11 hours ago, Old Bruce said:

Did a text only test that ran about 500 pages and another with very few (under 20) illustrations that ran 800 pages. The largest size is going to be dependant on a number of factors:

Number and size (in bytes) of illustrations and whether they are resized by Publisher or are placed at 100%. The latter is preferred I do know that it is not always practical. Shear volume of text.

Are there effects applied in Publisher to the images and or text.

Then there is the CPU, RAM and graphics card(s) plus disk speed (and you need to have a fair amount of empty space on the system drive for swapping memory) to take into consideration.

Thanks for those insights. The point about sizing illustrations is something I can investigate as I've been a bit erratic with including them, trying to take advantage of being able to import Designer files.

However, the CPU/RAM/Graphics issue seems less likely to be a cause of the failures I'm having. My desktop has a 8 core 4Ghz CPU, 32 Gb Ram and a 1070Ti GTX card. There is 300Gb free on the SSD C: drive and 2 Tb free on the data drive. However, the same point of failure (with the same symptoms descibed above) occurs with the same files on my laptop (4-core 2.5Ghz, 8Gb of Ram. 520 Intel graphics and 40Gb free on the only SSD drive). If computer resources were an issue, I would expect the laptop to fall over sooner.

You seem to have tested projects that approach my book size (on a Mac?) but what happenes when the test fails - what error messages of types of crashes do you experince? I'm particularly interested in the message about too many files being open.

Thanks for your help

[v_kyr]

On 12/31/2019 at 11:58 AM, Jeffjn said:

Sometimes the larger file I’m adding to just refuses to finish saving or loading – AP hangs at a CPU usage of 15% (30% on my laptop) and after a very long time stops responding.

Sometimes I get an error message that tells me the maximum number of files are open and AP can’t proceed.

Often when working with a big file, closing it and trying to open another smaller one, even the small file refuses to open until I've restarted AP so it doesn't have a menory overhead - generally once I'm seeing AP using over 3Gb

I’ve tried working with files that are all Embedded and files that are all Linked. I’ve tried compiling the sections a chapter at a time. I’ve added more memory so now I’m working with 32Gb. I’ve tried using my laptop instead.

I realise that it’s quite possible that the manner in which I first created the publication may have resulted in some corruption – I mostly copied and pasted text from PP9 to AP, a chapter at a time, re-added the illustrations, and re-wrote each chapter as required. Other chapters were created from scratch with text from Word, and screenshots added via Affinity Designer so that they could be labelled. However, I don't really want to start creating it from scratch again if there is an inherent issue with AP.

The error message "maximum number of files are open" sounds somehow like it has problems with file handles then, maybe linked in files are treated like open file handles internally. It's hard to tell since we don't know concrete how their file linking is implemented.

The memory overhead things you described (...closing big file, refuse to opening small file...) do sound like some objects which use resources such as file handles, graphics contexts, sockets, and so on aren't explicitely freed and thus still occupy the memory then.

Your hardware from the specs you mentioned should usually be easily handling the whole and so the problem is more app related memory handling and the like. For the copy/paste over of lot of text from PP or Word, it shouldn't be that much of a problem as far as APub doesn't try to preserve those initial formatting structures then. Meaning if it treats and keeps the copied over text portions as plain unformatted UTF-8 text then instead of possibly preformated text segments etc.

Did you tried all that always with and on the same initially created files (your chapters structure 4 files) or did you also tried to recreate individual ones of those newly with other proceed methods?

[walt.farrell]

57 minutes ago, v_kyr said:

The error message "maximum number of files are open" sounds somehow like it has problems with file handles then, maybe linked in files are treated like open file handles internally. It's hard to tell since we don't know concrete how their file linking is implemented.

That could also be related to the number of fonts installed, as Affinity's enumeration of the fonts seems to open each one (at least on Mac; not sure about Windows).

[v_kyr]

24 minutes ago, walt.farrell said:

That could also be related to the number of fonts installed, as Affinity's enumeration of the fonts seems to open each one (at least on Mac; not sure about Windows).

Good point, I've just thought about possibly many linked in image files here.

[Jeffjn]

Thank you gentlemen

I need to find a day to break down the book into chapters again, make sure that the illustrations aren't too big, and then reassemble the project. Hopefully I can then produce a reproducible bug that I can post on the bug forum (or even better, manage to create the whole book as one file).

Seems like the "too many files" error isn't something that is common?

[Old Bruce]

6 minutes ago, Jeffjn said:

Seems like the "too many files" error isn't something that is common?

I've never seen it. Fewer than 400 fonts on my system and I've yet to do more than 120 images and those are quite small size (bytes) pngs or tiffs.