Hugely inflated PDF export filesize after data merge - how to avoid?

[Blake_S]

I have a 3.5MB PDF file which I use a background, placed with passthrough mode, and then data merge 2 text fields and 1 image frame onto it.

There are total of about 400 records.

Image frame has a PNG qr-code merged into it, average size of one such image is 3 KB, the size of all PNG images together - 1.16 MB - this is pretty much the only variable data which actually takes space.  
A total size of a merged file should not take more than around 5-10 MB. 

So, I go to export the file as a PDF - and get a 180MB file as a result. Why? Any way to get an actually reasonable filesize?
Dropping the DPI doesn't really do much - with 192 DPI instead of 300 I still get a 140MB file, and further reduction would destroy image quality.

[NathanC]

Hi @Blake_S,

This could be related to the PDF file used for the background causing the PDF bloat. Could you provide me with the .afpub file along with the data merge file and image so I can investigate further? I've provided a private Dropbox link below.

https://www.dropbox.com/request/gC1ZAF8HYtLlocw8H0O0

If you could also provide me with a screenshot of your desired PDF export settings that would be great.

 

[Blake_S]

Sent the test files.
I cut down the number of records there to 200, and with export after data merge at 300 DPI this results in a 86.5 MB file being produced.

Some additional info:

I decided to check a "Reduce File Size" function from Acrobat Pro on the 180MB exported PDF,
and it generated a 5.5MB file, which is pretty much exactly what its size should have been in the first place.
However, it also destroyed PNG image quality, introducing artifacts, so the file itself becomes useless.
Everything except PNGs however looks just fine, since it was all vector content.

[NathanC]

Hi @Blake_S,

Thanks for sending over the files,

I found that after rasterising the PDF's on the master page and then generating the data merge resulted in a PDF of only 4mb~ at 300DPI in comparison to the 86MB PDF using passthrough. If you do want to keep the PDF content as vectors this does result in the file bloat to 86MB, This appears to be very similar to the issue below which was fixed in 1.10.0.

 

For now, rasterising with the document at 300DPI does prevent this, however I'll be looking into this further with regard to exporting using PDF passthrough.

[Blake_S]

Has this been fixed in v2?

Just got another file that is unusable from Publisher, had to re-make it in InDesign CS5 instead.

Initial file is a PDF with one page containing a b7unch of vectors and text, no images, 454 KB

5000 records, just one number field

Results:

Affinity Publisher: creating merged document is fast, then Publisher was hanging for half a minute when selecting export, creating PDF was slow.
Publisher also ballooned to 13GB in RAM, and had written 20 GB to harddrive, due to running out of free RAM.
Resulting merged PDF is 223 MB, unusable

InDesign CS5: quite fast merge directly to a PDF file, around 20 seconds, resulting file is 3MB. No noticeable increase in RAM consumption.

Before anyone chimes in with "just rasterise the initial file" - quality of rasterised text sucks on print, its very noticeably much worse.
In nearly all cases rasterising vectors or text is a terrible idea.

 

[lacerto]

Obsolete.

[Blake_S]

2 hours ago, lacerto said:

b) Having hardware acceleration enabled

Nope, not in my case. I disabled hardware acceleration, and this did nothing.

Affinity Publisher chugs RAM when it tried to display a merged file.
In the process of merging it doesn't use any additional RAM,
but as soon as the merging is done and it tries to display the newly created file, that's when it happens.
Atm looking at 11GB RAM Publisher, after disabling hardware acceleration.

_________________

About RGB - I re-printed the initial file in Acrobat, choosing the acrobat color management and convert all colors to sRGB options.
In Output Preview I confirmed that the result was indeed all in RGB - there was no CMYK or Grayscale objects left.
Re-printed file is 55 KB in size.

I also switched document color mode to sRGB in Publisher.

For the merging - nothing has changed. Publisher still chugged 12GB RAM and written the rest to disk, speed was the same.

Selecting Export locked the Publisher in "Not Responding" state. I have waited around 5 minutes now, its still not responding so far.

This is quite an "experience" compared to getting a merged PDF in 20 seconds in a 10+ year old version of InDesign.

 

[lacerto]

Obsolete.

[Blake_S]

After about a minute more, it actually unfroze.

Export completed successfully, about in the same time it took InDesign.

Resulting merged PDF file is 235 MB, so absolutely nothing had changed after converting to sRGB.

So nope, didn't help either.

[lacerto]

Obsolete.

[Blake_S]

1 minute ago, lacerto said:

Just for the sake of testing, what if the resulting PDF is RGB, too (not that it matters, if it needs to CMYK, but just to check if the color mode has anything to do with it).

when exporting I chose "colour space: as document", so it was exporting into sRGB
checked the merged file in Acrobat and indeed, its all in RGB

 

[lacerto]

Obsolete.

[Blake_S]

9 minutes ago, lacerto said:

Ok. Does your merge combine any images based on picture frames on actual pages, or are they all on master pages? I think that ones on master pages should save the image only once but I have not checked whether an identical image collected with actual pages would result in the same image saved multiple times.

For this test I chose pretty much an ideal case for merging - file has no images. Only text and some vector lines.
And no images or PDFs are to be merged into it, the only variable field is text.
I specifically checked this in Acrobat Output Preview - selecting "Images" shows nothing.

[Efvee]

Are you placing your images on a Master page and then assigning it to your content page?