UNFIXABLE?: Publisher PDF Export Changes CMYK Colors -- 100% Black to Rich Black

[Mike W077]

OK, you Publisher ninjas out there. Here is a tough, weird one:

THE BACKGROUND: We publish a large circulation (15k+) monthly magazine style newspaper on 32# newsprint on a cold-set web press. We set up the ads and special elements in Affinity Designer and export them to PDF/X-4 or X1-a. The color profiles for the document and the export are set up for newsprint. We also set up the Cover in Designer and export to PDF, have used both X4 and X1-a. For all of these, we set up the Ink Percentages just as we need them for the press. 

THE PROBLEM: When we export from Designer and check the Ink Percentages in Adobe Acrobat (Print Production/Output Preview), the ink density percentages are just as we set them. However, when we Place the PDFs exported from Designer into the publication in Publisher and then export the entire publication to PDF, either X-4 or X-1a, the colors SHIFT AND CHANGE. One major issue is black (the K in CMYK). There are times when the black needs to be 100% and the rest of the colors 0, and not a "rich black" (a mix of black with other colors to give it more richness). However, 100% black in PDF elements placed into Publisher ALWAYS seem to shift to a form of rich black, and other color densities can shift also. This even happens when we take the native Affinity Designer files and place them in Publisher and then export to PDF. When we export to PDF and check the colors with Acrobat Output Preview, the colors have shifted. We get rich black when we wanted 100% black only. When we sent these files to our printer, they cause no end of headaches with color separation and registration issues.

Yes, we tried setting the Assign/Convert under the Color setting in document setup to "Assign," but it automatically switches back! Argh!

Please don't just tell me to try various different color profiles and export settings. Have tried that approach and no color profile settings or export setting changes seem to make a difference. It seems to be a problem with the way the PDF export engine is implemented in Publisher. Am I wrong about this? Or maybe there something I'm missing. I have searched the forums and can't seem to find anything specific, although I seem to recall someone else having this type of issue too, but can't find it again. 

Here's how bad it is: Due to this and the inexplicable unchangeable default text box margin spinner settings in APub 2.0, we have gone back to InDesign. Color settings stay solid as a rock even after exporting to PDF with PDFs placed in the publication. However, I am still holding out hope for APub. 

OK, ninjas, anybody else run into this problem and have a fix for it. Fingers crossed. 

[Old Bruce]

1 hour ago, Mike W077 said:

THE PROBLEM: When we export from Designer

Here is my dumb question: Are you using a Print preset or a Press Ready preset? The former is RGB while the latter is CMYK.

[Mike W077]

3 minutes ago, Old Bruce said:

Here is my dumb question: Are you using a Print preset or a Press Ready preset? The former is RGB while the latter is CMYK.

No worries. The PDF presets for professional print work, i.e., X4 or X1-a, are for CMYK, and that is what we use. 

[laurent32]

Another dumb question (maybe ?)…

I remember having read "path through" possibilities when importing PDF.

And if I understood it correctly, the imported PDF would be treated differently ?

Have you tried that ?

[Mike W077]

On 3/8/2023 at 11:28 PM, laurent32 said:

Another dumb question (maybe ?)…

I remember having read "path through" possibilities when importing PDF.

And if I understood it correctly, the imported PDF would be treated differently ?

Have you tried that ?

Yes. Works for fonts. No difference on color changes.

[thomaso]

One more guess in this "compatibility rules" of Affinity when placing a PDF (or another container file type):

[Dazmondo77]

PDF passthrough in Affinity always seems a bit hit and miss for me especially when using client supplied PDF ads and usually flags in preflight as incompatible, which will export the RGB rasterised preview, which will convert all single blacks to rich blacks.

Ideally we really need a Text as curves import option for PDFs which would solve a lot of headaches

Seeing as you are creating all the content and have the fonts, and assuming you're using the same colour profile throughout your documents, have you tried changing PDF passthrough to Interpret which may solve this?

[lacerto]

The whole package is quite complex, but here are few rules of thumb whenever placing PDF content and exporting to PDF, and PDF/X is involved either in export or placement:

1) If there is placed content using PDF/X-1a or PDF/X-3 (which are version 1.4 when produced from within Affinity apps), you need to export using PDF/X-3 or PDF/X-4, or any non-PDF/X-based method, to not cause rasterization / translated color values (e.g. rich black).

2) If there is placed content using PDF/X-4 (which is version 1.6), you need to export using PDF/X-4, or any non-PDF/X-based method using PDF version 1.6 or later, to not cause rasterization / translated color values (e.g. rich black). 

3) If there is non-PDF/X-based placed content, you need to export using non-PDF/X-based method using the same or later PDF version than a placed PDF content, to not cause rasterization / translated color values (e.g. rich black). The most compatible choice would then be PDF 1.7. Whatever the non-PDF/X-based choice used, live transparencies (opacity values and blend modes) would not be flattened (because Affinity apps do not support PDF version 1.3, which would cause flattening). In Affinity apps transparency flattening is only used in PDF/X-1a and PDF/X-3. 

4) Placing PDF as interpreted causes failure to read the overprint status of native objects. Also, if the files use embedded CMYK profiles and there is a conflict with the export target, or if the files do not use embedded CMYK profile and the document working profile (as defined in Preferences) that these files will be assigned with, and there is a conflict with the export target, CMYK color values of the placed PDF content will be translated at export time. This basically corresponds a situation where non-document CMYK profile is defined at export time. Letting Affinity apps interpret the placed content may also result in failure to map fonts (even when they are installed on the system), especially if the PDFs have been created with non-Affinity software (e.g. Adobe apps or CorelDRAW), even on the same computer. 

Here are demo files:

a) Miscellaneous placed PDF content exported using PDF/X-1a (all placed content will be rasterized):

 pdfxcompatibility_pdfx1.pdf

b) Miscellaneous placed PDF content exported using PDF/X-4 (non-PDF-based content will be rasterized):

pdfxcompatibility_pdfx4.pdf

c) Miscellaneous placed PDF content exported using non-PDF/X-based PDF1.7 (note that Adobe Acrobat Pro shows the color values of the non-PDF/X-based PDF at the lower left corner incorrectly; the true color values are shown using the Object Inspector of the Output Preview dialog box):

pdfxcompatibility_pdf17.pdf

d) Example of an export file (PDF/X-4) using interpreted placed PDFs (note the lost overprints):

pdfxcompatibility_pdfx4_interpreted.pdf

A couple of further notes:

1. Affinity apps always convert RGB values of native objects (shapes and text) to CMYK, also when using PDF/X-3 or PDF/X-4 which allow RGB definitions (this is unlike e.g. InDesign). Additionally, PDF/X-3 also converts image color spaces to CMYK, disregarding the value of "Convert image color spaces" setting...[EDIT: After rechecking, this only seems to happen if there is need for transparency flattening, and InDesign does behave here similarly.]
2. When using PDF/X-1 or PDF/X-3, all transparencies are flattened by using rasterization instead of Boolean operations (which are tried to be used when exporting from Adobe apps or QuarkXPress).

A long story short: Export using PDF/X-4 (while having the content placed to be passed through), or using non-PDF/X-based version 1.7, and if using the latter, remember to uncheck the "Embed ICC profiles". The latter will keep placed non-PDF/X-based content unchanged, the former will not. In both cases, your export files will contain live transparencies, in case not flattened in the placed document or on the canvas. If you need to export transparency flattened PDF/X-1a content and you have them in placed PDF/X-4 content, you are out of luck. That means: the content will be rasterized unless you can flatten it in the source files and reproduce.

Note: These tests and files have been created using the 2.0.4 Windows version of Affinity apps, but versions 1.10.6 and the latest v2 beta behave similarly, on both Windows and macOS.

 

[Mike W077]

On 3/12/2023 at 1:13 AM, lacerto said:

The whole package is quite complex, but here are few rules of thumb whenever placing PDF content and exporting to PDF, and PDF/X is involved either in export or placement:

1) If there is placed content using PDF/X-1a or PDF/X-3 (which are version 1.4 when produced from within Affinity apps), you need to export using PDF/X-3 or PDF/X-4, or any non-PDF/X-based method, to not cause rasterization / translated color values (e.g. rich black).

2) If there is placed content using PDF/X-4 (which is version 1.6), you need to export using PDF/X-4, or any non-PDF/X-based method using PDF version 1.6 or later, to not cause rasterization / translated color values (e.g. rich black). 

3) If there is non-PDF/X-based placed content, you need to export using non-PDF/X-based method using the same or later PDF version than a placed PDF content, to not cause rasterization / translated color values (e.g. rich black). The most compatible choice would then be PDF 1.7. Whatever the non-PDF/X-based choice used, live transparencies (opacity values and blend modes) would not be flattened (because Affinity apps do not support PDF version 1.3, which would cause flattening). In Affinity apps transparency flattening is only used in PDF/X-1a and PDF/X-3. 

4) Placing PDF as interpreted causes failure to read the overprint status of native objects. Also, if the files use embedded CMYK profiles and there is a conflict with the export target, or if the files do not use embedded CMYK profile and the document working profile (as defined in Preferences) that these files will be assigned with, and there is a conflict with the export target, CMYK color values of the placed PDF content will be translated at export time. This basically corresponds a situation where non-document CMYK profile is defined at export time. Letting Affinity apps interpret the placed content may also result in failure to map fonts (even when they are installed on the system), especially if the PDFs have been created with non-Affinity software (e.g. Adobe apps or CorelDRAW), even on the same computer. 

Here are demo files:

a) Miscellaneous placed PDF content exported using PDF/X-1a (all placed content will be rasterized):

  pdfxcompatibility_pdfx1.pdf 5.26 MB · 0 downloads

b) Miscellaneous placed PDF content exported using PDF/X-4 (non-PDF-based content will be rasterized):

pdfxcompatibility_pdfx4.pdf 2.94 MB · 0 downloads

c) Miscellaneous placed PDFC context exported using non-PDF/X-based PDF1.7 (note that Adobe Acrobat Pro shows the color values of the non-PDF/X-based PDF at the lower left corner incorrectly; the true color values are shown using the Object Inspector of the Output Preview dialog box):

pdfxcompatibility_pdf17.pdf 762.37 kB · 1 download

d) Example of an export file (PDF/X-4) using interpreted placed PDFs (note the lost overprints):

pdfxcompatibility_pdfx4_interpreted.pdf 1.85 MB · 1 download

A couple of further notes:

1. Affinity apps always convert RGB values of native objects (shapes and text) to CMYK, also when using PDF/X-3 or PDF/X-4 which allow RGB definitions (this is unlike e.g. InDesign). Additionally, PDF/X-3 also converts image color spaces to CMYK, disregarding the value of "Convert image color spaces" setting...
2. When using PDF/X-1 or PDF/X-3, all transparencies are flattened by using rasterization instead of Boolean operations (which are tried to be used when exporting from Adobe apps or QuarkXPress).

A long story short: Export using PDF/X-4 (while having the content placed to be passed through), or using non-PDF/X-based version 1.7, and if using the latter, remember to uncheck the "Embed ICC profiles". The latter will keep placed non-PDF/X-based content unchanged, the former will not. In both cases, your export files will contain live transparencies, in case not flattened in the placed document or on the canvas. If you need to export transparency flattened PDF/X-1a content and you have them in placed PDF/X-4 content, you are out of luck. That means: the content will be rasterized unless you can flatten it in the source files and reproduce.

Note: These tests and files have been created using the 2.0.4 Windows version of Affinity apps, but versions 1.10.6 and the latest v2 beta behave similarly, on both Windows and macOS.

 

This is an amazing and exhaustive explanation. Thank you for your work. It also reveals a system that is WAAAAY too complicated for a production work flow. It shouldn't be this complicated. There should be a simple way to allow specified CMYK colors to flow through the export processes unchanged. We receive way too many PDF placeable files from customers and agencies, as well as elements — logos, other art — that we design up ourselves — to make anything this complicated work. APub should just let everything flow through to the final PDF, including the random RGB, and then let the RIP at the printer handle the interpretation. That's its job. That would make sure that specified CYMK values remain unchanged and let the RIP interpret the rest for best output. As it is, with APub apparently wants to interpret ALL color values if there are ANY rasterized or RGB elements in the publication — if I understand this correctly (and I might not. It is complicated.). Thanks again for your help and your work. I will review more extensively. 

[lacerto]

3 hours ago, Mike W077 said:

We receive way too many PDF placeable files from customers and agencies, as well as elements — logos, other art — that we design up ourselves — to make anything this complicated work.

Yes, this is a true problem -- it is unthinkable that 3rd party providers would be required to deliver material in so specific format as e.g. PDF/X-4 (and exclusively so without including mixed content in how they build up their PDF content). PDF/X-4 is often seen as a kind of a magic wand because of being basically tolerant of mixed content and letting late-bound (print-shop/RIP-based) rasterization. It is absurd that it should be incompatible with non-PDF/X based placed PDF content.

In addition, I have learned in last three or four years (during the time I have used Affinity app suite) that PDF production is not necessarily nearly as automated and "final" as I have thought it is. E.g. transparency flattening seems to be often something that is done as a prepress job rather than on RIP and basically something that requires skillful print personnel. Manual adjustments made by print personnel seem to be much more common than is generally known. Such routines might well be based on assumption that Adobe (or QuarkXPress or Corel) based production workflows have been used, so not only is the printer typically incapable of providing exact instructions (to avoid or correct erroneous output), but might actually end up producing unsatisfactory / unexpected results on paper because of such assumptions .

Affinity specific "PDF compatibility rules" are not something that print personnel is likely to know about.  There are other production quirks, too. All this means that in more complex productions some kind of prepress software is required not only to do proper preflight, but sometimes also to make corrections to production PDFs themselves.

[Mike W077]

1 hour ago, lacerto said:

There are other production quirks, too. All this means that in more complex productions some kind of prepress software is required not only to do proper preflight, but sometimes also to make corrections to production PDFs themselves.

You seem far more knowledgable on this than I am. What I know is that this last issue we went back to InDesign and things seemed to go smoother at the press, even though it was a bit of a re-learning curve for us. I really like Apub and Affinity products to use, and like Serif as a company, but if the suite is going to be used for serious production work, at least for us, I feel that this needs to be fixed. Again, thank you for your thoughtful and helpful posts. 

[akula7]

"Yes, this is a true problem -- it is unthinkable that 3rd party providers would be required to deliver material in so specific format as e.g. PDF/X-4 "

The question remains whether the developers are aware of this problem. Affinity is really a great suite, but without this problem being erased, Publischer really cannot be used professionally. I haven't heard any comments from the developers, maybe they don't know that this is a real problem?

[Mike W077]

5 minutes ago, akula7 said:

The question remains whether the developers are aware of this problem. Affinity is really a great suite, but without this problem being erased, Publischer really cannot be used professionally. I haven't heard any comments from the developers, maybe they don't know that this is a real problem?

Man, I don't know. It does not seem possible that they are not aware of it. However, stranger things have happened. How do we let the developers know other than posting here?

Is it some sort of strange licensing issue with Adobe? Does Quark have the same problem? Hmm. Stumped. 

BTW, I just tried some additional experiments yesterday. Same result. Anything created in APub that is exported to PDF retains its original CMYK values. However, any PDF imported and placed into an APub page, say an element from Designer, then re-exported from APub using PDF has its CMYK color values shifted dramatically. This is especially true with 100% black in a placed PDF, upon export to PDF from APub, being shifted to an almost equal mix of CMYK. Sorry, I can't use that when I go to press. Still back with InDesign. Bummer. 

[akula7]

 

10 hours ago, Mike W077 said:

Is it some sort of strange licensing issue with Adobe? Does Quark have the same problem? Hmm. Stumped.

No, Quark does not have this problem. I worked with Quark when InDesign did not exist
I don't think it's about Adobe licences. After all, the separation works with pdf 1.7 compatibility, but not with x-3/4