Small caps are not imported correctly

[mqudsi]

I have a PDF created in Adobe Illustrator that displays fine via multiple different SDKs and toolkits across different platforms. The PDF contains text (set in Iowan Old Style) that was configured via the "Character" panel in Illustrator 2022 to use small caps, as shown below:

 

 

The resulting PDF is not correctly imported; here is an example:

 

What it should look like:

 

What is displayed in Affinity Designer 1.10.4 when the PDF is opened (without selecting "favor editable text over fidelity"):

 

As you can see, the kerning is all wrong (all characters are rendered much closer together), the characters are all caps (the actually capitalized input is not rendered at a larger size than the remainder of the text), and the numbers no longer match the height of the neighboring letters.

Manually selecting the text input and turning on "Small Caps" in Affinity Designer corrects the height of the numeric characters at the end, but the text kerning remains incorrect - it seems that the underlying text is imported as all-uppercase even though it isn't:

This is verified by manually selecting the last three letters of "LIVE" and re-typing them (in lowercase). This, with the previous step of enabling small caps for the text box gives the corrected result:

 

I've attached the actual PDF exhibiting this incorrect behavior as exported by Adobe Illustrator (i.e. without any corrections from Affinity Designer).

Fundraising Dinner 2021 - Web.pdf

[walt.farrell]

Just to be sure: Do you have Iowan Old Style installed on the computer where you're running Designer and Opening that PDF file?

[mqudsi]

Walt, Please read my post carefully: this is not an issue with an unsupported font or else I wouldn't have been able to make the corrections in AD that I did to get fix it. I've identified the actual problem and it has to do with the associated text properties being lost on import. (Aside from that, this is all on the same PC.)

[walt.farrell]

17 minutes ago, mqudsi said:

and it has to do with the associated text properties being lost on import. 

You're right; I should have been able to figure out that you must have the font itself.

However, PDF files do not (as far as I know) support "text properties". They describe what fonts are being used, and tell you what code points or glyphs are being used, and contain glyph tables. They do not (again, as far as I know) have a concept of "small cap".

This often turns up as a problem in an Affinity application in a different way, when someone uses Affinity to export a PDF and sets (or leaves the default) option to "subset fonts", and then tries to Open the PDF again with their Affinity application. With only a font subset available, there is insufficient information to properly handle and recognize ligatures.

The solution (when exporting from Affinity) is to include the full fonts. Then the glyph tables in the PDF have appropriate information and ligatures can be recognized.

If I had to guess, since PDFs do not know what a small cap is, the same would apply to exporting PDFs from Affinity if the document used small caps. Just like ligatures, Affinity probably wouldn't be able to read the small caps properly if the fonts were subsetted in the exported PDF.

Does Illustrator also have an option to subset the fonts? Did you choose it? If so, could you tell Illustrator to include the full font, and if you do that does it resolve the problem?

[Old Bruce]

If you copy the text it is all uppercase. For small caps to work there needs to be uppercase and lowercase.

[mqudsi]

40 minutes ago, walt.farrell said:

You're right; I should have been able to figure out that you must have the font itself.

However, PDF files do not (as far as I know) support "text properties". They describe what fonts are being used, and tell you what code points or glyphs are being used, and contain glyph tables. They do not (again, as far as I know) have a concept of "small cap".

This often turns up as a problem in an Affinity application in a different way, when someone uses Affinity to export a PDF and sets (or leaves the default) option to "subset fonts", and then tries to Open the PDF again with their Affinity application. With only a font subset available, there is insufficient information to properly handle and recognize ligatures.

The solution (when exporting from Affinity) is to include the full fonts. Then the glyph tables in the PDF have appropriate information and ligatures can be recognized.

If I had to guess, since PDFs do not know what a small cap is, the same would apply to exporting PDFs from Affinity if the document used small caps. Just like ligatures, Affinity probably wouldn't be able to read the small caps properly if the fonts were subsetted in the exported PDF.

Does Illustrator also have an option to subset the fonts? Did you choose it? If so, could you tell Illustrator to include the full font, and if you do that does it resolve the problem?

Thanks for the reply and no worries about the earlier confusion. I just needed to make it clear that this wasn't a case of user error so we could get to the juicy stuff

I played around with import/export in Affinity Designer and I see what you mean about the subsetting. I think that the behavior in question is an AD bug though - an incorrect subset of the characters used in the document is subset in the PDF, because the mapping between lower- and upper-case underlying text vs rendered text is not taken into account. The lowercase glyphs from the font are subset instead of the uppercase glyphs that are actually rendered, so when I open the AD document in another editor (Illustrator), only the initial (capitalized) letter of the small caps text is rendered correctly, while the rest are rendered with the missing glyph symbol. This is not the behavior when going the other way around and exporting the document from Illustrator with font subsetting enabled.

I'm aware that the raw PostScript layer doesn't understand the concept of "small caps" but to go a step further, it doesn't understand any font-to-glyph (or glyph-to-font) mappings in the first place. Neither Illustrator nor AD are really working at the core PDF layer (intended for rendering, not editing), but rather using the additional metadata specifically emitted in the PDF to allow for editability.

For what it's worth, there is nothing preventing AD from doing whatever it is that Illustrator does when given the same blackbox PDF input. Illustrator is able to understand (via the metadata) that the text is in lowercase rendered in uppercase glyphs, and for AD to be truly compatible as a drop-in Illustrator replacement, imho it should too.

[quote]Does Illustrator also have an option to subset the fonts? Did you choose it? If so, could you tell Illustrator to include the full font, and if you do that does it resolve the problem?[/quote]

Illustrator has a font subsetting option enabled by default, but it does not matter if it is on or not with regards to how AD interprets the input (because Illustrator is correctly subsetting the glyphs in use).

BTW, this isn't just a matter of AD not being compatible with how Illustrator stores its text characteristics metadata (or else a lot more text metadata would be lost probably making it impossible to actually open any Illustrator-generated PDF in Affinity Designer for editing). From what I can see, the subtleties of how smallcaps text is exported, imported, and treated throughout the pipeline hasn't really been taken into account at all, even with Illustrator completely out of the picture. Aside from the matter of incorrect subsetting, AD can't understand its own smallcaps output if the exported PDF is imported right back into AD (the text is imported as lowercase). 

25 minutes ago, Old Bruce said:

If you copy the text it is all uppercase. For small caps to work there needs to be uppercase and lowercase.

That's just how smallcaps was interpreted for the PDF preview. The source file it was generated from had MiXeD caSE text w/ smallcaps on, the resulting text does copy as all-caps in a viewer but if the PDF file is opened in Illustrator instead, the correct MiXeD caSE text is preserved (w/ smallcaps effect).

[Old Bruce]

So as I understand the 'problem' Illustrator can understand/interpret the PDF which Illustrator has produced (using Adobe PDF library 16.03). Fine. Can Illustrator do so with another application's generated PDF?

[kenmcd]

38 minutes ago, mqudsi said:

From what I can see, the subtleties of how smallcaps text is exported, imported, and treated throughout the pipeline hasn't really been taken into account at all, even with Illustrator completely out of the picture. Aside from the matter of incorrect subsetting, AD can't understand its own smallcaps output if the exported PDF is imported right back into AD (the text is imported as lowercase). 

OpenType features are not stored in the PDF with the text, like in a word processor or DTP app.
So when you import a PDF, what is actually underlying those text shapes is a Unicode code point (usually).
Applying OpenType features in any application does not change the underlying code points.
The small caps feature is applied to lowercase text - it stays lowercase text - only the small cap glyph is displayed.
When you export a PDF from AD the lowercase text codes are under those small cap characters (as they should be).
In your example PDF from IL the underlying code points are for uppercase characters.
So any importing, or cut-and-paste is going to be uppercase.
That is what is there.

38 minutes ago, mqudsi said:

The source file it was generated from had MiXeD caSE text w/ smallcaps on, the resulting text does copy as all-caps in a viewer but if the PDF file is opened in Illustrator instead, the correct MiXeD caSE text is preserved (w/ smallcaps effect).

Based on this and a few other posts, it appears that Adopy may be embedding some sort of non-standard data in their PDFs which may enable them to round-trip OpenType features.
BUT, simply displaying the embedded small cap glyphs when the PDF is opened in IL does not mean they are doing this, it may be they are just displaying the outlines that are there.

What happens if you insert a cursor and start typing lowercase text?
Does it display as small caps? (that would be evidence that the OT feature is still applied).
If you can copy small cap text from this exported-and-re-opened-in-IL PDF, and paste it into a text editor,
and it appears as lowercase - that would be evidence it is still lowercase.

Both of these would point to something non-standard going on.
Because as I mentioned above, the underlying code points embedded are uppercase.

[mqudsi]

7 minutes ago, LibreTraining said:

Based on this and a few other posts, it appears that Adopy may be embedding some sort of non-standard data in their PDFs which may enable them to round-trip OpenType features.
BUT, simply displaying the embedded small cap glyphs when the PDF is opened in IL does not mean they are doing this, it may be they are just displaying the outlines that are there.

What happens if you insert a cursor and start typing lowercase text?
Does it display as small caps? (that would be evidence that the OT feature is still applied).
If you can copy small cap text from this exported-and-re-opened-in-IL PDF, and paste it into a text editor,
and it appears as lowercase - that would be evidence it is still lowercase.

Both the source text and the correct rendered result are preserved w/ full editing capabilities in Illustrator. The textbox contains lowercase, the smallcaps option is on, the text is rendered with uppercase glyphs. Changing the text (upper to lower or lower to upper) has the correct and expected result. Copying-and-pasting from Illustrator to a text editor presents the correct MixED cASe as expected. 

9 minutes ago, LibreTraining said:

Both of these would point to something non-standard going on.

Yes, though to be honest, I said as much when describing how the metadata was used to store the editability information. AD imports plenty of "non-standard" things from Illustrator-created PDFs to get them to render OK.

10 minutes ago, LibreTraining said:

Because as I mentioned above, the underlying code points embedded are uppercase.

We're in agreement on this, so long as you are saying the underlying code points for what is rendered in the PDF are uppercase. That's pretty much what I said above about PostScript, the AD bug regarding with which glyphs are subset, and what the editor source contains vs what the generated PDF has.

[walt.farrell]

15 minutes ago, mqudsi said:

Both the source text and the correct rendered result are preserved w/ full editing capabilities in Illustrator.

OK, but we know that Illustrator plays "games" like embedding its own proprietary data in EPS files that it exports. Do we know that it does not do that with PDF files that it exports?

If you have a sample PDF that works "properly" in Illustrator, and doesn't work in the Affinity applications, it might help if you provided it so we could examine it.

[mqudsi]

28 minutes ago, walt.farrell said:

If you have a sample PDF that works "properly" in Illustrator, and doesn't work in the Affinity applications, it might help if you provided it so we could examine it.

I've already attached it to the first post

 

Quote

OK, but we know that Illustrator plays "games" like embedding its own proprietary data in EPS files that it exports. Do we know that it does not do that with PDF files that it exports?

I'm assuming it does do just that.

[walt.farrell]

15 minutes ago, mqudsi said:

I've already attached it to the first post

 

I'm assuming it does do just that.

Sorry; missed that you had provided it. I'll take a look (though I'm not an expert).

[kenmcd]

3 hours ago, mqudsi said:

We're in agreement on this, so long as you are saying the underlying code points for what is rendered in the PDF are uppercase.

Yes. That is exactly what we are seeing in the PDF.
So that is what is imported by AD and it displays the correct characters.

 

2 hours ago, mqudsi said:

I'm assuming it does do just that.

Based on what you have said above, I would say that is confirmed.

I have rummaged around inside some of the Adopy PDFs before to try and see what is going on.
Your PDF has a section marked Illustrator > Private - most of which is encrypted.
(large enough to hold all of the content)
At the bottom of that section are two settings: RoundtripStreamType, and RoundtripVersion.
Did not find those terms in any of the PDF "standards" docs.

Just to be clear, that confirms for me that Adopy is storing another version of the content in an encrypted proprietary format which enables them to round-trip content while maintaining all the extended formatting, etc.
And since exporting the small caps as uppercase makes no sense at all,
I would assume that is simply sabotage for other apps.

[walt.farrell]

Thanks for checking that, @LibreTraining. That would seem to confirm that Affinity Designer does not have a bug, but just works with the publicly available data in the PDF file.

Still, I'm puzzled that non-Adobe PDF viewers display mixed small caps from the PDF in the first post, but Designer displays all one size caps.

[kenmcd]

2 hours ago, walt.farrell said:

Still, I'm puzzled that non-Adobe PDF viewers display mixed small caps from the PDF in the first post, but Designer displays all one size caps.

The PDF viewers are just displaying the actual outlines that are there in the PDF. So they are displaying a bunch of vector drawings. Designer is importing the text as editable text so it is looking at the actual underlying code points and displaying the outlines present in the font for those uppercase code points.

When you are editing text in ADesigner and you apply small caps, the underlying code points are still lowercase. It just displays the small cap outlines. And that is what ends up in the PDF - small cap outlines and lowercase code points.

So Adopy is saving lowercase code points for their small cap round-trip, and giving everyone else uppercase code points. Sleazy. 

[Old Bruce]

18 hours ago, LibreTraining said:

I would assume that is simply sabotage for other apps.

AKA Standard operating procedure.