Justification Not Working

[DaveLawrence]

Hi everyone, enjoying working in Affinity Publisher. Getting some great results for our Font Specimen documents!

--------------------------------------

The problem: When there is an alternate space character, affinity does not recognize it.

The sample font has this code:

 

Affinity does not recognize "space.alt" as a "space" when this ss05 is turned on.

An entire paragraph flow together like it is one word:

 

 

Steps:

1. Install the attached font.

AffinitySpaceTest-Regular.otf

2. Open the attached document.Affinity Space Test Document.afpub

3. In the typography panel, make sure that "Stylistic Set 05" is on.

4. Justification doesn't work.

[Guest]

Dave,

I’ve tested various approaches to implement different spaces for different scripts in the same font (GSUB-based as well as GPOS-based ones), but I couldn’t find a solution that would have worked across the range of standard design applications and web browsers. This is very unfortunate, from a font developers perspective, but I fear this it is something we will have to live with. Just saying this to emphasize that the Affinity suite is not the only app that doesn’t work as one might hope in this respect. 🙂

[stokerg]

Hi @DaveLawrence,

Thanks for reporting this.

I've just logged this with the Developers to resolve  

[DaveLawrence]

Hi @A_B_C, Thanks for the reply.

Hmmm... Interesting

There's not interpolation of GPOS features other than kerning, at least in the font software that I know.

Parenthetically, in 2018, I had talked about Adam Twardoch about interpolation of GPOS, especially interpolation of contextual positioning. He said that there is no native solution in font specs, and so it would require FontLab to do the interpolation through a proprietary means. It seemed like I was the only one who had asked about this, so probably not much demand.

So yeah since, in this example, spaces need to be different for different masters, GPOS won't work here. (Although it would seem to make the most sense.)

Have you gathered your results about GSUB and GPOS-based spaces for different scripts and softwares?

Here is a sheet I made at the end of 2022 for app support of features. (and I have a more complete spreadsheet). This PDF is after ilovetypography's old feature support table.Features Support In Publishing Apps.pdf

[DaveLawrence]

Hi @stokerg, 

Thanks for logging!

And just to show the scope of this error, besides the regular "space", there is also many other "breaking" or "non-joining" spaces. (Meaning that they should not keep words together.)

Find them here at the beginning of the unicode page for general punctuation.

In red, I have labeled here the spaces whose alternates should break—these spaces don't keep words together.

In green I labeled the alternates that probably should join glyphs together. These should not become separated by justification. These are also called non-breaking.

In blue, not sure. Also I don't know what the LRM and RLM marks are used for. Does anyone know the use of those glyphs?

Also, it seems that the figure space should not break, since it would keep the numbers together, when a language uses a space as a number delimiter.

And just as a reminder, we are talking about ALTERNATES of these spaces. Such as space.alt, enquad.alt etc. But also any other suffix or multiple suffixes such as space.case, space.sc, figurespace.onum, space.smcp.ss03 etc.

Besides these, please also consider the alternates for these spaces:

• U+20A0, no-break space (Joining)
• U+202F, the narrow no-break space (Joining)
• U+205F Medium Mathematical Space (non-joining?)
• U+2061–U+2064 (not sure about these)

-------------------------------------------------------------------

Also....If the developers want....I can make a font that has all these spaces, plus alternates. And I can make a Publisher test file.

[Guest]

Hi Dave,

it will be interesting to see how this is sorted out in the Affinity apps. Some background information can be gathered from this thread on the FontLab forum. Adam Twardoch and John Hudson suggested different ways to put alternative spaces into the font or adjust the width of the space glyph depending on the selected script and language parameters, but as I said, especially in web browsers, none of these methods worked really well. I can see whether I can find my test fonts in the archive to give you some more information.

Regarding interpolation, you are right: one has to distinguish between the capabilities of font editors and the OpenType specification. When you have a variable font setup in a font editor like FontLab or Glyphs, any glyph substitution or positioning rule that you define in the FEA syntax will apply to all masters in your font. Hence, you cannot do any useful positioning using the FEA syntax in a variable font setup. On the other hand, the OpenType specification clearly provides the means for varying positioning data along design axes. For GPOS data the variation deltas are stored in an Item Variation Store table in the GDEF table (see here and here). The problem is that there is no easy way to set up these variation deltas, given the current way in which font editors operate. So what is technically possible, according to the specification, is not really feasible at the moment, from the font developers perspective. (Hence, there are probably not many, if any, variable fonts out there that use GPOS other than for kerning and mark attachment.)

[Oufti]

6 hours ago, DaveLawrence said:

[…] besides the regular "space", there is also many other "breaking" or "non-joining" spaces. (Meaning that they should not keep words together.)

[…]

In blue, not sure.

Quote

200B
ZERO WIDTH SPACE
• commonlv abbreviate e ZWsP
• this character is intended for invisible word separation and for line break control; it has no width, but its presence between two characters does not prevent increased letter Spacing in justification

Not sure if this is supposed to keep letters together on a line

No, it is not. 

[Oufti]

6 hours ago, DaveLawrence said:

I don't know what the LRM and RLM marks are used for. Does anyone know the use of those glyphs?

These are used in bidirectional text, when combining in a line left-to-right and right-to-left languages. 

https://en.wikipedia.org/wiki/Right-to-left_mark

It seems not to be useable in AffPub.
(Even when inserted with U+code and ＾U, I don't see any effect — but this app is not supposed to be used with Right-to-left languages.)

[Guest]

To expand on what @Oufti said, the zero-width space (U+200B) can be used in scripts like Thai where words are not visually separated within a phrase. You can insert a zero-width space there to fine-tune word-boundary detection.

[kenmcd]

On 8/19/2023 at 10:43 AM, DaveLawrence said:

Affinity does not recognize "space.alt" as a "space" when this ss05 is turned on.

The solution is for space.alt to inherit the space's Line Breaking Class characteristics.
In your example doc the space.alt is treated like any other basic text character.
You would have the same result if you replaced space with the S character.
The test page is effectively one long single word.
There are no longer any line break opportunities, or the ability to vary the width of the spaces, or for the character width to disappear at the end of a line where there is a break (as the space would do).

The original source for most of this is:
Unicode Standard Annex #14, Unicode Line Breaking Algorithm
https://www.unicode.org/reports/tr14/

The Space is in its own SP class (Space) - for its unique combo of characteristics.
Most regular text would be in the AL class (Alphabetic)
The GL class (Glue) - for sticking things together - includes No-Break Space, Narrow No-Break Space, Word Joiner, Non-Breaking Hyphen, and Figure Space (to prevent numbers from breaking), etc.
Medium Mathematical Space is in the BR class (Break After) - yes it is "non-joining."
The Zero Width Space is also in its own class - ZW.
It simply adds back the ability to line break and adjust width where it would not normally be possible.

If you added a zero width space after each of your space.alt characters, the line breaking would work again, but the width adjustment of the space.alt would still not work, and the space.alt width would not disappear at the end of the line.

As mentioned above, the solution is for the substituted character to inherit the Line Breaking Class characteristics of the character it is replacing. This would affect other characters which may also sometimes have some special handling (like narrow no-break space, etc.).

 

-----
Side note re: OpenType Support in Desktop Publishing Apps
LibreOffice does support fina, init, and medi.

[Guest]

I think this is basically the correct explanation, although there are some details I would rephrase somewhat differently. The problem is that we are talking about two different levels in text processing here. Being assigned to a Line Breaking Class is a property of a Unicode character, while the kind of substitutions we are talking about in this thread is a substitution that happens on the glyph level. A character is an abstract entity (a location in an encoding space), while a glyph is, basically, a set of drawing rules that, when applied, result in a visual object, such as a letter or symbol.

In order to turn a sequence of Unicode characters into something visible, the codepoints belonging to this sequence must be mapped to the corresponding glyphs of the font that is selected for text rendering, and these glyphs must be positioned correctly in relation to each other. This mapping and positioning process is performed by a text shaping engine. If the selected font is an OpenType font, it may also include OpenType Layout (GSUB, GPOS) lookups that must be applied by the text shaping engine, either by default (e.g. depending on script and language settings for the Unicode input sequence) or based on user choices. Such a user choice would be enabling the GSUB feature ss05 in Dave’s font that replaces the /space/ glyph by the /space.alt/ glyph.

Now, a text shaping engine is typically not concerned with problems of line-breaking, hyphenation or justification. The output of a text shaper is usually a data stream that can be visualised as a glyph sequence laid out on a single, indefinitely long line. To find out where potential word, sentence or line break points are in this line, you will have to apply another process that iterates over the output of the text shaper and remembers the shaping process, starting from the Unicode input sequence. In Dave’s example, this process will have to retrace the character-to-glyph mapping from the codepoint U+0020 to the glyph /space/ (GID 3), and then the glyph-to-glyph substitution from /space/ (GID 3) to /space.alt/ (GID 1011). And by retracing, the process might be able to identify the position in the glyph sequence where an instance of the /space.alt/ glyph occurs as a potential break point for line breaking.

So since we are not substituting characters by characters (that is, codepoints by codepoints) when applying an OpenType Layout rule (the /space.alt/ glyph is not encoded anyway), but glyphs by glyphs, there is a slight conceptual problem in saying that “the solution is for the substituted character to inherit the Line Breaking Class characteristics of the character it is replacing”, but apart from that, I think you are right.

[kenmcd]

In Dave's demo font the space.alt is the same width as the space it is replacing.
The glyph substitution is not the cause of the lack of justification, or line breaks.
While a different glyph width will have an effect on the final result of justification and line breaks (when it is working properly), that is not the cause of the broken text here.

The Line Breaking Algorithm is based on Character Classes which are based on Unicode code points - and the assigned characteristics of those particular code points. Unicode Characters. Code-points. Not glyphs. You can swap the glyphs all day long and it will not break justification and line breaking if the character classes stay the same. What glyph is there does not matter.

Oddly the space.alt characters in the text do have the original code point behind them (U+0020), like normal OpenType substitutions should, but they are not acting like a space/U+0020 character anymore.
They are not being treated as Line Breaking Character Class SP (Space).
That is the problem.

[Guest]

Is there a point in my reply where I spoke about glyph widths? Or where I said that the glyph substitution would be responsible for the problem we see? Neither the first nor the second. 🙂

Rather, the only detail I meant to address was the following:

On 8/23/2023 at 9:39 PM, kenmcd said:

the solution is for the substituted character to inherit the Line Breaking Class characteristics of the character it is replacing

Since you are aware of the distinction between characters (codepoints) and glyphs, you will understand that this way of speaking can be misleading. In OpenType Layout, we do not substitute characters for characters, but glyphs for glyphs. Hence, it does not make much sense to speak of “substituted characters” and properties such “characters” would “inherit.” This is likely to lead to a confusion between characters and glyphs and to paint a wrong picture of OpenType Layout processing.

But as I said, the way of expression in the quote above does not invalidate your diagnosis (and there’s certainly no need to put every word under close scrutiny, if the underlying distinctions are clear – so no offense). As you noted, the application seems to lose track of which glyph corresponds to which character (with its particular line breaking property) when glyph substitution occurs. It does not retrace the text shaping process correctly before entering the stage where the output of the text shaper is distributed to paragraph lines.