[Publisher 2.5.7][BUG][Searching for "s" and "ſ" (long s)]

[Franz Rogar]

BASIC INFO

• App & version: Publisher 2.5.7
• OS & version: Windows 11 (up-to-date)
• Graphic card: NVIDIA GeForce GTX 1070 (up-to-date)
• Hardware acceleration: On & Off
• Reproducibility: Always
• Affects new document?: Yes

WHAT HAS HAPPENED
There are three possible "s" to type in: normal "s" (#A), the "ſ" constructed as normal "s" + OpenType "historical form" on (#B), and Unicode long-s "ſ" U+017F, if available in the font (#C).

• When searching for #A, it shows #A and #B.
• When searching for #B, it shows #B.
• When searching for #C, it shows #C. 

WHAT SHOULD HAVE HAPPENED
Or (#A -> #A; #B -> #B; #C -> #C), typesetting approach, or (#A -> #ABC, #B and #C -> #BC), language approach, which IMHO should be preferred.

MINIMAL STEPS TO REPRODUCE

1. Open Publisher
2. Create a new document
3. Create a text frame
4. Typeset the three different "s" as previously noted.
5. Search for each kind.

BUG
Current Search results are incongruous. This might affect other characters.

WORKAROUND

It has been detected that a faulty setting triggers this bug (maybe there are other triggers). So, by restoring the default settings, this bug might stop; but before clearing, it'd be great that you post yours settings in this thread so the developers can track exactly the trigger.

• Back up "faulty" settings (thread with information for Windows)
• Cleaning settings: go to "Settings > Miscellaneous" and clear them all. Restore the app and the bug should stop.

Edited January 11 by Franz Rogar
Added workaround information

[walt.farrell]

Personally, I would not expect a search for "s" to also find "ſ" when the text uses the Unicode U+017F. I can certainly see wanting to find all "s"-like characters, and that sounds like a nice improvement, but I would not call the lack of that function a bug.

I am curious how you did the search for your #B case, but from what I understand I think that the results you have showed for searching for #A and #B are also working as I would expect them to work. If I ask to search for an "s" with a particular formatting characteristic such as "historical form" I would be upset if it also found an "s" that did not have it.

[MikeW]

Perhaps the results have something to do with how the font is constructed?

If the long s is manually inserted, likely searching for a short s will never find the long s.

Here I'm searching a document for a short s. As you see, it finds every short and long s.

[Franz Rogar]

25 minutes ago, MikeW said:

Here I'm searching a document for a short s. As you see, it finds every short and long s.

And that "result" is wrong, because the highlighted "s" is not "s" is "long s", which is a completely different character and sound, and thus, it should not be shown.

It's as if you search for "th" and it includes in the results "þ" (thorn). They might have represents the same sound lately but not originally, making them two different characters (and Unicode registered them separately because of that).

[Franz Rogar]

1 hour ago, walt.farrell said:

Personally, I would not expect a search for "s" to also find "ſ" when the text uses the Unicode U+017F.

"ſ" should be treated exactly the same regardless of having being created via "s"+"historical ligature" or "direct input", mainly because the OpenType feature is meant to be a "hotkey" (or fast-rendering) of a language feature, not as a separate reality.

1 hour ago, walt.farrell said:

I can certainly see wanting to find all "s"-like characters, and that sounds like a nice improvement, but I would not call the lack of that function a bug.

I marked this as a "bug" because it is incongruous with what I marked as "expected" result because they represent two different characters: "s" (s) and "ſ" (long s) are *not* the same character at all, even though it's registered in OpenType feature as "historical form", they are not the same and are not interchangeable as they represent different consonants, not different symbols of the "same" consonant.

1 hour ago, walt.farrell said:

I am curious how you did the search for your #B case, but from what I understand I think that the results you have showed for searching for #A and #B are also working as I would expect them to work. If I ask to search for an "s" with a particular formatting characteristic such as "historical form" I would be upset if it also found an "s" that did not have it.

Here is where you reasoning fails: if you "ask to search for s with ... historical form" you *must* get the Unicode direct input, because s+"historial form" is an "hotkey" for the Unicode entry, not a different entity at all.

[MikeW]

Technically, it is a correct search result. In the above instance, because the long s is an OpenType substitution, the underlying character is a short s.

I believe that is why you are also getting #B when searching for #A.

I get all 3 results when searching for a short s.

[Franz Rogar]

2 hours ago, MikeW said:

Technically, it is a correct search result. In the above instance, because the long s is an OpenType substitution, the underlying character is a short s.

I believe that is why you are also getting #B when searching for #A.

I get all 3 results when searching for a short s.

Did you got the 3 results? Then, that'd be the "typesetting approach" I wrote in the expected results, which I (sadly) didn't get, hence why I posted this bug. What I get when searching "short s" are only "short s" (#A)and "short s"+"historical ligature" (#B).

After your reply, I thought maybe was a problem with the font I'm using in my doc (Adobe Garamond Premier Pro), but it also happens with Junicode. The "Unicode long s" (#C) does *not* shows up when searching for "short s".

[MikeW]

Yes, all 3 are displayed when searching for #A (short s).

But the reason why is because the font I am using uses OpenType substitution for the long s.

If I manually insert a long s, searching for short s will not ever find it. And APub shouldn't. 

[Franz Rogar]

3 minutes ago, MikeW said:

Yes, all 3 are displayed when searching for #A (short s).

But the reason why is because the font I am using uses OpenType substitution for the long s.

If I manually insert a long s, searching for short s will not ever find it. And APub shouldn't. 

No, I did use the three of them "short s", "short s"+OpenType + Unicode on the same frame, but only 2 of them are shown when searching for "short s".

[MikeW]

Hmm. Using the same font, standard short s, short s with the hist feature applied, and the long s inserted via the glyph browser...

Frankly, I'm amazed it finds the long s when inserted via the glyph browser.

I'm using Windows 10, APub release 2.5.7

[kenmcd]

Known bug.
Previous discussion marked as: #APL-1727
From here: https://forum.affinity.serif.com/index.php?/topic/205822-afpub2-v252-can-unicode-characters-be-learned-mapped-for-the-spell-checker/

That discussion was also about long s.
Further down that page I posted tests which included Garamond Premier Pro and Junicode.
https://forum.affinity.serif.com/index.php?/topic/205822-afpub2-v252-can-unicode-characters-be-learned-mapped-for-the-spell-checker/#findComment-1233714

The bug is that Affinity is actually replacing the character - changing the Unicode code point - rather than doing the correct OpenType glyph substitution.

Affinity is doing the something similar with legacy ligatures.
https://forum.affinity.serif.com/index.php?/topic/203218-unwanted-ligature-feature-added-to-fonts-not-good/
(in that case a monospace font with no OpenType ligatures, Affinity is replacing the two "fi" characters with the one legacy ligature character)

This causes problems with search, screen readers, and copy-and-paste.

Peter, the Junicode developer, talks about the issues and how Junicode works here (and in the PDF manual):
Searchable web pages with MUFI and Junicode 2

The fonts are not the problem.
The problem is Affinity.

You cannot use Garamond Premier Pro or Junicode to create searchable historical documents - because Affinity will break the text.

You cannot use a monospace font to create sample programming code documents - because Affinity will break the code.

@Franz Rogar Unfortunately the only work-around is to use a different application.

 

[MikeW]

Except it works here using GPPro as seen in my last screen shot. 

If the search has the purpose of seeing the 3 characters in context, one could always use a regex search for s| (pipe character) plus a copy/pasted long s character immediately after the pipe character. 

However, I don't understand why it works for me and not the OP.

[Old Bruce]

1 minute ago, MikeW said:

... one could always use a regex search for s| (pipe character) plus a copy/pasted long s character immediately after the pipe character. 

<pedant On>

To clarify, the pipe character is used as an Or, as in logic.

<pedant Off>

[kenmcd]

12 minutes ago, MikeW said:

Except it works here using GPPro as seen in my last screen shot. 

If the search has the purpose of seeing the 3 characters in context, one could always use a regex search for s| (pipe character) plus a copy/pasted long s character immediately after the pipe character. 

However, I don't understand why it works for me and not the OP.

There seems to be a trigger to do the character replacements.
Export to PDF definitely triggers the replacements.

So not sure if saving, or closing the file or the app, or reopening the file or app, or something else is the trigger.

[Franz Rogar]

1 hour ago, kenmcd said:

There seems to be a trigger to do the character replacements.
Export to PDF definitely triggers the replacements.

So not sure if saving, or closing the file or the app, or reopening the file or app, or something else is the trigger.

But the test (and the bug report) are from scratch (new/empty docs) and suffer from the "not finding the GlyphBrowser/Unicode long s" in my PC, whereas for @MikeW it does work...

FOUND A TRIGGER!

After the confirmation that it does work, I was about to ask where the "settings" were stored (I thought it was only possible to clear them by "brute force", aka: erase files/registry entries), but I found that there were a series of "reset" at "Settings > Miscellaneous" (it'd be great if there were a "reset all" button though). So I clicked on them all, restarted the program... and it now DOES find the GlyphBrowser/Unicode long s!

So, it's a "setting" somewhere, somehow, where the bug is fed. I can't say which one as I cleared all of them, stupidly, instead of try one by one.

[Franz Rogar]

2 hours ago, kenmcd said:

You cannot use Garamond Premier Pro or Junicode to create searchable historical documents - because Affinity will break the text.

...

@Franz Rogar Unfortunately the only work-around is to use a different application.

 

Off-topic: I'm not creating a searchable document. I'm creating a 3-volume critical edition of 15 texts in parallel of a 17th century play. Because I worked on it directly with Affinity Publisher, I was learning the program at the same time, so I have parts that are "s+historical alts" and parts that are the Unicode glyph directly. So, it was a nightmare to search and check the text that might have been in one or other typesetting. And because those books are meant to be in printed form (otherwise it is impossible to work with them opened side-by-side at the same time in the digital world unless you have three monitors in front of you), it doesn't matter that they are not "searchable" because they are not meant to be