Affinity Publisher: Search match option for apostrophes (idem spaces and idem quotes)

[Pyanepsion]

Hello everyone,

When you type an ASCII apostrophe U+0027 (') in a text box in Affinity Publisher, it automatically changes to a typographical apostrophe U+2019 (’), which is often a good thing.

However, this is not the case in the search input cell. For example, if you have written a text about Alice’s Restaurant and you search for occurrences by typing the sequence Alice's Restaurant in the search bar, you will not find it.

Just as upper and lower case letters are not differentiated by the search, the 3 kinds of apostrophes should not be differentiated, unless a specific option is checked in the search, and the same for the 18 kinds of spaces, and the 14 kinds of quotes.

[MikeTO]

38 minutes ago, Pyanepsion said:

When you type an ASCII apostrophe U+0027 (') in a text box in Affinity Publisher, it automatically changes to a typographical apostrophe U+2019 (’), which is often a good thing.

However, this is not the case in the search input cell. For example, if you have written a text about Alice’s Restaurant and you search for occurrences by typing the sequence Alice's Restaurant in the search bar, you will not find it.

Just as upper and lower case letters are not differentiated by the search, ASCII and typographical apostrophes should not be differentiated unless a specific option is checked in the search.

Agreed, this is an annoyance for me, too. I wish there was an option like Match Case for apostrophes.

I put in a feature request for this a year ago but I can't find it right now but there are several threads on this topic.

[loukash]

On 1/13/2023 at 7:52 PM, Pyanepsion said:

When you type an ASCII apostrophe U+0027 (') in a text box in Affinity Publisher, it automatically changes to a typographical apostrophe U+2019 (’), which is often a good thing.

That's because you have enabled Preferences > AutoCorrect > Change straight quotes…
(Disclosure: I've never used autocorrect. In any application. Ever. I want to be in control.)

On 1/13/2023 at 7:52 PM, Pyanepsion said:

However, this is not the case in the search input cell. For example, if you have written a text about Alice’s Restaurant and you search for occurrences by typing the sequence Alice's Restaurant in the search bar, you will not find it.

Hopefully not!
I definitely wouldn't want any false positives.  

[Pyanepsion]

@loukash,

I think you misunderstood the description.

It’s a good thing that Affinity Publisher automatically (according to our settings) replaces the straight apostrophe with a typographical apostrophe, as this is the standard in most countries.

It is a major malfunction, however, if Affinity Publisher does not show hits with a typographical quote when a straight quote is typed into the search box.

10 hours ago, loukash said:

I definitely wouldn't want any false positives. 

What you are asking for should be an option in the search menu as for the differentiation of upper and lower case input. This would make it possible to find both occurrences of this short example. Here, only the case can be undifferentiated in the search, not the type of apostrophe or quotation mark.

[loukash]

3 hours ago, Pyanepsion said:

It is a major malfunction, however, if Affinity Publisher does not show hits with a typographical quote when a straight quote is typed into the search box.

No, it's not.
It's a feature that the standard search finds exactly the Unicode character you're looking for.
As I said, I do not want any false positives. This is how search is supposed to work. 

Also keep in mind that APu's AutoCorrect doesn't "hide" the character you're typing, it replaces it. What you have originally typed is gone. (Don't know if it's being recorded by History though.)

3 hours ago, Pyanepsion said:

Here, only the case can be undifferentiated in the search, not the type of apostrophe or quotation mark.

If you want to find ' and ’ , then use regex. That's why this option is there:

In my example, searching for (d)['’]oh will find both apostrophes, and replacing it with $1’oh will keep the upper/lowercase D or d as is.
Yes, it requires learning regex. I'm a (eternal) regex noob, too. But the effort trying to master regex is worth it every time.
The best regex overview that I'm looking up everytime is in my opinion regular-expressions.info/quickstart.html

[walt.farrell]

In the meantime, you can do a search with the Regular Expression option enabled, that will find both.

You can use [’'] to find either. But yes, it would be nice to have some option. I think Firefox will find both automatically, but I'm not sure which (if any) text editors do it automatically.

[Pyanepsion]

Regex is not at all suitable when it comes to finding all occurrences of a large number of denominations that can each be written in a large number of ways.

To continue the previous example of the denomination Alice’s Restaurant (a title of a film of Arthur Pen):

alice's restaurant, Alice's restaurant, alice's Restaurant, Alice’s Restaurant

alice’s restaurant, Alice’s restaurant, alice’s Restaurant, Alice’s Restaurant

aliceʼs restaurant, Aliceʼs restaurant, aliceʼs Restaurant, Aliceʼs Restaurant

It is obviously much more time-consuming to have to write a new Regex for each title searched than to use four clicks in the search options, once and for all, to satisfy everyone:

1. Match Case (the only choice currently available)
2. Match Space
3. Match Quote
4. Match Apostrophe

As much as it would be abnormal if the software forced us to confuse upper and lower case, straight apostrophe and the 2 others, straight quote and the 14 others, normal space and the 16 others, it would be appalling if it did not allow us differentiate between them.

@walt.farrellHere are some possibilities with Microsoft Word.

By default.

 

Match case

 

[MikeTO]

I think we can all agree that it's good that you can search precisely for a straight or typographic quote. But it would be convenient to also have the ability to search for either, without having to jump through hoops. An option, similar to Match Case, would be desirable.

MS Word searches for both by default since this is best for business users who don't even know the difference between types of quotation marks. For those who want to find just straight quotes they would enter ascii codes such as ^39 (single) or ^34 (double). This works but isn't discoverable - it would be nice to have more options in Publisher's Find and Replace along the lines that Pyanepsion suggested:

• Match Any Quotation Mark - match both straight or typographic single and double quotes, base quotes, and guillemots - I don't think we need separate options
• Match Any Space - match all types of spaces, non-breaking, em, en, etc.
• Match Any Hyphen - match all types of hyphens and dashes, non-breaking, soft, em, en, etc.

Cheers

[loukash]

3 hours ago, MikeTO said:

• Match Any Quotation Mark - match both straight or typographic single and double quotes, base quotes, and guillemots - I don't think we need separate options
• Match Any Space - match all types of spaces, non-breaking, em, en, etc.
• Match Any Hyphen - match all types of hyphens and dashes, non-breaking, soft, em, en, etc.

Alright, this proposal sounds reasonable even though I, for one, can easily live without it.

[loukash]

4 hours ago, Pyanepsion said:

alice's restaurant, Alice's restaurant, alice's Restaurant, Alice’s Restaurant
alice’s restaurant, Alice’s restaurant, alice’s Restaurant, Alice’s Restaurant
aliceʼs restaurant, Aliceʼs restaurant, aliceʼs Restaurant, Aliceʼs Restaurant

Find (Regular Expression on, Match Case off):

alice['’ʼ]s restaurant

Replace with (for example):

Alice’s Restaurant

If you want to keep the case as is:
Find: (a)lice['’ʼ]s (r)estaurant
Replace: $1lice’s $2estaurant

[walt.farrell]

3 hours ago, MikeTO said:

Match Any Quotation Mark - match both straight or typographic single and double quotes, base quotes, and guillemots

But don't forget high-9 and low-6 single quotes (and their double-quote equivalents if such exist; I don't remember). Also, guillemots can point either direction.

9 minutes ago, loukash said:

Match Any Space - match all types of spaces, non-breaking, em, en, etc.

That exists already as native RegEx handling. We do not need an option for that, in my opinion.

Unless you're proposing that this be separate from RegEx handling, but that would make it much more complex. And the big problem with doing this with the RegEx processing is that all these proposals require changes to the RegEx engine, I think.

[walt.farrell]

1 minute ago, loukash said:

If you want to keep the case as is:
Find: (a)lice['’ʼ]s (r)estaurant
Replace: $1lice’s $2estaurant

Or:

Find: (alice)['’ʼ](s restaurant)
Replace: $1'$2 

[loukash]

3 minutes ago, walt.farrell said:

15 minutes ago, loukash said:

@MikeTO said that, actually…  

[loukash]

Just now, walt.farrell said:

Or

Of course, there are too many variants possible…  

[walt.farrell]

6 minutes ago, loukash said:

@MikeTO siad that, actually…  

I quoted it directly from your post, where you said it unquoted  

[loukash]

I meant this:

30 minutes ago, walt.farrell said:

  44 minutes ago, loukash said:

Match Any Space - match all types of spaces, non-breaking, em, en, etc.

[walt.farrell]

55 minutes ago, loukash said:

I meant this:

Sorry. Quoted from the wrong post for that. I'll try to be more careful.

[Pyanepsion]

Quote

But don't forget high-9 and low-6 single quotes (and their double-quote equivalents if such exist; I don't remember). Also, guillemots can point either direction.

Maybe you didn't see it. I was pointing out that there are a number of each type of character. Here is the complete list of Unicode characters with the name quotation apostrophe and spaces, but not all of them are actually considered as such.

Apostrophe (8)

U+0027 = APOSTROPHE
U+0149 = LATIN SMALL LETTER N PRECEDED BY APOSTROPHE
U+02BC = MODIFIER LETTER APOSTROPHE
U+02EE = MODIFIER LETTER DOUBLE APOSTROPHE
U+055A = ARMENIAN APOSTROPHE
U+07F4 = NKO HIGH TONE APOSTROPHE
U+07F5 = NKO LOW TONE APOSTROPHE
U+FF07 = FULLWIDTH APOSTROPHE

Quotation (26)

U+0022 = QUOTATION MARK
U+00AB = LEFT-POINTING DOUBLE ANGLE QUOTATION MARK
U+00BB = RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK
U+2018 = LEFT SINGLE QUOTATION MARK
U+2019 = RIGHT SINGLE QUOTATION MARK
U+201A = SINGLE LOW-9 QUOTATION MARK
U+201B = SINGLE HIGH-REVERSED-9 QUOTATION MARK
U+201C = LEFT DOUBLE QUOTATION MARK
U+201D = RIGHT DOUBLE QUOTATION MARK
U+201E = DOUBLE LOW-9 QUOTATION MARK
U+201F = DOUBLE HIGH-REVERSED-9 QUOTATION MARK
U+2039 = SINGLE LEFT-POINTING ANGLE QUOTATION MARK
U+203A = SINGLE RIGHT-POINTING ANGLE QUOTATION MARK
U+275B = HEAVY SINGLE TURNED COMMA QUOTATION MARK ORNAMENT
U+275C = HEAVY SINGLE COMMA QUOTATION MARK ORNAMENT
U+275D = HEAVY DOUBLE TURNED COMMA QUOTATION MARK ORNAMENT
U+275E = HEAVY DOUBLE COMMA QUOTATION MARK ORNAMENT
U+275F = HEAVY LOW SINGLE COMMA QUOTATION MARK ORNAMENT
U+2760 = HEAVY LOW DOUBLE COMMA QUOTATION MARK ORNAMENT
U+276E = HEAVY LEFT-POINTING ANGLE QUOTATION MARK ORNAMENT
U+276F = HEAVY RIGHT-POINTING ANGLE QUOTATION MARK ORNAMENT
U+2E+42 = DOUBLE LOW-REVERSED-9 QUOTATION MARK
U+301D = REVERSED DOUBLE PRIME QUOTATION MARK
U+301E = DOUBLE PRIME QUOTATION MARK
U+301F = LOW DOUBLE PRIME QUOTATION MARK
U+FF02 = FULLWIDTH QUOTATION MARK

Space (21)

U+0020 = SPACE
U+00A0 = NO-BREAK SPACE
U+1361 = ETHIOPIC WORDSPACE
U+1680 = OGHAM SPACE MARK
U+2002 = EN SPACE
U+2003 = EM SPACE
U+2004 = THREE-PER-EM SPACE
U+2005 = FOUR-PER-EM SPACE
U+2006 = SIX-PER-EM SPACE
U+2007 = FIGURE SPACE
U+2008 = PUNCTUATION SPACE
U+2009 = THIN SPACE
U+200A = HAIR SPACE
U+200B = ZERO WIDTH SPACE
U+202F = NARROW NO-BREAK SPACE
U+205F = MEDIUM MATHEMATICAL SPACE
U+2408 = SYMBOL FOR BACKSPACE
U+2420 = SYMBOL FOR SPACE
U+3000 = IDEOGRAPHIC SPACE
U+303F = IDEOGRAPHIC HALF FILL SPACE
U+FEFF = ZERO WIDTH NO-BREAK SPACE

Quote

[space] That exists already as native RegEx handling. We do not need an option for that, in my opinion.

There are 18 kinds of real spaces, but not all of them are found without a complicated regex, hence the proposal of the checkbox feature.

Quote

Replace by

Searching does not necessarily mean replacing!

[walt.farrell]

3 hours ago, Pyanepsion said:

There are 18 kinds of real spaces, but not all of them are found without a complicated regex, hence the proposal of the checkbox feature.

All should be found by the whitespace search terms in RegEx processing. Which ones aren't?

3 hours ago, Pyanepsion said:

Searching does not necessarily mean replacing!

No, of course not. You would only Replace when you want to do that.

[Pyanepsion]

1 hour ago, walt.farrell said:

All should be found by the whitespace search terms in RegEx processing.

Just use the information I have provided! 😂

In short! Some spaces are not detected (Red) by the usual sequence \s, others are detected (Green), but are not spaces as such, but something else.

Example:
- The character U+1361 ETHIOPIAN SPACE belongs to the general category Po (Punctuation, other). It is a word separator. It is not considered a space.
- The character U+1680 OGHAM SPACE MARK is a visible line but without character.
- The character U+303F IDEMOGRAPHIC HALF SPACE belongs to the general category So (Symbol, other). It visibly indicates a display cell 'padding' used when ideographic characters in a double-byte character set have been rendered as separate characters. It is not considered a space.
etc.

[walt.farrell]

7 minutes ago, Pyanepsion said:

In short! Some spaces are not detected by the usual sequence, others are detected, but are not spaces as such, but something else.

Example:
- The character U+1361 ETHIOPIAN SPACE belongs to the general category Po (Punctuation, other). It is a word separator. It is not considered a space.
- The character U+1680 OGHAM SPACE MARK is visible character.
- The character U+303F IDEMOGRAPHIC HALF SPACE belongs to the general category So (Symbol, other). It visibly indicates a display cell 'padding' used when ideographic characters in a double-byte character set have been rendered as separate characters. It is not considered a space.
etc.

So, are these found by RegEx "whitespace" processing, or not?

Did you want them to be found, even when they are "considered" punctuation, not spaces?

Sorry, I've lost some of the thread of what you want.

[Pyanepsion]

It would be consistent for all 17 Unicode characters referenced as space in the general category Zs (a space character of various non-zero widths) to be detected without going through a regex, with reservations about Ogham Mark Space: this character may be considered invisible even though it is visible (invisible: It is the absence of a character on the writing line of the ogham, visible: we see the line). I would opt for yes.

As for the 43 characters in the Cf category (a format control character), it can be discussed for the 2 (but only 2 from general category Cf) that are zero width spaces. I would opt for yes, as these two null spaces sometimes cause layout problems.

[loukash]

3 hours ago, Pyanepsion said:

some spaces are not detected (Red) by the usual sequence \s,

Zero Width Space is under Special Characters. Use a pipe to find it as well:

The one special character missing – apart from the "esoteric" ones like Ethiopian or CJK – is Zero Width Non-Joiner.
Is it what you have listed as "Zero Width No-Break Space"?
It can be found nonetheless if you copy it as text and paste it into the Find field, again using a pipe if you use regex.

[Pyanepsion]

Quote

Zero Width Space is under Special Characters. Use a pipe to find it as well:

It is:

U+200C = ZERO WIDTH NON-JOIN

The ZERO WIDHT NON-JOIN (en français ANTILIANT SANS CHASSE) is not a space but a punctuation. It belongs to the general category Cz (a format control character).
It is part of the General Punctuation subset (U+2000 to U+206F).