Factory Defaults: Dictionary "Unknown (en-DE)"

[thomaso]

The Factory Defaults set for the dictionary in a German macOS an entry "Unknown (en-DE)" and cause an according Preflight message. The issue is regardless of the app language, see this initial post. – The button "Revert Defaults" is not affected, it seems to be independently since setting this language to "None".

I assume that "None" should be the Factory Default, too. It should always cause the same as the button "Revert Defaults" as long this Defaults are not user-defined by using "Synchronise...".

Another little issue which might confuse users is the displayed language if a text frame is selected with the Move Tool. Then it seems to display always the language of the first character, as it does for text colour, too. If more than 1 frame is selected then the displayed language and color vary, they depend on the order of selection or the position on the page.
(It used to be like this also for text size and font face but fortunately got fixed in a previous update)

 

 

[garrettm30]

2 hours ago, thomaso said:

The Factory Defaults set for the dictionary in a German macOS an entry "Unknown (en-DE)" and cause an according Preflight message. The issue is regardless of the app language, see this initial post.

There have been various reports of these odd (non-existent) dialects of English for the past few versions, but if I remember correctly, the cause has not been identified. It may be fortuitous that you are now observing the issue too, because your proven dedication at getting to the bottom of these things may be just what is needed to finally identify the problem.

[thomaso]

1 minute ago, garrettm30 said:

There have been various reports of these odd (non-existent) dialects of English for the past few versions

Oh, sorry, in fact, with my passionate ignorance, I've never even noticed;) – Does it mean it is an issue not only with German but exists in a similar way for e.g. French, too, as kind of "Inconnue (en-FR)" ?

[garrettm30]

28 minutes ago, thomaso said:

Does it mean it is an issue not only with German but exists in a similar way for e.g. French, too, as kind of "Inconnue (en-FR)" ?

I have not personally experienced it, but there do seem to be other cases. Here is one such thread where en-BE (English as spoken in Belgium?) was encountered, and in a quote from another user en-ES (Spanish English?) was mentioned.

 

This thread references a missing "en-IE":

Or this one, "en-CR":

This one references "en-HU", which the user takes to be his/her own mistake, but it seems to be more likely the same issue:

 

I also seem to recall seeing something for the Singapore dialect of English on another thread, but I can't find it.

I have the vague idea that the user's system language or location is involved somehow. 

On the other hand, I did see reports of similar missing dictionary issues from languages that do exist. These are the few I noticed that I think are real: en-NZ, en-AU, and fr-CA.

[thomaso]

On 8/7/2020 at 4:52 PM, garrettm30 said:

I have not personally experienced it, (...)
I have the vague idea that the user's system language or location is involved somehow. 

Thank you for the list, the number of cases is indeed weird. But it's also interesting that You haven't experienced it yet. – Could you check this please:

– Your recent "wrong_frame.afpub", created in French OS + French Apub + in France?, opens to me, in German OS + app language, with Unknown (en-US).
– Your bottom filler text frame opens to me as set to French. Do you see the same in your file?

I wonder why...
– a.)  What initiates here English at all "(en-US)" and what in particular US English "(en-US)"?
– b)  Why is filler text set to French? (I assume you didn't apply it?)
– c)  Latin text doesn't create spelling marks in your file (it would for my frames if set to Deutsch)
– When I create a new frame in your document its initial language is "None" (as expected), if I place filler text it gets auto-set accordingly.
– Also I don't experience any oddity and no difference when applying "Revert Defaults" or "Factory Reset" within your document (different to mine).

This makes me currently feel the entire issue of "Unknown(xx–XX)" is not only OS-language or location or app language related – but also document related, which is weird, too, because there is no customisable language setting for documents.

[thomaso]

Meanwhile I noticed that both A.) the app without open documents and B.) documents without selected objects show in the Character Panel an entry for the spelling language. This seems to depend on the OS language + app language and also can vary between documents, though there is no language UI for entire Affinity documents but for selected items only.

This video shows this entry for 3 different documents with no object selected:
– older .afpub of mine (begun in ~March / previous app version) –> "Unknown (en-DE)"
– an .afpub of garrettm30 (french & france?) –> "Unknown (en-US)"
– a new document (app preset) –> "Deutsch"

Additionally it shows the varying entries for each of the 3 documents...
– when a new text frame gets created
– when Filler Text is created
– when "Revert Defaults" gets used
– when "Factory Defaults" gets selected

Obviously an .afpub has a language setting saved, regardless of any text content.
So there should be an UI to adjust/repair it in cases of "Unknown (xx-XX)".

App UI language: The next video shows the spelling language for a new document for two different app languages: English vs. Default (my macOS is German).
– without object
– with new text frame
– with filler text
– "Revert Defaults"
– Defaults "Factory Reset"
The language entries appear to be the same in both files except for "Defaults" > "Factory Reset", which causes "Unknown (en-DE)" when the app is English.
Note that even after a defaults factory reset the button "Revert Defaults" causes a different spelling setting, this implies that "Revert Defaults" does not really relate on "Factory Reset". I would expect these functions to cause the same result unless the "Synchronise Defaults" button became used.

[garrettm30]

On 8/8/2020 at 6:11 AM, thomaso said:

But it's also interesting that You haven't experienced it yet. – Could you check this please:

– Your recent "wrong_frame.afpub", created in French OS + French Apub + in France?, opens to me, in German OS + app language, with Unknown (en-US).
– Your bottom filler text frame opens to me as set to French. Do you see the same in your file?

I knew you'd be the right person to really dig in the details. I confess I'm starting to really get confused. But if I just focus on the part that concerns the file I posted in the other thread, I do think there may be a clue from that file when we compare what you see with what I see.

First, here is a brief summary of my language situation.

1. I live in the US, and my native language is the US dialect of English.
2. My macOS system spelling is configured to use US English and French:
   
3. My Publisher UI language setting (bottom of the General preferences window) is set to "default," which I take to mean it follows system default.
4. I do speak French, and most of my layout work involves French text.

With those things in mind, here are the specifics to the file we are talking about. When I download it and open it, I notice that the character language when nothing is selected is "English" (not "US English" or similar). When I select either of the frames, the language is "Français." That is not a surprise, because the sample frames were originally copied from a working document that originally had real French text rather than filler text.

So I would say that the French part is not a surprise, but what shows up simply as "English" on my system and as "Unknown (en-US)" on yours is the point to focus on.

I have observed before that the languages on offer in Publisher are directly tied to the settings in my screenshot above. Currently in my Publisher, the only options for language spelling in the Character Studio are "None," "English" (with no indication of variant) and "Français," which matches perfectly with what I have configured in the system. In the past, I also tested activating multiple dialects of English, and in such cases Publisher was more specific about the variants of English:

 

Notice that even there, my own dialect is still called simply "English" rather than "English (United States)."

Now here, I believe, is a breakthrough: if I disable the U.S. English and enable British English in the system settings, when I relaunch Publisher and open the document, now the language is "Unknown (en-US)" where it was simply "English" before.

[thomaso]

1 hour ago, garrettm30 said:

Now here, I believe, is a breakthrough: if I disable the U.S. English and enable British English in the system settings, when I relaunch Publisher and open the document, now the language is "Unknown (en-US)" where it was simply "English" before.

To me the odd thing of this change is less that it's occurring locally on the same computer but more when opened on a different computer, where the app "actually" can't know about the OS language of a former user. But this "actually" seems to become a wrong assumption, since a document language appears to get saved within an .afpub (besides its frames and text languages), – although there is no user interface in Affinity for a document language.

Furthermore / But: If a document language gets saved and is related to the current OS then I would expect that a "Save As..." would save a different language when the document got opened and saved on a different OS language. It did not for your file to me. That might mean the document language gets saved only once and with the first save action of a file (like the fields "Author" and "Title", whereas they have an UI).

Can you report for the attached .afpub what languages you get displayed when you
a.) open this document?
b.) select the text frame?

v184 spelling.afpub

EDIT Supplement: I noticed that your .afpub files have quite a small file size (~15 kB) whereas mine, even with same color settings and when downloaded from my post, show never less than 1.8 MB on my disk. – What file size does my .afpub show to you?

[garrettm30]

1 hour ago, thomaso said:

Can you report for the attached .afpub what languages you get displayed when you
a.) open this document?
b.) select the text frame?

Sure:

a) "unknown (de-DE)"

b) "None"

1 hour ago, thomaso said:

What file size does my .afpub show to you?

Your file's size after downloaded is 1.8 MB. That is very odd. I don't see any reason it should be so big with nothing more than a single text frame with not very much text in it.

Rather than a specific document language, do you think that each document saves all the current defaults? I notice that if I draw a new text box, it continues with the same settings (for example 18pt text, spelling: none) as the text in the frame you created.

[thomaso]

4 hours ago, garrettm30 said:

a) "unknown (de-DE)"

Aha. So a document saves a specific language assignment, which causes "unknown (xx-XX)" if the dictionary doesn't exist in the Affinity installation or the language isn't active in the OS, – regardless whether this language is used in the file at all. – I can't imagine when it may be useful to have an app or system language saved in an .afpub, especially with its first creation/save only. – There rather appears to be a bug in Affinity.

A bug in particular if applying "Factory Reset" causes this "unknown" issue, especially if "Revert Defaults" doesn't, as I experienced as shown in the videos above.

4 hours ago, garrettm30 said:

Your file's size after downloaded is 1.8 MB. (...) do you think that each document saves all the current defaults?

I don't think so. You could check if my document shows any 'foreign' items, e.g. assets, custom object styles, text styles or color palettes. I did not create any for/in this .afpub, so I expect there are none. But even if, they are hardly 1.8 MB (my currently installed assets are 300 kB when I export them).
Before I got your slim documents I had color profiles as culprits in mind – but obviously they aren't.

So it really interests me what is hidden in this file size?
And why does any new document have this 1.8 MB to me – whereas when I do Save As with your slim .afpubs they maintain their ~ 15 kB?

[Gabe]

Hi both,

Sorry for the delayed reply. 

The default spelling option is driven by the app language and os region. 

For non-English app language, the default spelling should match the app languag, regardless of the OS region. 

For English app language and any OS region, the default spelling should be "unknown en-{osRegion}". If one has a "dual language" setup (en-{language}), they need to install that dictionary. 

[thomaso]

@Gabe, I experience it different. My setup:

macOS: DE
region: DE
APub app language: EN (UK)

– With NO document opened the character spelling language appears to be "Deutsch"
– New document: the language remains "Deutsch"
– As soon I select a Text tool this setting jumps to "None" - I even didn't use the tool
– Adding filler text remains "None" - and causes "Latin" for both Hyph. & TypoScript, o.k.
– Clicking "Revert Defaults" remains "None" and resets "Latin" to "auto"
– Choosing "Factory Reset" reverts "None" to "Unknown (en–DE)"

According to your description "Deutsch" wouldn't appear by itself with a new document, – right?

3 hours ago, Gabe said:

For English app language and any OS region, the default spelling should be "unknown en-{osRegion}".

For what purpose does any "Unknown" get set as default, – instead "None" for instance?
While "None" doesn't cause a preflight entry, "Unknown (xx-XX)" obviously does.
Furthermore, since an .afpub appears to have an initial creation language saved (see previous posts, like the "Title" field, but without UI) a default "None" would be more flexible (independent) than the current "Unknown (xx-XX)" default.

EDIT: The recent note of Loquos "I should add, the Preference Panel Auto Correct setting seems to be linked to the working document."  made me try again. It gets bit more confusing. I run a similar routine once more but with closing the app first, this resulted in a different language with NO document opened, it displayed "None", not "Deutsch" as in the video above. – Is there a redraw or update issue with a correct display of the current spelling language?

[Gabe]

I can't replicate this here  

This is what I see on my end:

 

• With NO document opened the character spelling language appears to be "Unknown (en–DE)"
• New document: the language remains "Unknown (en–DE)"
• As soon I select a Text tool this setting jumps to - Nothing. Still "Unknown (en–DE)"
• Adding filler text remains "Unknown (en–DE)"
• Expanding changes it to "None". Expected, as the filler text is in latin
• Clicking "Revert Defaults" set it back to "Unknown (en–DE)"
• Choosing "Factory Reset" : "Unknown (en–DE)"

Can you try a ctrl startup equivalent to see if that fixes it?

For MAS version: 

• Go to Macintosh HD\Users\username\Library\Containers\
• rename the 'com.seriflabs.affinityPublisher' folder to 'com.seriflabs.affinityPublisher.old'

For AS version:

• Go to Macintosh HD\Users\username\Library\Application Support\
• rename the 'Affinity Publisher' folder to 'Affinity Publisher Old'

[thomaso]

On 8/19/2020 at 10:13 AM, Gabe said:

Can you try a ctrl startup equivalent to see if that fixes it?

Sorry for my delay. – I got reminded to your answer by this recent question:

>> Can anyone tell me why the default language when I create a new text style in Publisher is 'Unknown (en-PT)'? <<
https://forum.affinity.serif.com/index.php?/topic/121408-publisher-missing-uk-english-dictionary/&do=findComment&comment=676990

So, after renaming my APub pref folder I get this languages: now it's always "None" by default, and even Edit > Defaults > Factory Reset causes your expected "Unknown (en–DE)" only if a frame is selected when the command gets chosen. – Quite strange, compared to your experience, but actually this default "None" appears very useful to me.
– However, I still would like to understand why a "Unknown (xx_XX)" should be a reasonable default, since it will cause unnecessary preflight warnings. As it also obviously does for SarahB in the post linked above.

[Gabe]

On 9/14/2020 at 12:22 AM, thomaso said:

However, I still would like to understand why a "Unknown (xx_XX)" should be a reasonable default, since it will cause unnecessary preflight warnings.

With an app language in English and a system region of Spanish, it expects a "en-ES" dictionary. I think it's a pretty "safe" assumption, based on the OS region and the app language.  

[thomaso]

23 minutes ago, Gabe said:

With an app language in English and a system region of Spanish, it expects a "en-ES" dictionary. I think it's a pretty "safe" assumption, based on the OS region and the app language.  

Sorry, I can't follow. In which way is it "save" when the app expects a language or dictionary which simply can't exist?

We know for sure that there is no dictionary for English with German 'dialect' (or spelling etc.), as "en_DE" would require.
Also there is no dictionary for English with Spanish 'dialect' ("en_ES").
But those default "Unknown"-combinations do cause preflight warnings.

To call this "save" means to welcome preflight warnings by default, well knowing the user can't install the required dictionary. Such a default would force the user for every document, text frame or text style to make a language selection (or 'None') – while a default with "None" simply would not cause preflight warnings. Consider: spell-check isn't necessary (or desired by the user) for every text item in general, for some projects it may be even disturbing, e.g. for letterheads, business cards etc.

So a default like "en_XX" doesn't make sense to me – unless XX is any English, too. Why is the app looking for the current system language at all, if this does not have to be related at all to the used language in Affinity document texts.

Besides that and to my recent video above:
Do you have an idea what went different on my computer that I do get after a reset the (personally preferred) "None" as default while other's do get the kind of paradox but forecasted "Unknown (en_XX)"? – And what causes to me the different app behaviors when I choose "Factory Defaults".

And: can you confirm that a document gets a "general" language saved in the .afpub (regardless of language settings assigned to text, styles or frames)?
If not, how does it happen that an .afpub created in en_EN can cause on a different computer an "Unknown (en_US)" with no object selected? (see above here or here).
But if a document has a general language saved, then for what purpose? And where is the Affinity UI to set that document specific default?

[Aleksandar Kovač]

An old thread, but still very, very relevant. @thomaso condenses it very well:

Quote

Why is the app looking for the current system language at all, if this does not have to be related at all to the used language in Affinity document texts.

and...

Quote

... if a document has a general language saved, then for what purpose? And where is the Affinity UI to set that document specific default?

App should not assume OS or Affinity GUI languages have any relation or relevance to document's content and full separation here is justified. In language-specific markets, that assumption would mostly be wrong since UIs are seldomly translated in smaller languages; pro users prefer English UI (my observation); the content created is rarely in any of the languages Affinity offers, and frequently multi-language.

Additional argument against assuming language on document level is that you have already given us the finest language control imaginable: on character level; which is also a part of Paragraph style's definiton.

And finally, an assumption that is incorrect and malformed (e.g. en-PT, en-HR, en-DE, etc.) is worse than no assumption at all.

I believe that ideally a new document should not be language-specific - 'None'.

...

Now, that Auto-correction in Preferences is another thing completely. I cannot grasp where is it positioned in the context of language control and language tools.

I believe that Auto-correction should be more like: WHEN 'French' THEN 'Auto-correct: French', and not a specific, application-level preference. Especially not one that does not (or does it?) have effect on other language settings in Publisher.

...

For fun (and perhaps for an indication that something is broken) look what I see for spelling language: a separator 😆. So symbolic, considering my location, Croatia.

 

[walt.farrell]

4 hours ago, Aleksandar Kovač said:

I believe that Auto-correction should be more like: WHEN 'French' THEN 'Auto-correct: French', and not a specific, application-level preference.

That is exactly how it operates. It is an application preference, but one which applies to any text in the language specified.

You have separate Auto-Correct settings for English, German, French, and any other language dictionaries you have installed.

[Aleksandar Kovač]

2 hours ago, walt.farrell said:

That is exactly how it operates.

(did experiments with fresh eyes)

Oh. I see. Oh... All of the language dictionaries are available all the time and utilized depending on Character's language definition and pertinent options used. That is exactly what I desired.  How... unexpected (?) 😲.

Then, the source of (my) confusion was: Language dropdown under 'Replace text while typing' in 'Preferences/Auto correct' is not following the convention established by the rest of the 'Preferences' panel. Every other dropdown element is used to set a preference. Dropdown here does not set any preference. It is only a shortcut to a specific list of custom auto-correction/text-expanding entries that are added to the language selected in the same dropdown. I.e. selection here does not matter while working. All of the entries are available if language needs it. If that is right,this is a bit 'on the edge' case for a dropdown. Some extra explanation could make it easier to understand, like this?:

2 hours ago, walt.farrell said:

You have separate Auto-Correct settings for English, German, French, and any other language dictionaries you have installed.

Actually, if on, AP auto-replaces only the entries listed here. Misspelt words will be underlined red. At least, this is what I am getting on Mac. 'Main menu/Text/Spelling/Check spelling while typing' is on, as is 'System preferences/Correct spelling automatically'

+Croatian spelling dictionary is present and active system-wide. But, it does not show up here (it does show up for 'Typography language' and 'Typography script'). Maybe that dictionary is not up to the mark needed here... That leaves only awkwardly named 'None' list for my puny little language. Custom named list would be nice...

...

To everyone, sorry for attempting to expound on this, entire official documentation on this topic is: "Use these options to control how auto-correction intelligently and automatically replaces or formats your art and frame text". The issue with Preflight 'Missing dictionary for language en-XX' still exists.

 

 

[garrettm30]

5 hours ago, Aleksandar Kovač said:

App should not assume OS or Affinity GUI languages have any relation or relevance to document's content and full separation here is justified.

I agree with that. I think the problem is that there has to be a starting value when a user opens a new document. For one thing, I am not convinced that there is any document-level language setting, even internally. I just think rather that when you first start typing, the text of necessity has to have some values defined, such as font and text size, for example. Likewise, the language attribute must be set to something, even if that setting is [none] (which is in itself a setting).

So I think the question then becomes very simple: what should the initial language attribute value be when opening a new document? [None] is one sensible option. But I think it would make it harder on new users who (as our experience on the forum has shown) have trouble grasping how language works in Affinity. So it might be better to give them some more useful starting point.

I think that has led to the current behavior, where the software is trying to make the best guess by combining the user’s UI choice with the OS region. But while it tries to be helpful, it also results in these languages that do not exist and cause problems even harder for the inexperienced user to figure out. Even we in this thread had a good deal of trouble to figure out what was going on, although it seems clear to me know.

So as a third option, I suggest that the starting point could simply be based on the UI setting and left at that, not combined with a region setting that may or may yield a real language.

So probably we all agree that we should recommend Serif move away from the current possibility where we get languages like en-BE or other oddities. So should it be [none] as the starting point or a sensible setting corresponding to the app UI language (or some other option I haven’t thought of)?

Of the two options that I have thought of, I suggest going with the UI (apart from the region) as the starting point rather than [none]. If [none] is the starting point, that means 100% of users who care to use language must change the language attribute in the Character Studio or the corresponding text style (better). If we go with the language UI, then probably better than half already have a setting that works for them, that they never even need to think of, while the people for whom it does not work will still have to change it, which is the same amount of effort as changing from [none] to their desired setting.

Me, I have a UI setting of U.S. English and do most of my Publisher work in French. So my starting point turns out to not be the language I need, but I am okay with that. Even if it were [none], I would still have to change it.

Since we are all talking about character-level formatting anyway, it is exactly analogous to the starting point for the font. How could the software determine the starting point for the font? Good luck with that! The best that could be done is start with a readable font installed on just about everyone’s computer, so Arial. And just about everyone is going to select a different font for nearly every document.

The point is that font is just a starting point for a character-level attribute, and the same is true for language. We just need a better starting point, one that doesn’t try to guess in a composite way that could result in non-existent languages.

[walt.farrell]

9 minutes ago, Aleksandar Kovač said:

Actually, if on, AP auto-replaces only the entries listed here. Misspelt words will be underlined red. At least, this is what I am getting on Mac. 'Main menu/Text/Spelling/Check spelling while typing' is on, as is 'System preferences/Correct spelling automatically'

I have no idea whether that Mac System Preference is supported. There's certainly no such thing on Windows.

[garrettm30]

8 minutes ago, Aleksandar Kovač said:

Then, the source of (my) confusion was: Language dropdown under 'Replace text while typing' in 'Preferences/Auto correct' is not following the convention established by the rest of the 'Preferences' panel. Every other dropdown element is used to set a preference. Dropdown here does not set any preference.

Yes that preference item has confused many people. They think it is a setting, while in fact it just exposes a different set of settings for each language but is not itself a value. Your suggested improvement in the screen shot would be quite an improvement in my opinion.

For the other part, I am not sure, but I have the impression that you are still looking to that preference pane for spell check settings. Let me suggest instead that you forget that preference pane when it comes to spell checking. That pane concerns extra replacements, ones that you can define differently per language, but it is distinct from spell check. The spell check instead happens for each string of text based on the language attribute for the selected text in the Characters Studio or the corresponding setting in the text styles.

If that part was already clear to you, please excuse the unnecessary advice; maybe it will be helpful to someone.

[garrettm30]

4 minutes ago, walt.farrell said:

I have no idea whether that Mac System Preference is supported.

To clarify, on Mac, the languages offered in the language character-level attribute in Affinity are those that are activated at the system level. This screenshot I posted earlier shows what I am talking about:

[walt.farrell]

21 minutes ago, garrettm30 said:

To clarify, on Mac, the languages offered in the language character-level attribute in Affinity are those that are activated at the system level

Yes, but the comment I responded to was about a Mac System setting to automatically correct spelling errors.

[garrettm30]

7 minutes ago, walt.farrell said:

Yes, but the comment I responded to was about a Mac System setting to automatically correct spelling errors.

Although Affinity relies generally on the OS spelling system, it does not appear that either the system “Correct spelling automatically” setting or the user-defined text replacements from the system setting are followed. Instead, the Affinity setting “Replace text while typing” is what controls auto-correction. Or to put it more simply, it appears all auto-correction is controlled by the Auto-Correct settings in Affinity. To put it another way:

• Spelling and hyphenation dictionaries are provided by the system.
• Check spelling while typing and learning new words are controlled through Affinity using system resources
• Auto-correction behavior is controlled by Affinity apart from the system.