Emoji in filenames

Everything about development and the OpenMW source code.
Post Reply
User avatar
AnyOldName3
Posts: 2668
Joined: 26 Nov 2015, 03:25

Emoji in filenames

Post by AnyOldName3 »

Partly because I was wondering, and partly because a user mentioned having trouble with loading mods with an é in the name, I just did some experimentation with loading data files and plugins with emoji in the names. All this testing was done under Windows.

The good news:
  • Both data folders and plugin files can be loaded when they have emoji in the name.
The bad news:
  • The launcher renders emoji as a rectangle.
  • MyGUI renders emoji as a pair of question marks.
  • At least under Windows, the console prints emoji as several characters.
Last edited by AnyOldName3 on 30 Aug 2017, 12:06, edited 1 time in total.
User avatar
scrawl
Posts: 2152
Joined: 18 Feb 2012, 11:51

Re: Emoji in filenames

Post by scrawl »

The launcher renders emoji as a rectangle.
That could be a bug. Do any other unicode characters work?
MyGUI renders emoji as a pair of question marks.
To be expected, as the Morrowind font does not include such characters. Any unsupported glyph will show as a question mark.
User avatar
AnyOldName3
Posts: 2668
Joined: 26 Nov 2015, 03:25

Re: Emoji in filenames

Post by AnyOldName3 »

I've not tested other ranges of Unicode yet, so don't know what Qt is and isn't displaying. I'll investigate further.

As for the Morrowind font, how should that be resolved? A bunch of software falls back to an emoji font if the current font doesn't support a character, and it's so common that it might possibly be just part of how a modern OS works. Would we have to manually add this behaviour to MyGUI? Googling to see if anyone's tried supporting it before just brings up this thread and a bunch of unrelated stuff that just happens to have similar words.
User avatar
AnyOldName3
Posts: 2668
Joined: 26 Nov 2015, 03:25

Re: Emoji in filenames

Post by AnyOldName3 »

So other Unicode characters seem to work mostly fine except where they don't. I tried é and saw that:

Good:
  • Stuff still loaded just fine
  • The launcher rendered it correctly
  • MyGUI rendered it
Bad:
  • When MyGUI displayed é, it was displayed slightly lower than other characters. This could well just be a flaw with the font, though.
  • The console rendered é as two other characters. This is possibly a Windows-only issue
I'm going to check to see what happens under Linux, as I reckon things are being weird after reading this thread: viewtopic.php?f=8&t=4563
User avatar
AnyOldName3
Posts: 2668
Joined: 26 Nov 2015, 03:25

Re: Emoji in filenames

Post by AnyOldName3 »

Everything that works under Windows is working for me under Ubuntu, with the addition that the launcher displays emoji. As only the Windows version seems to pop up a terminal behind OpenMW, there's no terminal to incorrectly render Unicode, but if I run it from a terminal, it's rendered correctly there. Apparently, without complicated meddling, Windows terminals just suck at Unicode.
Chris
Posts: 1625
Joined: 04 Sep 2011, 08:33

Re: Emoji in filenames

Post by Chris »

Windows and UTF-8 don't have a good history together. While most systems have moved on to use UTF-8 by default, on Windows the char-based string functionality tends to be interpreted according to the system's code-page, and the only reliable way to handle Unicode characters with system functions is by using the wide-char variants. Toolkits like Qt help with this when you use their classes, but if for example you use std::string or char* with std::cout, who knows how Windows will interpret it.
redsteakraw
Posts: 3
Joined: 29 Mar 2017, 01:36

Re: Emoji in filenames

Post by redsteakraw »

This could be a font issue try installing the symbola font as it includes all the emoji in monochrome text form. Most systems fill in the missing unicode code points in a font with another font so if you install symbola it should help with the displaying of the emoji. you can get it here http://users.teilar.gr/~g1951d/.
User avatar
AnyOldName3
Posts: 2668
Joined: 26 Nov 2015, 03:25

Re: Emoji in filenames

Post by AnyOldName3 »

I did a test where I forced the launcher to use a different font, and it started displaying emoji under Windows (but the whole font was changed). Does anyone know anything about how the default font is selected and which one it ends up being?
User avatar
AnyOldName3
Posts: 2668
Joined: 26 Nov 2015, 03:25

Re: Emoji in filenames

Post by AnyOldName3 »

I've just done some meddling, and now I can't get the launcher to display emoji as a box (i.e. incorrectly) under Windows using a build based on the master branch, even when trying another computer. I'm going to assume that someone else fixed it since 0.42, where it definitely doesn't display emoji.

The current issues with emoji in filenames are as follows:
  • MyGUI doesn't display any emoji, using question marks instead. I get the impression it would work if we told it to use a font which supported them.
  • macOS only: Loading mod files when emoji (or even more sane characters, such as é) are in the filename results in a 'content file not found' error. As there are mods which actually have an é in the name, this really should be investigated and fixed.
Things that I don't have test results for, so may either be broken or not:
  • The launcher may or may not display emoji correctly when listing content files with such characters in the name
Post Reply