Page 1 of 1

Emoji in filenames

Posted: 29 Aug 2017, 17:36
by AnyOldName3
Partly because I was wondering, and partly because a user mentioned having trouble with loading mods with an é in the name, I just did some experimentation with loading data files and plugins with emoji in the names. All this testing was done under Windows.

The good news:
  • Both data folders and plugin files can be loaded when they have emoji in the name.
The bad news:
  • The launcher renders emoji as a rectangle.
  • MyGUI renders emoji as a pair of question marks.
  • At least under Windows, the console prints emoji as several characters.

Re: Emoji in filenames

Posted: 29 Aug 2017, 21:24
by scrawl
The launcher renders emoji as a rectangle.
That could be a bug. Do any other unicode characters work?
MyGUI renders emoji as a pair of question marks.
To be expected, as the Morrowind font does not include such characters. Any unsupported glyph will show as a question mark.

Re: Emoji in filenames

Posted: 29 Aug 2017, 21:34
by AnyOldName3
I've not tested other ranges of Unicode yet, so don't know what Qt is and isn't displaying. I'll investigate further.

As for the Morrowind font, how should that be resolved? A bunch of software falls back to an emoji font if the current font doesn't support a character, and it's so common that it might possibly be just part of how a modern OS works. Would we have to manually add this behaviour to MyGUI? Googling to see if anyone's tried supporting it before just brings up this thread and a bunch of unrelated stuff that just happens to have similar words.

Re: Emoji in filenames

Posted: 30 Aug 2017, 12:06
by AnyOldName3
So other Unicode characters seem to work mostly fine except where they don't. I tried é and saw that:

Good:
  • Stuff still loaded just fine
  • The launcher rendered it correctly
  • MyGUI rendered it
Bad:
  • When MyGUI displayed é, it was displayed slightly lower than other characters. This could well just be a flaw with the font, though.
  • The console rendered é as two other characters. This is possibly a Windows-only issue
I'm going to check to see what happens under Linux, as I reckon things are being weird after reading this thread: viewtopic.php?f=8&t=4563

Re: Emoji in filenames

Posted: 30 Aug 2017, 12:31
by AnyOldName3
Everything that works under Windows is working for me under Ubuntu, with the addition that the launcher displays emoji. As only the Windows version seems to pop up a terminal behind OpenMW, there's no terminal to incorrectly render Unicode, but if I run it from a terminal, it's rendered correctly there. Apparently, without complicated meddling, Windows terminals just suck at Unicode.

Re: Emoji in filenames

Posted: 30 Aug 2017, 14:23
by Chris
Windows and UTF-8 don't have a good history together. While most systems have moved on to use UTF-8 by default, on Windows the char-based string functionality tends to be interpreted according to the system's code-page, and the only reliable way to handle Unicode characters with system functions is by using the wide-char variants. Toolkits like Qt help with this when you use their classes, but if for example you use std::string or char* with std::cout, who knows how Windows will interpret it.

Re: Emoji in filenames

Posted: 30 Aug 2017, 21:49
by redsteakraw
This could be a font issue try installing the symbola font as it includes all the emoji in monochrome text form. Most systems fill in the missing unicode code points in a font with another font so if you install symbola it should help with the displaying of the emoji. you can get it here http://users.teilar.gr/~g1951d/.

Re: Emoji in filenames

Posted: 30 Aug 2017, 23:00
by AnyOldName3
I did a test where I forced the launcher to use a different font, and it started displaying emoji under Windows (but the whole font was changed). Does anyone know anything about how the default font is selected and which one it ends up being?

Re: Emoji in filenames

Posted: 06 Sep 2017, 16:41
by AnyOldName3
I've just done some meddling, and now I can't get the launcher to display emoji as a box (i.e. incorrectly) under Windows using a build based on the master branch, even when trying another computer. I'm going to assume that someone else fixed it since 0.42, where it definitely doesn't display emoji.

The current issues with emoji in filenames are as follows:
  • MyGUI doesn't display any emoji, using question marks instead. I get the impression it would work if we told it to use a font which supported them.
  • macOS only: Loading mod files when emoji (or even more sane characters, such as é) are in the filename results in a 'content file not found' error. As there are mods which actually have an é in the name, this really should be investigated and fixed.
Things that I don't have test results for, so may either be broken or not:
  • The launcher may or may not display emoji correctly when listing content files with such characters in the name