filenames are not translated to/from utf-8

[Arne Döring]

when I loaded a file, the file browser showed a folder that had an 'ä', but the 'ä' was replace by two other random characters. This is a clear indication that an utf8 string is read by a single byte encoding. When I save a file with an ä in it's filename I get a filename where my operating system tells me that it is an invalid filename.

[miniupnp]

show the problem on a linux machine

[miniupnp]

OK, I've started to fix that.
with "Work_in_progress_issue10.patch​" file and directories name are displayed correctly in the "Load" dialog.

[miniupnp]

fully working patch

[miniupnp]

I just need to add a #define in order for the code to be enabled only for OS with UTF8 file names ;)

[pulkomandy]

Hi,
Thanks for working on this!

Some comments:

• We definitely need a #ifdef for this, because some platforms may not have libiconv. Something similar to what we do for Lua could be used.
• This is only a partial fix: it only works if the file name can be converted to CP1252. This issue was originally opened because of lack of support for cyrillic chars in file names, and that would still be a problem.

[yrizoud]

About displaying characters that are not in the Windows-1252 code page, I would really like to use a similar system as OpenXcom : A single font image contains individual characters, no matter which encoding / codepage they are originally. A configuration file defines which characters are present, in which order (It can be a simple UTF-8 string of these characters, easy to edit and expand)
With 384 characters, they cover a large number of languages, without useless symbols.
​https://github.com/SupSuper/OpenXcom/blob/master/bin/common/Language/FontBig.png

But I think this should wait for the SDL2 port, because in this one I increased the base resolution. Characters would be 16x16 pixels, this leaves much more room for even the latin accented characters.

[pulkomandy]

We already have the latin accented chars (well, most of them). Fitting cyrillic should not be too hard, if someone can draw the glyphs.

Maybe we could try switching the whole GUI to SDL_ttf and some ttf fonts. Proportional fonts in some places could be interesting, and it would solve that problem.

[miniupnp]

This patch is using #ifdef to enable iconv only when applicable (linux/FreeBSD/etc.)

[pulkomandy]

Thanks, applied in r2172. I'm leaving the issue open for people using cyrillic chars, still.

[Thomas Bernard]

About cyrillic chars, can someone make us a cyrillic font in the same format as font_Classic.png etc. ?

8x8 pixel chars, with 32 chars per line, with the 256 characters from the Unicode 0400-04FF range. The resulting PNG should be 256x64
​http://www.unicode.org/charts/PDF/U0400.pdf

[yrizoud@…]

There is one here.
​http://pixeljoint.com/forum/forum_posts.asp?TID=22464&PID=201060#201060

[Thomas Bernard]

Work in progress

[Thomas Bernard]

almost done !

[Thomas Bernard]

​https://gitlab.com/GrafX2/grafX2/merge_requests/31

still some work to make it work for windows (with small filenames/long filenames)
but I'm close

[Thomas Bernard]

I think everything is OK with the merge request now.
Only thing missing are fonts for missing characters !

[Thomas Bernard]

The code has been merged, so we could close the ticket now,
but let take time to detect all border effects...

I just noticed the Factory displays the short names under Windows :(

[Thomas Bernard]

Factory displaying Windows Short names

[Thomas Bernard]

everything is fixed now. Just need more 8x8 fonts for non-latin characters ;)