unzip unicode issue? [SOLVED]

skoegl · Last edited by skoegl on Sun Sep 03, 2017 1:49 pm; edited 1 time in total

Hey,

I've encountered some zip archives with unicode filenames (german). I can unzip them properly on debian based and fedora systems, but not on my beloved gentoo systems.

First of all my locale settings:

mike155 · Posted: Wed Aug 30, 2017 10:24 pm Post subject:

1) LC_ALL= looks wrong. Unset it! You don't need it, since you already set the more specific LC-variables. If you really want to set it, set it to "de_DE.utf8", but not to the empty string - this causes exactly the problem you see: filenames are interpreted as ASCII text and not as Unicode.
2) Please verify that unzip was installed with USE flag 'unicode'

skoegl · Posted: Thu Aug 31, 2017 1:00 am Post subject:

charles17 · Advocate Joined: 02 Mar 2008 Posts: 3664

mike155 · Posted: Thu Aug 31, 2017 8:15 am Post subject:

If locale settings are correct and if you compiled unzip with USE flag 'unicode', then maybe the filename in the zip file is faulty.

Here is what I get when I create a file 'Außendienst', pack it into a zip file and then look at the zip file:

mike155 · Posted: Thu Aug 31, 2017 10:22 am Post subject:

Another possibility: you would also see 'AuÃŸendienst' if you use PuTTY to connect to your Gentoo server, but PuTTY itself is configured to "Latin1". In this case, your ZIP file and your Gentoo server are configured correctly and the filename of the extraced file is OK, but your terminal has the wrong configuration. So you might want to check the configuration of your terminal.

skoegl · Posted: Sat Sep 02, 2017 3:11 pm Post subject:

Now I've digged somewhat deeper into the problem.

I've created a small testcase like you did. Here is the snippet I'm using:

skoegl · Posted: Sat Sep 02, 2017 4:13 pm Post subject:

It seems, that my ubuntu system seems to use a different date formatting as my gentoo box. That could be a sign, that something is not configured properly on that side.

I'm trying to understand what are the differences to specify a utf8 suffix as in "de_DE.UTF-8" compared to "de_DE.utf8". The SUPPORTED file suggests to use the UTF-8 suffix - both on gentoo as on ubuntu. But when calling locale -a it will be displayed as "utf8" anyway. Mhm... strange

Ant P. · Watchman Joined: 18 Apr 2009 Posts: 6920

Try using p7zip instead (7z x file.zip).

NeddySeagoon · Posted: Sun Sep 03, 2017 9:28 am Post subject:

skoegl,

Have you tried the -U option to unzip?
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

skoegl · Posted: Sun Sep 03, 2017 1:47 pm Post subject:

Hey everyone,

good news. I've just managed to unzip correctly the testcase zipfile. After reading a lot of threads about ubuntu localization on docker containers, I've played (again) with DOCKERFILE ENV setting. Against the recommendation to set LANG, LANGUAGE and LC_ALL, now I'm only set LANG. Et voila, it works. I will incorporate the ziptest in my QA suite to be sure everything is in line in the feature.

Thank you all for your support and your time. I'm just a little concerned that I did not understand up until now, what exactly was going wrong in the interaction between the ubuntu container and my gentoo box. Perhaps cross-cultural misunderstandings

Please consider this thread as solved.

steveL · Posted: Sat Sep 09, 2017 11:50 pm Post subject:

LC_ALL showing in the output of locale, does not mean it is set; to verify: