Handle /usr/lib/locale and /usr/share/locale

[probonopd]

Some Linux distributions such as NixOS and Chimera Linux don't have /usr/lib/locale and /usr/share/locale, so at least for glibc-based applications these need to be bundled.

[Samueru-sama]

Do we always need to deploy the locales btw? For example I found out we don't always need to deploy gconv from deploying glibc, something that go-appimage will always do as far as I know.

[probonopd]

At least PrusaSlicer is giving me

on NixOS...

[Samueru-sama]

on NixOS...

I assume that the error goes away when you copy /usr/lib/locale and /usr/share/locale over right? or is there a env variable that also needs to be set? sharun already sets XDG_DATA_DIRS to include the share dir but I'm not sure if there is a variable for /usr/lib/locale.

[probonopd]

Don't quite know what is going on but as soon as I set the LOCPATH environment variable I get the error...

[Samueru-sama]

Just reproduced the issue with GIMP, setting LOCPATH to even the location of the host locales breaks them 🤔

[probonopd]

On Debian, indeed LOCPATH=/usr/lib/locale:/usr/share/locale ./AppRun fails, and without setting LOCPATH it runs (and I think I just used the Debian default paths in this test). Strange. Need to find out whether this is caused by sharun.

[Samueru-sama]

On Debian, indeed LOCPATH=/usr/lib/locale:/usr/share/locale ./AppRun fails, and without setting LOCPATH it runs (and I think I just used the Debian default paths in this test). Strange. Need to find out whether this is caused by sharun.

I tested with the arch gimp package and it breaks as well.

[probonopd]

I wonder what happens if you do the same with a distribution-installed gimp.

[Samueru-sama]

I wonder what happens if you do the same with a distribution-installed gimp.

Isn't the gimp I just tested a distribution-installed gimp?

[probonopd]

Sorry, didn't catch this. So either we are using LOCPATH wrong or or it is entirely broken?

[Samueru-sama]

Sorry, didn't catch this. So either we are using LOCPATH wrong or or it is entirely broken?

https://www.gnu.org/software/libc/manual/html_node/Locale-Names.html

According to that the default for LOCPATH is /usr/share/locale but it breaks if I set it to that.

[Samueru-sama]

Qt apps will also complain about this, so hopefully I'm doing something wrong:

[probonopd]

According to https://www.man7.org/linux/man-pages/man7/locale.7.html,

LOCPATH
A list of pathnames, separated by colons (':'), that
should be used to find locale data. If this variable is
set, only the individual compiled locale data files from
LOCPATH and the system default locale data path are used;
any available locale archives are not used (see
localedef(1)). The individual compiled locale data files
are searched for under subdirectories which depend on the
currently used locale. For example, when en_GB.UTF-8 is
used for a category, the following subdirectories are
searched for, in this order: en_GB.UTF-8, en_GB.utf8,
en_GB, en.UTF-8, en.utf8, and en.

I wonder whether this is the reason.

And indeed when I move /usr/lib/locale/locale-archive away on my host system, I also get the error with LOCPATH unset.

In this case, it searches for LC_CTYPE files which don't exist on my system shortly before failing.

openat(AT_FDCWD, "././/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/locale/en_US.UTF-8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US.UTF-8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/locale/en_US.utf8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US.utf8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/locale/en_US/LC_CTYPE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en_US/LC_CTYPE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/locale/en.UTF-8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en.UTF-8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/locale/en.utf8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en.utf8/LC_CTYPE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/locale/en/LC_CTYPE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/share/locale/en/LC_CTYPE", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
prctl(PR_SET_NAME, "slic3r_main")       = 0
futex(0x7ff3b2d7d210, FUTEX_WAKE_PRIVATE, 2147483647) = 0
write(2, "An error occured while setting u"..., 248An error occured while setting up locale.
You may need to reconfigure the missing locales, likely by running the "locale-gen" and "dpkg-reconfigure locales" commands.
PrusaSlicer will now terminate.

But...

$ locale-gen --output-dir=./share/locale --all
Generating locales (this might take a while)...
  en_US.UTF-8...cannot create temporary file: /usr/lib/locale/locale-archive.0nfDnJ: Permission denied
 done
Generation complete.

[Samueru-sama]

haha, I was checking strings on all the libs to see which one contained /usr/lib/locale-archive to see if patching it works and just noticed you recent commit.

[probonopd]

Testing this with NixOS now.

Another option:
At least on my Ubuntu, /usr/lib/locale/ did not contain many files. Installing sudo apt install locales-all populated them. Now LOCPATH is no longer broken. Maybe we need to bundle the contents of that package if the libarchive-locale patching doesn't work out.

[probonopd]

sed -i -e 's|/usr/lib/locale|././/lib/locale|g' ./shared/lib/libc.so.6 works, the bundled lib/locale-archive gets used.
If we absolutely want to stay away from this patching, then bundling the contents of the locales-all Debian package would likely also work.

And with that, my demo application (PrusaSlicer) now even runs on NixOS (with --appimage-extract for now), albeit "WebKit encountered an internal error".

[Samueru-sama]

then bundling the contents of the locales-all Debian package would likely also work.

Noted, and yeah I've noticed I was locale errors when running other appimages on alpine but they didn't stop the app from working, this is really needed because changing the working directory is an issue with some apps.

with --appimage-extract for now

Doesn't it work if you set FUSERMOUNT_PROG="$(command -v fusermount)"?

[probonopd]

FUSERMOUNT_PROG=$(which fusermount) ./My.AppImage works nicely. So this is something we definitely need to fix in the AppImage type2-runtime. Apparently it doesn't search for fusermount on the PATH properly yet.

• Find fusermount on the $PATH AppImage/type2-runtime#31

[VHSgunzo]

It will work if you just use uruntime. And it is also likely to help reduce the size of the appimage and speed up its work (If you use DwarFS)

[VHSgunzo]

@probonopd

./uruntime-appimage-dwarfs-x86_64 --appimage-mkdwarfs -f --set-owner 0 --set-group 0 --no-history --no-create-timestamp --header uruntime-appimage-dwarfs-x86_64 -i squashfs-root -o PrusaSlicer-2.9.0-alpha1-x86_64.dwfs.AppImage --compression zstd:level=22 -S20 -B9

[probonopd]

Very promising @VHSgunzo. Will come to uruntime later. Might consider it as the default for AppImage if it turns out as great as it sounds.

[probonopd]

So, for my personal test case I found solutions for all of the above. But it is a lot of manual work, Is it in the scope of this project to provide automation for this? Thanks.

[Samueru-sama]

Looks like all locales have to be bundled siince the patch on libc.so isn't enough.

[probonopd]

@Samueru-sama thanks for the pointer. Regarding PrusaSlicer issue 13653, do you have more insight into what exactly needs to be changed?

[Samueru-sama]

@Samueru-sama thanks for the pointer. Regarding PrusaSlicer issue 13653, do you have more insight into what exactly needs to be changed?

The user that had the locale issue is from Romania, so I guess the locale that's used there isn't available in the locale-archive of the AppImage.