Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Localization support and transifex integration #55

Open
wants to merge 31 commits into
base: master
Choose a base branch
from

Conversation

NaitLee
Copy link

@NaitLee NaitLee commented Feb 8, 2024

This contributes to progress of #54 by the following additions:

  • Script lang/makepot.sh for gathering & updating strings from source code (of all branches) for translation, and it excludes vanilla strings from this codebase
  • Script lang/merge.sh for “merging” translations: merge vanilla strings when packaging for distribution as a whole as a pk3 (will also be run by build.sh by default)
  • Script lang/updatepo.sh for updating existing po files manually, useful for offline translation
  • Transifex configuration .transifex.yml for integration with the service
  • Some initial Chinese translations are also included

Before merge, check if the scripts work on your machine… :)
NOTE: merge these code to a local branch of official repository (not my fork) and then test. it needs other gameplay branches to work

After merging, consider:

  • Create your own team on transifex, set this project up with provided configuration, and put the project link to readme
  • (or probably I can manage a team, if you want)
  • Run lang/makepot.sh, to have new strings (those from other branches) included for translation; consider running it routinely
  • Make more strings translatable; move server side strings to client if necessary; some short strings may be better to have prefix for precious meaning
  • After some time, make an optional content pack with newest translations for the mod by running lang/merge.sh, and consider to distribute it

If I missed something just ask… :)

@NaitLee NaitLee force-pushed the NaitLee/more-localizable-strings branch from 8c4442b to 9bfc953 Compare May 20, 2024 00:22
@NaitLee
Copy link
Author

NaitLee commented May 20, 2024

@LegendaryGuard just have added you to the testing transifex team. I was busy for a very long time and didn't take time to manage that.

@LegendaryGuard
Copy link
Collaborator

@LegendaryGuard just have added you to the testing transifex team. I was busy for a very long time and didn't take time to manage that.

Nice! Right now, I just entered into the testing transifex project and I translated few japanese strings to see if it updates in the next time.
How long does that last to be updated? Every (2, 3, 4, ...) days, a week, a month, or...? I don't have clue how it's configured.

@LegendaryGuard LegendaryGuard linked an issue May 20, 2024 that may be closed by this pull request
2% of minimum 1% translated source file: 'lang/common.pot'
on 'ja_JP'.

Sync of partially translated files: 
untranslated content is included with an empty translation 
or source language content depending on file format
@NaitLee
Copy link
Author

NaitLee commented May 20, 2024

How long does that last to be updated? Every (2, 3, 4, ...) days, a week, a month, or...? I don't have clue how it's configured.

I just modified the config to 1% translated before pushing to repo. It's been 50%, though I noted that the updated pot contains so many strings... :)

I see a push was issued after I added a string afterwards, with some minutes delay. (I set a 5 mins cooldown as for now)

@LegendaryGuard
Copy link
Collaborator

LegendaryGuard commented May 21, 2024

Great!
I tested your branch, the translated strings in-game work as expected. All ok!
Details: merge.sh outputs dirty errors when there's no directory or file:

mkdir: created directory 'compiled/zzz-smb-l10n.pk3dir'
rm: cannot remove 'compiled/zzz-smb-l10n.pk3dir/*': No such file or directory
Merging localization files to compiled/zzz-smb-l10n.pk3dir
rm: cannot remove 'compiled': Is a directory
rm: cannot remove 'compiled/zzz-smb-l10n.*.pk3': No such file or directory
Making localization package compiled/zzz-smb-l10n.a0491c0c.pk3
  adding: common.ja_JP.po (deflated 80%)
  adding: common.zh_CN.po (deflated 78%)
  adding: common.zh_HK.po (deflated 78%)
  adding: common.zh_TW.po (deflated 78%)
  adding: zzz-smb-l10n.pk3.serverpackage (stored 0%)

The console/terminal output should be clean, so I made some adjustments to check the directories and pk3 file in merge.sh:

#!/usr/bin/env bash
cwd="`pwd`"
cd "`dirname $0`/.."

base=xonotic
if [ ! -d "$base" ]; then
    echo "RTFM (README.md)"
    cd $cwd
    exit 1
fi

pk3name=zzz-smb-l10n
dst=compiled/${pk3name}.pk3dir

if [ ! -d "$dst" ]; then
    mkdir -vp $dst
fi
if [ -d "$dst" ] && [ "$(ls -A $dst)" ]; then
    rm -v $dst/*
fi

echo "Merging localization files to $dst"
for i in $(ls lang); do
    if [ -f lang/$i ] && [ -f $base/$i ] && [[ "$i" != "common.pot" ]]; then
        msgcat --to-code UTF-8 $base/$i lang/$i > $dst/$i
    fi
done

if [ ! -n "${ZIP:-}" ]; then
    if [ -e "compiled/${pk3name}.*.pk3" ]; then
        rm -v compiled/${pk3name}.*.pk3
    fi
    # unique id for different versions of localization
    sha256=$(cat compiled/${pk3name}.pk3dir/* | sha256sum)
    pk3path=compiled/$pk3name.${sha256:0:8}.pk3
    echo "Making localization package $pk3path"
    echo "localization package for smb server" >$dst/${pk3name}.pk3.serverpackage
    zip -9 -j $pk3path $dst/*
fi

cd $cwd

It would be ok to update the script in the PR.

Suggestion: that could be interesting is that you can see the percentage of each language of how much it's translated, an example would be something like this:
https://gitlab.com/xonotic/xonotic-data.pk3dir/-/blob/master/languages.txt
https://gitlab.com/xonotic/xonotic-data.pk3dir/-/commit/567ff7590cb21d3910d74a966a9563f0bb86b369#a436ade39b3ccbf92fb013034c399e506c1d7d92

That could save time going to check the completion percentage in transifex. But now I see that the percentage here is included inside commits like these:
13007fa
efc9171

P.D. Going to add some translated japanese strings in transifex to see if it's going as expected besides than adding a new .po file.

2% of minimum 1% translated source file: 'lang/common.pot'
on 'ja_JP'.

Sync of partially translated files: 
untranslated content is included with an empty translation 
or source language content depending on file format
@LegendaryGuard
Copy link
Collaborator

LegendaryGuard commented May 21, 2024

How long does that last to be updated? Every (2, 3, 4, ...) days, a week, a month, or...? I don't have clue how it's configured.

I just modified the config to 1% translated before pushing to repo. It's been 50%, though I noted that the updated pot contains so many strings... :)

I see a push was issued after I added a string afterwards, with some minutes delay. (I set a 5 mins cooldown as for now)

Ok. We think that the transifex bot should check once a week, what do you think, @MarioSMB ? Or should be once a month? If not, how long do you prefer for the bot to check the updates in transifex of some languages and make a commit?

Once all that is done, let's rename the transifex project name by replacing test-smbmod with xonotic-smbmod (or whatever @MarioSMB names it) and add the updated link to the README in this pending line:
https://github.com/MarioSMB/modpack/pull/55/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R57

P.D. Transifex bot commit from right now: Nice, it checks and updates correctly as expected!

@NaitLee
Copy link
Author

NaitLee commented Jul 19, 2024

It would be ok to update the script in the PR.

done

Why do these words from another branch (cyber) appear there?

makepot.sh gathers translatable strings from all mod branches, so that the translation can be used for all variation of mods.

@LegendaryGuard
Copy link
Collaborator

LegendaryGuard commented Aug 13, 2024

Understood. You need to clarify that requires installed this package to execute the translations (msgmerge and xgettext commands):

  • Linux:
sudo apt-get update
sudo apt-get gettext
  • Windows MSYS:
pacman -S gettext

Ok, so the manual process is:

  • Go to lang/ directory
  • Execute: ./makepot.sh
  • Execute: ./merge.sh
  • Copy/Move zzz-smb-l10n*.pk3 into xonotic-data.pk3dir/data, Xonotic/data (or the PATH in README)
  • Run the game and enjoy with the translated languages from the mod!
  • Execute if there are new translations: ./updatepo.sh

updatepo.sh should remove the *.po~ files if needed, I think.

Btw, I created a new organization to attach the modpack: https://app.transifex.com/team-xonsmb/
Here the empty created project to attach: https://app.transifex.com/team-xonsmb/xonotic-smb-modpack/dashboard/

So, I think you can replace that line in the README to the URL I just shared: https://github.com/MarioSMB/modpack/pull/55/files#diff-b335630551682c19a781afebcf4d07bf978fb1f8ac04c6bf87428ed5106870f5R57

If you do, consider updating the documentation I just shared.

And I guess the instructions to execute successfully from the organization are there, right? #55 (comment)

NaitLee and others added 8 commits September 15, 2024 14:12
34% of minimum 1% translated source file: 'lang/common.pot'
on 'zh-Hant'.

Sync of partially translated files: 
untranslated content is included with an empty translation 
or source language content depending on file format
34% of minimum 1% translated source file: 'lang/common.pot'
on 'zh_TW'.

Sync of partially translated files: 
untranslated content is included with an empty translation 
or source language content depending on file format
34% of minimum 1% translated source file: 'lang/common.pot'
on 'zh_CN'.

Sync of partially translated files: 
untranslated content is included with an empty translation 
or source language content depending on file format
2% of minimum 1% translated source file: 'lang/common.pot'
on 'ja_JP'.

Sync of partially translated files: 
untranslated content is included with an empty translation 
or source language content depending on file format
34% of minimum 1% translated source file: 'lang/common.pot'
on 'zh_HK'.

Sync of partially translated files: 
untranslated content is included with an empty translation 
or source language content depending on file format
@NaitLee
Copy link
Author

NaitLee commented Sep 15, 2024

Things are updated.

./updatepo.sh are intended for those who translate via editing .po files directly (such as me)
(and they just upload .po files to Transifex)
Code maintainers don't have to run it, because updates to .po files are done by Transifex.

To avoid pull request spams, you may configure Transifex to directly push translations to master branch.

@MarioSMB
Copy link
Owner

MarioSMB commented Oct 3, 2024

Thank you for continuing to work on this - I've given LegendGuardian permission to set it up at the earliest convenience.

@LegendaryGuard
Copy link
Collaborator

@NaitLee Sorry for asking you some questions, but I want to make sure I apply everything correctly. Since the team and project have already been created, I'm not sure if I missed any steps in Transifex before running these scripts via GitHub Actions:

  1. Is this ok?
opensrcprojattach
  1. Does GitHub integration need to be enabled? I think this step requires the owner's permission and the installation of the Transifex GitHub Actions.
transifexghintegration

@NaitLee
Copy link
Author

NaitLee commented Feb 2, 2025

@LegendaryGuard

The project url should be ok.

The "integration" should be installed, and should be confirmed by the GitHub account owning this repository, then probably configured with target .pot file(s):

image

Note that this integration isn't GitHub Action but GitHub "App" for a repository:

image

(Nevertheless you may create a GitHub Action to automatically build translation pk3s. I'm not experienced to this yet.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Translation support
3 participants