Bug: Matching Locale with Negotiate Locale does not work in some cases as expected

[crustamet]

PHP Version

8.3

CodeIgniter4 Version

4.5.5

CodeIgniter4 Installation Method

Manual (zip or tar.gz)

Which operating systems have you tested for this bug?

Linux

Which server did you use?

apache

Database

No response

What happened?

public string $defaultLocale = 'en-US';
public bool $negotiateLocale = true;
public array $supportedLocales = ['en-US','en-GB'];

why like this ? Because i want to show different content for United Kingdom vs United States

Steps to Reproduce

to reproduce you must place UK first instead of the US

The app does not have {locale} specific routes

in your view just put it should always be equal to the $Request->getLocale()

In my Language Folder
I have them in separate folders en-US and en-GB

In the request header i have this
accept-language: en-US,en-GB;q=0.9,en;q=0.8,ro;q=0.7

and if i switch to English - United Kingdom the accept-language changes to this
accept-language:en-GB;en-US;q=0.9,en;q=0.8,ro;q=0.7

Expected Output

When i have the United Kingdom version as my default i should see

when i get the Locale from the Request the en-GB version instead of the en-US

echo $Request->getLocale(); always output en-US in both cases
it should return the en-GB version !

The problem is why is this happening is because of this file
https://github.com/codeigniter4/CodeIgniter4/blob/develop/system/HTTP/Negotiate.php#L181

There is a loop over the Supported languages and the first two are both english from my SupportedLocales so it returns only the first Locale matched and not the exact matched one.

I have test this it only works when i change the order from the
App Config
public array $supportedLocales = ['en-US','en-GB']; always shows the US page even if you have the default language en-GB
public array $supportedLocales = ['en-GB','en-US']; always shows the GB page even if you have the default language en-US.

Anything else?

No response

[michalsn]

The problem here is that we want a loose comparison for locales most of the time.

Setting the last variable to false should help: https://github.com/codeigniter4/CodeIgniter4/blob/develop/system/HTTP/Negotiate.php#L130

Should we make that configurable? Is it worth creating another config file/property? Personally, I'm not convinced, although I would like to hear other people's opinions.

The instant solution would be to extend the Negotiate class and declare changed service in app/Config/Services.php file - which I recommend.

Anyway, not really a bug, maybe a missing feature.

[ALTITUDE-DEV-FR]

Hello,
I also have an issue with {locale} in url_to() and route_to().

For example, if I create a page in French at /fr/mypage (/{locale}/mypage)
and use url_to('Mycontroller/', 'slug');, it returns the page as /en/mypage.

I'm not sure if this issue is related?

Thanks.

[michalsn]

@ALTITUDE-DEV-FR No, I don't think so.

Please make sure you have updated the App::supportedLocales array with fr. If that's not it, please create a separate issue and provide a minimal code sample so we can reproduce the problem.

[crustamet]

So any update on this, because from my POV the negotiate locale does not do what it says

"Once this is enabled, the system will automatically negotiate the correct language based upon an array of locales that you have defined in $supportLocales. If no match is found between the languages that you support, and the requested language, the first item in $supportedLocales will be used. In the following example, the en locale would be used if no match is found:"

It just does not do that

So it appears that it does not find en-GB but it find en-US

Entering from en-GB returns en-US because from the algorithm there who wrote that searches for the first segment of the string

"en" so it matches then returns the first from the array, even thoug i have exact match on en-GB still returns en-US

how come this is not a bug

[michalsn]

As I explained in my first post:

The problem here is that we want a loose comparison for locales most of the time.

The intention here was quite clear if you look at the current implementation - locale subtypes are matched to a broad type. The examples in the user guide also fit into this scenario.

Feel free to send a PR with improvements.

[crustamet]

Well i have managed to get it working but i don't know man how to PR !

can i just put the code here and you manage it ?

public function language(array $supported): string
{
	return self::getBestLanguageMatch($supported, $this->request->getHeaderLine('accept-language'));
}

public static function getBestLanguageMatch(array $supportedLocales, string $acceptLanguageHeader): string
{
    $acceptedLocales = self::parseAcceptLanguage($acceptLanguageHeader);

    // Sort accepted locales by their quality value in descending order
    usort($acceptedLocales, function ($a, $b) {
        return $b['q'] <=> $a['q'];
    });

    // Find the best match
    foreach ($acceptedLocales as $acceptedLocale) {
        foreach ($supportedLocales as $supportedLocale) {
            if (self::isLocaleMatch($acceptedLocale['locale'], $supportedLocale)) {
                return $supportedLocale;
            }
        }
    }

    // Default to the first supported locale if no match is found
    return $supportedLocales[0];
}

private static function parseAcceptLanguage(string $header): array
{
    $locales = [];
    $parts = explode(',', $header);

    foreach ($parts as $part) {
        $sections = explode(';', trim($part));
        $locale = $sections[0];
        $quality = isset($sections[1]) ? (float)str_replace('q=', '', $sections[1]) : 1.0;

        $locales[] = [
            'locale' => $locale,
            'q' => $quality,
        ];
    }

    return $locales;
}

private static function isLocaleMatch(string $acceptedLocale, string $supportedLocale): bool
{
    // Exact match or language-only match (e.g., 'en' matches 'en-US')
    return $acceptedLocale === $supportedLocale ||
        strpos($acceptedLocale, '-') === false && stripos($supportedLocale, $acceptedLocale . '-') === 0;
}

with this it returns the exact match if exist despite the order of SupportedLocales array

"loose comparison my ass"

getBestMatch execution time: 4.0054321289062E-5 seconds
getBestLanguageMatch execution time: 1.0967254638672E-5 seconds

[crustamet]

while trying out these updates and playing with the supported langs
i accidently removed all supportedLangs from the app config
so that supportedLangs = [];

and then there was no 'en' language folder because i changed it to en-US

my server went down because of this with current implementation

[michalsn]

Well i have managed to get it working but i don't know man how to PR !
can i just put the code here and you manage it ?

No, sorry - this is not how open-source works.

You may find this helpful: https://github.com/codeigniter4/CodeIgniter4/blob/develop/contributing/pull_request.md

However, I don’t think we will accept changes that reinvent the wheel and duplicate methods we already have.

[crustamet]

i've updated my Codeigniter to 4.5.7

the locales now works as expected but there is an issue that I cannot solve for some locales because different browsers have different locale codes, not all of them.

I just want to point this out maybe for future or something..

These are my chrome settings

Accept-Language: mo,en-US;q=0.9,en;q=0.8,ro;q=0.7,en-GB;q=0.6

These are my firefox settings

Accept-Language: ro-MD,ro;q=0.8,ro-RO;q=0.7,en-GB;q=0.5,en-US;q=0.3,en;q=0.2

For some whatever reason Moldovian language has two annotations for the same locale but this is not true in fact they have just changed the official language to Romanian, but chrome does not have this locale updated

So i need a way to point to the right directory.

mo needs to point to the same directory as ro-MD because these are the same ? i just don't know maybe it will get clarified in the future. Even if this is not the case i still need a way to point to a directory.

Currently i need to duplicate my locale folder for it to work so i tried to change the $supportedLanguages to key value array
For sure that this does not work :)

[
	'en' 	=> 'en-US', 
	'en-US' => 'en-US',
	'en-GB' => 'en-GB',
	'ro-MD' => 'ro-MD',
	'mo' 	=> 'ro-MD',
	'ro-RO' => 'ro-RO',
	'ro'	=> 'ro-RO',
]

You guys think that this is wrong ? I just don't want duplicate local folders and duplicate locale urls.

I found this:

[neznaika0]

Instal CI 4.6 version.
You will get ro or ro-* if the header has the wrong locale, for example, ro-MD = ro-Ro or ro-MD = ro.

Alternatively, you can add a filter for Request and replace ro-MD with mo.

CI4 does not know how to do what you want.

[crustamet]

i've updated my Codeigniter to 4.5.7, there is no 4.6, on that branch the version is still 4.5.7 as is states in the Codeigniter.php CI_VERSION

For sure this 4.5.7 got your update on strictLocale im using it already and works perfectly, im still adjusting my code and testing this...

[michalsn]

@crustamet There isn't much we can do about it. mo is deprecated. Chrome uses it probably due to some compatibility reasons - IDK really.

As for the solution - this is quite a specific case and can be handled via a simple Filter, so I don't think we gonna add support for such a thing to the framework.

[michalsn]

Resolved via #9360