Failing tests for IDN TLD that contains uppercase

[rugabarbo]

Q	A
QA	yes

These are just failing tests for top-level IDN domains that contain uppercase letters.

I tried to fix it, but found that the fix would be non-trivial. The problem is caused by using the built-in PHP functions strtoupper() and strtolower(). For example:

laminas-validator/src/Hostname.php

Line 2037 in 3f5e678

if (! in_array(strtolower($this->tld), $this->validTlds)

These functions do not work correctly with multi-byte encodings. But for processing multi-byte encodings, the code uses wrappers in the \Laminas\Stdlib\StringWrapper namespace, which do not yet contain functions strtoupper() and strtolower().

Thus, a full fix will look like this:

• Add to \Laminas\Stdlib\StringWrapper\StringWrapperInterface methods strtoupper() and strtolower()
• Implement these in the concrete wrappers
• Release new version of https://github.com/laminas/laminas-stdlib
• Use the new methods to fix the problem in the current package

So this is a subject for discussion...
Maybe I misunderstood something.

[weierophinney]

Alternately, we could depend on symfony/polyfill-iconv and/or symfony/polyfill-mbstring, allowing us to use the relevant iconv and/or mbstring functions directly in the code. On systems that have the extension(s) installed, these are ignored, but otherwise, we'd still be able to rely on them, without needing to write our own wrappers.

[rugabarbo]

As I see it, a strategic decision is needed here, because problems with multi-byte strings will occur over and over again in a wide variety of components.

Possible options:

1. Develop further your own string wrappers in the package https://github.com/laminas/laminas-stdlib without using third-party components.
2. Develop further your own string wrappers in the package https://github.com/laminas/laminas-stdlib using Symfony components (under the hood).
3. Declare your own string wrappers as deprecated components and switch from them to using Symfony components directly.

[ghost]

pull: pass --signoff/--no-signoff to "git merge"

merge can take --signoff, but without pull passing --signoff down, it is inconvenient to use; allow 'pull' to take the option and pass it through.

[ghost]

mm: remove gup_flags FOLL_WRITE games from __get_user_pages()
This is an ancient bug that was actually attempted to be fixed once
(badly) by me eleven years ago in commit 4ceb5db9757a ("Fix
get_user_pages() race for write access") but that was then undone due to
problems on s390 by commit f33ea7f404e5 ("fix get_user_pages bug").

In the meantime, the s390 situation has long been fixed, and we can now
fix it by checking the pte_dirty() bit properly (and do it better). The
s390 dirty bit was implemented in abf09bed3cce ("s390/mm: implement
software dirty bits") which made it into v3.9. Earlier kernels will
have to look at the page state itself.

Also, the VM has become more scalable, and what used a purely
theoretical race back then has become easier to trigger.

To fix it, we introduce a new internal FOLL_COW flag to mark the "yes,
we already did a COW" rather than play racy games with FOLL_WRITE that
is very fundamental, and then use the pte dirty flag to validate that
the FOLL_COW flag is still valid.

[rugabarbo]

@Rahul-coder01 It looks like you sent these messages here by mistake

[glensc]

@rugabarbo perhaps you assumed the polyfills that @weierophinney mentioned are Symfony components, they are not, they are absolutely framework agnostic, they are just composer packages under Symfony namespace:

• https://github.com/symfony/polyfill#symfony-polyfill

but if you meant really components, you mean string component? then it would require to use Symfony 5.0 which in turn requires PHP 7.2:

• https://symfony.com/blog/new-in-symfony-5-0-string-component
• https://github.com/symfony/string
• https://github.com/symfony/string/blob/v5.0.0/composer.json#L19

but this project is yet at PHP 7.1:

• https://github.com/laminas/laminas-validator/blob/2.13.4/composer.json#L32

I personally would just go using polyfills, which can be replaced with real extensions using composer "replace" key:

• https://github.com/eventum/eventum/blob/v3.9.3/composer.json#L96-L99

[weierophinney]

I've pushed changes that demonstrate an attempt at fixing this using symfony/polyfill-mbstring. I installed that package, and then updated all string-related functions that have mbstring equivalents to the mbstring versions. When I was done, I noted the following:

• Your first new test case passed.
• The second new test case failed.
• The lower-case version of the second case, which already existed, also failed.

Out of curiousity, I tried the same experiment using symfony/polyfill-iconv. However, in that scenario, the tests you've created continued to fail entirely.

I'm not quite sure where to go with this at this point, but I've left my symfony/polfill-mbstring experiment in here for you to work with. You'll need to fetch and rebase your branch.

[weierophinney]

I've rebased this off current 2.14.x sources so that it can pick up our new CI. However, the tests as presented are still failing and it looks like it's likely due to the letter casing. I attempted to change the regexp to add the u flag, but that made no difference, so I did not push that change here.

At this point, I need somebody with experience in i18n casing issues to assist.

[Ocramius]

IMO to be declared should be ext-mbstring, while we can decide to use symfony/polyfill-mbstring in require-dev