Beware of p-filter!

[gunters63]

I tried to diagnose the underlying problem of issue 3185 and found out that the culprit was an external npm package p-filter respective a quite dangerous default behavior.

The problem is that the call pFilter(input, filterer, [options]) by default runs all input promises in parallel with no restriction of concurrency. For the mentioned issue above this leads to an too many files open exception on windows with the pnpm store status call when you have more than about 500 packages in your global store on a windows platform. You can pass an additional options object to the call and limit the amount of concurrently excecuted promises. This is something you should always do in my opinion. This fixes the mentioned issue.

I found about half of a dozen other uses of p-filter throughout the project. I suggest to change all calls of pFilter anywhere in the project to add an options object and limit the concurrency to a reasonable amount (I think 16 is a good value).

[zkochan]

It is not necessarily a problem in every case, only if many files are being opened for reading as in #3185. I assume in most cases it is not an issue because we have some tests that install thousands of packages on Windows and the test is passing.

[gunters63]

I still think its dangerous, even with tests in place.
Its not only the open file handles. If the number of concurrent promises is unlimited, any code which allocates system resources (memory included) inside a filter promise (if they are not freed until the pFilter call is finished) is a potential problem.
Tests may not gonna find the problem because:

• they are missing (there seems to be no tests for pnpm store status)
• The amount of available system resources is heavily dependent on OS-version, Node-version, amount of memory and so on. Tests may pass on one machine and errors still may occur on others.