Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In Express 5, how to make a query param parsed as an array when it is an array of string of one element ? #6207

Open
SophieDel opened this issue Dec 5, 2024 · 5 comments
Labels

Comments

@SophieDel
Copy link

SophieDel commented Dec 5, 2024

Hello everybody,

I am currently migrating my app from Express 4.17.21 to Express 5. I am using Node 22.9. The only problem I have is for parsing the query params when it is array of string of 1 element.

Here is an example of the url :

http://localhost/api/myApi/myBaseUrl?myQueryParam=firstElement

The query param firstElement is not recognized as a an array, and it was the case in Express 4.x.x.

This problem can be fixed by this code (found https://github.com/cdimascio/express-openapi-validator/issues/599 thanks to @djMax)

app.use((req, res, next) => {
    // Express 5 re-parses the query string every time. This causes problems with
    // various libraries, namely the express OpenAPI parser. So we "freeze it" in place
    // here, which runs right before the routing stuff does.
    const { query } = req;
    if (query) {
      Object.defineProperty(req, 'query', {
        value: query,
      });
    }
  next();
});

But this is not very clean.

I know if I use the "extended" query parser (app.set("query parser", "extended")), and use the url

http://localhost/api/myApi/myBaseUrl?myQueryParam[]=firstElement

the problem can be fixed but for reasons - such as our APIs are used by consumers - I cannot use this solution.

So I wonder whether there is a "clean" way for parsing as Express 4 does...?

Thank you !

@wesleytodd
Copy link
Member

First, just to clear up any confusion, it does not parse the url every time: https://github.com/pillarjs/parseurl/blob/master/index.js#L43

Second, seems like this lib is breaking some assumptions because of this misunderstanding, but I will not comment more on that other than to say this seems like a bad idea on their part.

But this doesn't appear to have anything to do with the question you asked, which is about the breaking changes from qs. You can check out the changelogs there: https://github.com/ljharb/qs

The query string parser would not treat this as an array because it has no indication of it being an array, so the best solution if you always want that to be an array is to check and make it an array if it is not:

if (req.query.myQueryParam && !Array.isArray(req.query.myQueryParam)) {
  req.query.myQueryParam = [req.query.myQueryParam];
|

@kerlos-alfy
Copy link

Migrating from Express 4 to Express 5 introduced some changes to how query parameters are parsed, particularly with regards to arrays and single-element arrays. This behavior stems from the underlying querystring vs qs modules used for parsing.

Why the Issue Occurs

Express 4 uses qs by default, which is more robust and recognizes ?param=value as param: ['value'] when a consumer expects an array. However, Express 5 defaults to querystring, which does not handle this the same way.

Solutions

Here are clean and practical solutions for resolving this issue:


1. Use the qs Module Explicitly

You can replicate Express 4 behavior by setting up a custom query parser using the qs library.

Install qs:

npm install qs

Update your application:

const qs = require('qs');

// Set a custom query parser
app.set('query parser', str => qs.parse(str));

This makes the query parser behave like it did in Express 4.


2. Restore the "Extended" Parser

The extended parser uses qs under the hood. You can explicitly set it up without requiring clients to change their URL syntax.

app.set('query parser', 'extended');

This re-enables the extended query parsing without forcing clients to use [] for array elements.


3. Manually Normalize Query Parameters

If you need fine-grained control over how query parameters are handled, you can write middleware to normalize query parameters into arrays.

app.use((req, res, next) => {
  for (const [key, value] of Object.entries(req.query)) {
    if (!Array.isArray(value)) {
      req.query[key] = [value];
    }
  }
  next();
});

This ensures all query parameters are treated as arrays, even if they are single values.


Recommendation

The cleanest and most backward-compatible solution is Option 1 (using qs). It closely mimics Express 4 behavior and avoids unexpected surprises for consumers of your API.

Additional Notes

  • Consider documenting the behavior change for your API consumers if it's relevant.
  • Test your API thoroughly after the migration, especially with complex query strings.

@SophieDel
Copy link
Author

Hi @kerlos-alfy,

Thank you for your solutions ! However they don't work... Before asking my question I had tried the first one already, and the second one does not work. I didn't try the last one since I don't want that all the query params to be treaded as array.

I think it is because the problem is not the query parser, the problem is about the parameter "query" of the object "request" - which is actually not an object param in Express 5, but a getter. And this getter does not only "get" the param, it parses the query from the url and makes some transformation - and I think this is why I have the problem.

How the query param is returned in Express 4

req.param = function param(name, defaultValue) {
  var params = this.params || {};
  var body = this.body || {};
  var query = this.query || {};

  var args = arguments.length === 1
    ? 'name'
    : 'name, default';
  deprecate('req.param(' + args + '): Use req.params, req.body, or req.query instead');

  if (null != params[name] && params.hasOwnProperty(name)) return params[name];
  if (null != body[name]) return body[name];
  if (null != query[name]) return query[name];

  return defaultValue;
};

How the query param is returned in Express 5

defineGetter(req, 'query', function query(){
  var queryparse = this.app.get('query parser fn');

  if (!queryparse) {
    // parsing is disabled
    return Object.create(null);
  }

  var querystring = parse(this).query;

  return queryparse(querystring);
});

Thank you for your answer !

@SophieDel
Copy link
Author

Hello @wesleytodd,

Thank you for your answer ! I don't want my query params to be always treated as array... So maybe there is no solution to my problem...

Thank you !

@Fuzzyma
Copy link

Fuzzyma commented Dec 8, 2024

@kerlos-alfy does that mean, that qs is always installed even tho its not used by default? I still see it listed in the package.json and it comes with so much baggage: https://npmgraph.js.org/?q=qs

// EDIT: realized that the answer looks like chatgpt... so nvm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

4 participants