RFC: Performance Proposal

[gdborton]

I've been running lots of profiles to find the bottlenecks in our webpack build. Probably the biggest bottleneck right now is resolving requests.

To highlight this, I've created a simple test case for our build. Basically I intercepted all calls to Resolver.resolve during a build, and wrote the parameters passed to disk. Then I created a script that loaded the list of requests and used enhanced-resolve to resolve each of the requests, and also used resolve-from.

The results are pretty crazy, for 14850 requests:

• enhanced-resolve: 11601 ms
• resolve-from: 484 ms

Note that I didn't do much investigation into where this time is going, but I think that this project (and webpack) would benefit greatly from simplifying the architecture.

I'd prefer to see this library only respect an extensions option, and then stuff like custom description files (like bower.json or component.json) could be added via a loader (whose results could be cached). We could add support for browser fields and aliases the same way (cacheable loader).

[gdborton]

@sokra @mikesherov @TheLarkInn I'd love to unlock the performance potential here, but recognize that my proposed changes would mean a breaking change in both enhanced-resolve and webpack.

This is what I envision the config would look like after this change.

module.exports = {
  entry: 'somefile.js',
  output: {
    path: path.resolve('./dist'),
    filename: 'outputname.js',
  },
  module: {
    use: [
      // These loaders would transform the require statements, so that
      // enhanced-resolve only sees what is intended to be imported.
      {
        loader: 'bower-library-loader',
        include: ['./src']
      },
      {
        loader: 'alias-loader',
        aliases: {
          jquery: path.resolve('./bower_components/jquery'),
          path.resolve('./src/someFile'): path.resolve('./src/someOtherFile'),
        },
      },
    ],
  },
}

[gdborton]

@sokra poke.

[mikesherov]

@gdborton, I think this is super interesting, but I was wondering if a different approach can work, where we introduce an option to use native resolution. That way, for most modules, you'd get the fast path of native resolution, with fallback to the other important resolvers (extensions comes to mind as an important one for React / Vue / Typescript) and other fast resolvers like the unsafeCachePlugin.

Thoughts?

[gdborton]

I'm guessing that would work for most people outside of aliasing and those still using bower. Native resolving works for any file so long as you provide the extension in your import.

[mikesherov]

Right but the key would fallback to slow manual resolution if fast resolution fails to find a module. So best of both worlds. You just wouldn't be able to do things like prioritize .vue over .js, etc.

[mikesherov]

so aliases are interesting in that some folks use them to override stuff that would otherwise still resolve.

[mikesherov]

So there's a tradeoff for sure. But I guess as a first step, you can inspect passed in options, and see if they match nodes native behavior, and if so, just go straight to native resolution.

[gdborton]

Do you think that this should be a change in webpack without looking at enhanced-resolve at all?

Something like...

try {
  // resolve-from
  resolveFrom(/* options */)
} catch(e) {
  // enhanced-resolve
  Resolver.resolve(/* options */);
}

Or would this take place in enhanced-resolve and would be transparent to webpack?

I'm guessing the latter...

[mikesherov]

I'm guessing latter, especially since it's a resolve option, which are automatically passed to a resolver factory.

[blowery]

We're doing some profiling of the build for https://github.com/Automattic/wp-calypso and finding similar results; we're spending a lot of time resolving requests. However, we also use aliases heavily...

[alexander-akait]

Let's close a lot of improvements were done and we are faster than require.resolve (used in resolve-from), but not faster import.meta.resolve, we still working on perf improvements