Add support for ES Module and CommonJS output, and <script type="module">

[devongovett]

Related: #3011 #1168.

This is a refactor of the scope hoisting compiler to support multiple module output formats, including ES modules, CommonJS, and browser script output. This allows much improved support for building libraries with Parcel, and also enables native browser ES module output with <script type="module">. 🎉

The babel transformer has been adjusted to take ES module targets into account when building, and automatically use a higher browser target for ES module supporting browsers when that target is seen. Also our loader infrastructure will automatically include an import() polyfill when needed, as not all esmodule supporting browsers support dynamic imports. But, if you only target dynamic import supporting browsers, then the polyfill is not included. Without any loader runtime code or prelude, bundle sizes should be significantly smaller with native esmodules.

In addition, the behavior of the TargetResolver has been changed to treat the package.json main, module, and browser fields as library targets rather than app targets. This means that you will get CommonJS or ES Module output by default with those fields instead of browser script output. Also, loaders for async resources (e.g. css) will be excluded and native imports will be left in place instead, which can be picked up by another bundler later.

This required introducing two new fields to the Environment object: outputFormat, and isLibrary. The TargetResolver sets these fields automatically based on package.json information, but they can also be configured manually. In addition, outputFormat is automatically set to esmodule by the HTML transformer when a <script type="module"> tag is seen, and turned off when <script nomodule> is seen. This makes it really easy to create multiple targets if you use an HTML entry point.

Limitations: these new output formats are only supported with scope hoisting enabled, so won't work in development mode. Features like HMR just aren't possible with ES modules, and we also don't currently track enough import/export metadata without the scope hoisting compiler. This is probably fine though as this is meant as an optimization, and will result in faster compilation during development (only need to compile the nomodule version).

Other changes:

• Quite a few scope hoisting bugs are fixed in this PR, including working with TypeScript, and dynamic imports.
• The includeNodeModules flag now supports taking either a boolean or a whitelist of module names to include for more granular control.
• includeNodeModules is now processed by the resolver instead of in the transformer. A dependency is always created now, but sometimes isn't resolved to anything. This allows us to keep the metadata about what symbols were imported etc., but not cause any transforming to occur. An isExcluded flag can be set by the resolver to cause the ResolverRunner to return null instead of throwing an error.
• Babel preset env's useBuiltins: 'usage' option has been reverted to useBuiltins: 'entry'. Usage is far too aggressive about including polyfills, and seemed to result in much larger bundle sizes unnecessarily. If you need polyfills, you can include them yourself.
• The PostCSS transformer now merges dependencies into CSS module objects.

Future work:

• Compile a single script into multiple with module/nomodule automatically
• Better support for relative URLs in loaders instead of making everything absolute.
• Include regenerator polyfill automatically (possibly babel-transform-runtime?)
• External module support for browser script targets?
• Module prefetching
• More tests

[mischnic]

I don't think the current design for includeNodeModules is very useful, if you write a React library, you might want to bundle everything except react. If you develop a vscode plugin you want to bundle everything except vscode (which is somehow injected dynamically at runtime) - I think aws does something similar.

What usecases did you have in mind when chosing this syntax?

---

"outputFormat": "esModules", ❌
"outputFormat": "esmodules", ❌
"outputFormat": "esmodule",
🙈

[devongovett]

I don't think the current design for includeNodeModules is very useful, if you write a React library, you might want to bundle everything except react.

Yeah this is the opposite usecase: when building a library you usually don't want to include any node_modules, but sometimes want to include one or two, for example, modules that are in the same monorepo. I have a repo which includes a bunch of react components that will be published as libraries, and I want to include the CSS along with it parts of which come from another package. We will eventually support a blacklist as well for externals, which are more useful when building applications.

[mischnic]

We can do this in a followup, but I found a case that could be better optimized:

<script src="./main.js"></script>

import("./lib.js").then(v => {
	console.log(v);
});

Browserslist: Chrome 70 (so dynamic import is supported)


Using dynamic import works also with the global output format (of main.js in this case), currently both are global and therefore the dynamic import polyfill is used.
The lib.js bundle should be esmodule and get imported with the native import()

[devongovett]

Oh import() works even without <script type="module">?

[mischnic]

Oh import() works even without <script type="module">?

Yes!

[mischnic]

This is really an URL rather than a path (because it returns a URL with forward slashes also on Windows).

[height]

Mark task as "Done".

Tip: use "Close T-74" or "Fix T-74" next time.

[devongovett]

Yes!

Awesome, had no idea. Gonna leave that as a followup task.

[height]

This pull request has been linked to 1 task:

• T-74 Inline source maps (unlink task)

💡Tip: Add "Close T-74" to the pull request title or description, to a commit message, or in a comment to mark this task as "Done" when the pull request is merged.