Only distribute bundled web components

[pwolfert]

Background

After the last web-components RFC, we built out and published an alternate library of web components, and we now have at least one team using them in production. However, as more teams have started to adopt web components, we've seen flaws in our NPM distribution channel for web components that make it difficult for certain projects to adopt. We have an opportunity to streamline the way our web are components imported into JavaScript projects. It would improve the developer experience for our users and reduce edge cases that we have to support.

Problem

An industry-standard pattern among JavaScript projects is to define dependencies through a package manager (like npm) and have a compiler/bundler analyze the project code to find all dependencies and sub-dependencies and include them in a final JavaScript bundle that has all the code needed to run your frontend application in a client's browser. While modern build tools can make this process seem simple and easy, it is a complex and evolving pattern that requires cooperation and standardization between many vendors—many independent authors of open source libraries that make up the toolchain and application code.

The specific problem we've run into here is that we must transform code in our dependencies to make them compatible with our web components. While that's perfectly reasonable to do in our own project, it becomes problematic when we expect our users' compilers to be responsible for those same transformations. As a concrete example, we use the react-day-picker library in our SingleInputDateField component, which contains the following import:

import { jsx, jsxs, Fragment } from 'react/jsx-runtime';

Because our web components use Preact instead of React, we must alias this package import so it reads like this:

import { jsx, jsxs, Fragment } from 'preact/compat/jsx-runtime';

Currently if a downstream project tries to import our web components through npm in the way we've prescribed, their import will look something like this

import '@cmsgov/design-system/web-components';

which would begin the following chain of imports in the projects' bundler:

@cmsgov/design-system/web-components
└── node_modules/@cmsgov/design-system/dist/preact-components/esm/web-components/index.js
    ├── (other components)
    └── ./ds-date-field
        ├── preact/compat/jsx-runtime
        ├── (other modules)
        └── ../../DateField
            ├── (other modules)
            └── react-day-picker
                └── node_modules/react-day-picker/dist/index.esm.js
                    ├── (other modules)
                    └── react/jsx-runtime

In a downstream project, we have control over what's in node_modules/@cmsgov/design-system, but we do not have control over what's in node_modules/react-day-picker/dist/index.esm.js, which means we're powerless to change its import of react/jsx-runtime or other React-specific imports unless we ask our users to configure their build systems to achieve that end. Not only would this configuration be inconvenient, but we can't possibly write enough documentation to handle all scenarios, so the burden of battling build systems would rest on our users.

The actual issue we can observe in a project that is trying to import web components from our npm module is that their compiler will be unable to resolve those React imports, and even if they did they'd be wrong and our components wouldn't work.

Solution

We don't have this problem in our web-component CDN bundles. In our CDN bundles, we provide everything needed to register our web components in a client's browser and start using them in the page. Our own build system resolves dependencies in the right way and spits out pre-packaged bundles that are ready to go. What if we used the same strategy we use for our CDN for our NPM packages? While we do currently distribute the same bundles through the NPM package that we do through the CDN, they're not front-and-center, and we're also distributing the un-bundled modules described in the Problem section, and this has become a foot-gun. What I'm proposing is that we only distribute the bundled version of our web components.

What were the advantages of including the un-bundled modules before? There are two main benefits, which are common to most modern JavaScript projects, but which I will illustrate are not advantages over the bundled web components:

1. Selective component imports
2. Dependency deduplication and optimization

1. Selective component imports

If a team is only using a handful of our components, they don't want to load our entire component library into their clients' browsers. That would be wasteful and cause slower page load times. When teams use our React components, they can specify which components they want to import, and their build systems will trim the rest out (through a process called tree-shaking in JavaScript contexts). However, web components rely on side effects by nature, which means that our web component modules can't be tree-shaken and receive this kind of optimization. Luckily, we've already solved this problem in our bundles by offering à-la-carte web-component bundles. We've got bundles for each of our components, so you can just import the bundles for the components you want.

2. Dependency deduplication and optimization

Another thing that modern JavaScript build systems can do is deduplicate common dependencies. Say our library and the application both require the React framework. The application's build system wouldn't include two copies. It would only include one. The theoretical benefit, then, of not bundling all our code and allowing the application team's build system to aggregate everything is that it can look for common dependencies and deduplicate. However, when we reason about what kinds of projects will consume the web components, the potential for overlap is actually fairly low.

Here are our largest dependencies and notes about their potential for overlap:

• react - only used for React projects
  • We don't need to worry about overlap for this because React projects should use our React components, not our web components
• preact - used for Preact projects and web components
  • Similarly, there won't be overlap with a downstream project unless that project uses Preact, in which case they should use our Preact components, not our web components
• react-day-picker - a React component
  • Same story. If they're using this directly, they have a React project and can use the React components.
• date-fns - used for react-day-picker
  • This one has some potential for overlap, but it's already on our radar to address the amount of space this takes up in our library in general
• lodash - generic JavaScript utilities
  • Same story as date-fns

While we'd like to be able to take advantage of every opportunity to optimize, there is a trade-off to be made for ease of use for application teams as well as our own maintenance cost.

Implementation

The simplest change would be to modify our package.json files to turn this entry

"./web-components": {
  "types": "./dist/react-components/types/web-components/index.d.ts",
  "import": "./dist/preact-components/esm/web-components/index.js",
  "require": "./dist/preact-components/cjs/web-components/index.js"
},

into something like this

"./web-components": {
  "types": "./dist/react-components/types/web-components/index.d.ts",
  "import": "./dist/web-components/bundle/all.js",
  "require": "./dist/web-components/bundle/all.js"
},
"./web-components/*": "./dist/web-components/bundle/*",

Optional idea

Another change to explore could be to move the src/components/web-components folder so that it's parallel with the existing components folder so what now looks like this

src/
└── components/
    └── web-components/

looks like this

src/
├── react-components/
└── web-components/

That would clean out the copy of the web-components folder that exists in the compiled dist/react-components and dist/preact-components folders so it's not tempting to our users to try to import from there. The web-components would need to be fed through the TypeScript compiler separately, though, to generate the type definition files. It's unclear to me at the moment whether this idea would be a help or a hindrance, but it might be worth doing some quick discovery on.

Conclusion

We aren't gaining much by offering an un-bundled version of our web components. Those modules can't be tree-shaken by downstream build systems and optimized. It would also require those downstream build systems to be configured to modify the abstract syntax tree to alias certain dependency imports so we can compile all React into Preact. The bundles we already have are easy to use and and are ready to eat, like a pizza from Domino's vs Papa Murphy's. Let's scrap our unbundled web components from dist and double-down on the bundles.

[jack-ryan-nava-pbc]

@pwolfert This is an excellent write up. Are the only changes needed to implement this in the package.json or would we have to augment our web component build process as well?

[pwolfert]

Thanks! I believe the only necessary changes would be in the package.json files for each design system package, but there's that optional idea of updating some folder structures and the build code.

[pwolfert]

Even though it's such a small effort to change, I wanted to document the thought process behind it and the decision made so it can be referenced in the future. This could possibly be considered a breaking change.