[Feature]: Improved type safety for component testing

[debel27]

🚀 Feature Request

Playwright v1.62.0 introduces a story/gallery workflow for component testing. It allows tests to mount story components into the gallery webpage, using the mount function. The function is typed as follows:

export interface PlaywrightTestArgs {
  mount: <Story = Record<string, any>>(
    storyId: string, 
    props?: StoryProps<Story>
  ) => Promise<Locator & { update(props?: StoryProps<Story>): Promise<void>, unmount(): Promise<void> }>;
}

The type definition of mount has a few caveats:

• The storyId parameter is typed as string, which prevents auto-completion and allows for spelling mistakes
• The Story type parameter is not constrained to match the props of the function identified by storyId

To address these caveats, Playwright can follow an approach similar to TanStack Router's file-based routing. They enabled type safety for their router by using a file watcher that monitors route files and generates types automatically.

Example

Consider the following test (Playwright v1.62.0):

import { test, expect } from '@playwright/test';
import type { WithTitle } from '../../src/components/Button.story';

test('renders primary button', async ({ mount }) => {
  const component = await mount<typeof WithTitle>('Button/WithTitle', { children: 'Hello' });
  await expect(component.getByRole('button')).toHaveText('Hello');
});

The test has a few issues:

• TypeScript does not emit an error if the 'Button/WithTitle' story does not exist
• The developer has to import WithTitle to enable type safety on props, which is boilerplate
• TypeScript does not enforce WithTitle to be to be consistent with the story identified by 'Button/WithTitle'

Here is how it can be improved.

Playwright code

The playwright package will expose a ComponentTestingConfig interface. Userland code will use declaration merging on this interface, to register the stories and their associated prop types.

export type DefaultStoryProps = Record<string, unknown>;
export type DefaultStoriesRegistry = { [storyId: string]: DefaultStoryProps };

export interface ComponentTestingConfig {}

export type ResolvedComponentTestingConfig = MergeObjects<
  { storiesRegistry: DefaultStoriesRegistry }, 
  ComponentTestingConfig
>;
export type StoriesRegistry = ResolvedComponentTestingConfig['storiesRegistry'];

The signature of the mount function can then be updated to leverage StoriesRegistry and enable type safety:

export interface PlaywrightTestArgs {
  mount: <Id extends StoriesRegistry>(
    storyId: Id, 
    props?: StoriesRegistry[Id]
  ) => Promise<Locator & { update(props?: StoriesRegistry[Id]): Promise<void>, unmount(): Promise<void> }>;
}

This example is simplified. The mount function should make the props argument required if the Story has required props. It could also accept a new type parameter, to allow developers to override the prop types as a workaround. Etc.

Userland code

In userland, a file watcher will look for story files and generate the StoriesRegistry type. The watcher should be integrated with the bundler, as done in TanStack router. It will produce the following file:

playwright/storiesRegistry.gen.ts (auto-generated)

// This file is automatically generated from story files. Any changes will be will be overwritten.
export type StoriesRegistry = {
  'Button/WithTitle': { children: string }
};

The developer can then use declaration merging to update the ComponentTestingConfig interface with that generated registry type:

tests/playwright-ct.d.ts

import type { StoriesRegistry } from "../playwright/storiesRegistry.gen.js";

declare module "playwright" {
  interface ComponentTestingConfig {
    storiesRegistry: StoriesRegistry;
  }
}

Finally, the tests can be written with type-safety:

tests/button-with-title.ts

import { test, expect } from '@playwright/test';

test('renders primary button', async ({ mount }) => {
  // Type-safe: no spelling mistakes in the story ID, props types are naturally derived from it
  const component = await mount('Button/WithTitle', { children: 'Hello' });
  await expect(component.getByRole('button')).toHaveText('Hello');
});

Motivation

As illustrated, using a file watcher generate a "story registry" type would provide useful guardrails and reduce boilerplate in tests. It however implies a breaking change for the signature of mount.

The implementation of the watcher should likely be out of Playwright's responsibility. Playwright should just provide the primitives to support type safety on mount/update, and let userland deal with providing the correct "story registry", which includes implementing the watcher. And since playwright/gallery/main.tsx defines how stories are looked up, it is easy to avoid mismatches.

Arguably, this proposal can be implemented entirely outside of Playwright's codebase, as developers can use declaration merging on PlaywrightTestArgs to override the signature of mount/update. I am not comfortable with that approach, because the PlaywrightTestArgs interface itself is likely not considered as part of the public API, in the sense that refactoring it might not be communicated in the release notes. Having a dedicated StoriesRegistry interface can help in exposing a clear public API, that is meant to be augmented with declaration merging.