Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create npm workspace for E2E tests #10119

Open
wants to merge 29 commits into
base: develop
Choose a base branch
from

Conversation

benbowler
Copy link
Collaborator

@benbowler benbowler commented Jan 27, 2025

Summary

Addresses issue:

Relevant technical choices

puppeteer and puppeteer-core are used in storybook tests currently, we should remove these from the main package.json as part of #10091.

I created a script install-global-npm to help developers and the GH actions to install npm v7.

PR Author Checklist

  • My code is tested and passes existing unit tests.
  • My code has an appropriate set of unit tests which all pass.
  • My code is backward-compatible with WordPress 5.2 and PHP 7.4.
  • My code follows the WordPress coding standards.
  • My code has proper inline documentation.
  • I have added a QA Brief on the issue linked above.
  • I have signed the Contributor License Agreement (see https://cla.developers.google.com/).

Do not alter or remove anything below. The following sections will be managed by moderators only.

Code Reviewer Checklist

  • Run the code.
  • Ensure the acceptance criteria are satisfied.
  • Reassess the implementation with the IB.
  • Ensure no unrelated changes are included.
  • Ensure CI checks pass.
  • Check Storybook where applicable.
  • Ensure there is a QA Brief.
  • Ensure there are no unexpected significant changes to file sizes.

Merge Reviewer Checklist

  • Ensure the PR has the correct target branch.
  • Double-check that the PR is okay to be merged.
  • Ensure the corresponding issue has a ZenHub release assigned.
  • Add a changelog message to the issue.

Copy link

github-actions bot commented Jan 27, 2025

Build files for 2d318a0 are ready:

@benbowler
Copy link
Collaborator Author

Note the one failing E2E test will be fixed in #10111

@benbowler benbowler changed the title Infrastructure/10013-npm-workspaces-e2e Create npm workspace for E2E tests Jan 28, 2025
Copy link
Collaborator

@eugene-manuilov eugene-manuilov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @benbowler. Nice work, but we need to move more dependencies. Please, see my comments.

tests/e2e/package.json Outdated Show resolved Hide resolved
tests/e2e/package.json Outdated Show resolved Hide resolved
tests/e2e/package.json Outdated Show resolved Hide resolved
tests/e2e/package.json Outdated Show resolved Hide resolved
@benbowler
Copy link
Collaborator Author

@aaemnnosttv

I found that the scripts workspace scripts appeared to be running with mixed npm versions and found npx npm -v -w tests/e2e returned v6 so I installed npm v7 on the dev dependencies of the e2e tests too which fixes this.

However, one thing to note is that the workspace wants the root package to be upgraded as well so npx npm ci -w tests/e2e will try to update the root package-lock with the warning The package-lock.json file was created with an old version of npm (here meaning the root package-lock). I assume this is because npm workspaces resolves packages across all package.json files in the project so having package-lock files generated by two versions of npm (one of which doesn't support workspaces) is causing issues. In this case npx npm ci -w tests/e2e fails with sh: patch-package: command not found as it's trying to run the root postinstall script.


To me it seems that trying to run multiple npm versions is causing more problems than it's worth. Back in my first implementation I sent to review with globally installed npm v7, I had the workspaces and E2E tests working. With #6026 in EB, what do you think about requiring npm v7 globally in the interim, all devs need to do is run npm install -g [email protected] .

@benbowler
Copy link
Collaborator Author

I'm open to suggestions if you can get the dev dependency method working reliably, allowing install commands and scripts to run reliably at the root and package level.

@benbowler
Copy link
Collaborator Author

benbowler commented Feb 4, 2025

However, one thing to note is that the workspace wants the root package to be upgraded as well so npx npm ci -w tests/e2e will try to update the root package-lock with the warning The package-lock.json file was created with an old version of npm (here meaning the root package-lock).

One possible way to avoid this is by updating packages in the workspace like this for now: cd tests/e2e && npx npm ci

@@ -30,11 +30,10 @@
"test:js": "cross-env BABEL_ENV=test NODE_ENV=test jest --config=tests/js/jest.config.js --passWithNoTests",
"test:js:watch": "npm run test:js -- --watch",
"test:storybook": "npm run test:js -- --testMatch '<rootDir>/.storybook/*.test.js'",
"pretest:e2e": "./bin/local-env/env-check.sh && ./tests/e2e/bin/test-docker-connectivity",
Copy link
Collaborator Author

@benbowler benbowler Feb 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aaemnnosttv suggested removing the pretest command as it's not needed to be run from root.

@benbowler
Copy link
Collaborator Author

Removed pre e2e script from root as these are run by the scripts.

@aaemnnosttv I tried npm 9 then 8 but these failed to install packages the following error so I've stuck with npm v7 and this can upgrade along with future node.js updates.

197 error code ERESOLVE
198 verbose stack TypeError: Cannot read property 'length' of null
198 verbose stack     at explainDependents (/Users/benbowler/.nvm/versions/node/v14.21.3/lib/node_modules/npm/lib/utils/explain-dep.js:94:22)
198 verbose stack     at explainNode (/Users/benbowler/.nvm/versions/node/v14.21.3/lib/node_modules/npm/lib/utils/explain-dep.js:5:3)
198 verbose stack     at explain (/Users/benbowler/.nvm/versions/node/v14.21.3/lib/node_modules/npm/lib/utils/explain-eresolve.js:25:26)
198 verbose stack     at report (/Users/benbowler/.nvm/versions/node/v14.21.3/lib/node_modules/npm/lib/utils/explain-eresolve.js:63:21)
198 verbose stack     at errorMessage (/Users/benbowler/.nvm/versions/node/v14.21.3/lib/node_modules/npm/lib/utils/error-message.js:41:37)
198 verbose stack     at exitHandler (/Users/benbowler/.nvm/versions/node/v14.21.3/lib/node_modules/npm/lib/utils/exit-handler.js:181:53)
198 verbose stack     at module.exports (/Users/benbowler/.nvm/versions/node/v14.21.3/lib/node_modules/npm/lib/cli-entry.js:72:12)
199 verbose cwd /Users/benbowler/Code/sitekit.10uplabs.com/app/public/wp-content/plugins/google-site-kit

Later npm versions also have different module resolution rules which causes an issue with these now deprecated modules. If we upgrade to later npm versions we will need to resolve this issue so as not to run all install/ci commands with --legacy-peer-deps.

npm ERR! code ERESOLVE
npm ERR! ERESOLVE could not resolve
npm ERR! 
npm ERR! While resolving: @storybook/[email protected]
npm ERR! Found: @storybook/[email protected]
npm ERR! node_modules/@storybook/addon-storyshots
npm ERR!   dev @storybook/addon-storyshots@"^6.5.16" from the root project
npm ERR! 
npm ERR! Could not resolve dependency:
npm ERR! peer @storybook/addon-storyshots@"6.3.6" from @storybook/[email protected]
npm ERR! node_modules/@storybook/addon-storyshots-puppeteer
npm ERR!   dev @storybook/addon-storyshots-puppeteer@"^6.3.6" from the root project
npm ERR! 
npm ERR! Conflicting peer dependency: @storybook/[email protected]
npm ERR! node_modules/@storybook/addon-storyshots
npm ERR!   peer @storybook/addon-storyshots@"6.3.6" from @storybook/[email protected]
npm ERR!   node_modules/@storybook/addon-storyshots-puppeteer
npm ERR!     dev @storybook/addon-storyshots-puppeteer@"^6.3.6" from the root project
npm ERR! 
npm ERR! Fix the upstream dependency conflict, or retry
npm ERR! this command with --force or --legacy-peer-deps
npm ERR! to accept an incorrect (and potentially broken) dependency resolution.
npm ERR! 
npm ERR! 
npm ERR! For a full report see:
npm ERR! /Users/benbowler/.npm/_logs/2025-02-04T17_39_00_347Z-eresolve-report.txt

@benbowler
Copy link
Collaborator Author

@eugene-manuilov this is now running E2E tests locally and on GH actions, failing checks are existing known instabilities.

In terms of the CLA check it appears this joint commit was not verified which is causing the failure:

Screenshot 2025-02-05 at 12 48 46

"jest": "^26.6.3",
"jest-puppeteer": "^6.0.1",
"jsdom": "^16.5.0",
"puppeteer": "^10.4.0"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@benbowler, please, also remove this dep from the main package.json file. And seems like puppeteer-core is not used anywhere, so we can drop it as well.

"expect-puppeteer": "^4.4.0",
"jest": "^26.6.3",
"jest-puppeteer": "^6.0.1",
"jsdom": "^16.5.0",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same for jsdom, seems like we can remove it from the main package.json as well.

package.json Outdated
@@ -184,24 +184,18 @@
"@testing-library/jest-dom": "^5.16.5",
"@testing-library/react": "^10.4.9",
"@testing-library/react-hooks": "^3.3.0",
"@wordpress/babel-preset-default": "^4.17.0",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like removing this dependency caused issues in some of the workflows because they somehow depend on it. Probably because we use @wordpress/default in the babel config 🤔. Let's revert this and add this dependency in the main package.json as well.

@techanvil
Copy link
Collaborator

Hey @benbowler, as per the sync earlier I've taken a look to revisit the idea of running a dev-installed version of NPM.

I noticed there's a bit of a chicken and egg situation when trying to initially install NPM 7 with workspaces configured in package.json when the global version is NPM 6.

This can be resolved, here's a potential approach with a bootstrap-npm7 to do the initial npm install, thus cleanly creating the initial package-lock.json. It could use a modification to support running npm ci for a clean start with the package-lock.json in place: infrastructure/10013-npm-workspaces-e2e...infrastructure/10013-npm-workspaces-e2e-local-npm

However, one thing to note is that the workspace wants the root package to be upgraded as well so npx npm ci -w tests/e2e will try to update the root package-lock with the warning The package-lock.json file was created with an old version of npm (here meaning the root package-lock). I assume this is because npm workspaces resolves packages across all package.json files in the project so having package-lock files generated by two versions of npm (one of which doesn't support workspaces) is causing issues.

Hmm, there should only be a single package-lock.json, at the repo root. And, this of course implies we should not expect to have to support multiple versions of NPM within the repo.

In this case npx npm ci -w tests/e2e fails with sh: patch-package: command not found as it's trying to run the root postinstall script.

This case, however seems to be due to npm ci -w tests/e2e only installing the direct dependencies for the workspace. Even with NPM 7 installed globally, I'm seeing the same thing. Given that patch-package seems to have quite a few default patches, it might be prudent to make this a dependency for all workspaces (this would merit a double check, I'm basing this on a quick search for now).

patch-package search results
34 results - 5 files

package-lock.json:
     146  				"npm": "7.24.2",
     147: 				"patch-package": "^8.0.0",
     148  				"postcss": "^8.4.31",

   55125  		},
   55126: 		"node_modules/patch-package": {
   55127  			"version": "8.0.0",
   55128: 			"resolved": "https://registry.npmjs.org/patch-package/-/patch-package-8.0.0.tgz",
   55129  			"integrity": "sha512-da8BVIhzjtgScwDJ2TtKsfT5JFWz1hYoBl9rUQ1f38MC2HwnEIkK8VN3dKMKcP7P7bvvgzNDbfNHtx3MsQb5vA==",

   55148  			"bin": {
   55149: 				"patch-package": "index.js"
   55150  			},

   55155  		},
   55156: 		"node_modules/patch-package/node_modules/ansi-styles": {
   55157  			"version": "4.3.0",

   55170  		},
   55171: 		"node_modules/patch-package/node_modules/chalk": {
   55172  			"version": "4.1.2",

   55186  		},
   55187: 		"node_modules/patch-package/node_modules/ci-info": {
   55188  			"version": "3.9.0",

   55201  		},
   55202: 		"node_modules/patch-package/node_modules/color-convert": {
   55203  			"version": "2.0.1",

   55213  		},
   55214: 		"node_modules/patch-package/node_modules/color-name": {
   55215  			"version": "1.1.4",

   55219  		},
   55220: 		"node_modules/patch-package/node_modules/cross-spawn": {
   55221  			"version": "7.0.3",

   55233  		},
   55234: 		"node_modules/patch-package/node_modules/has-flag": {
   55235  			"version": "4.0.0",patch-package

   55242  		},
   55243: 		"node_modules/patch-package/node_modules/path-key": {
   55244  			"version": "3.1.1",

   55251  		},
   55252: 		"node_modules/patch-package/node_modules/rimraf": {
   55253  			"version": "2.7.1",

   55264  		},
   55265: 		"node_modules/patch-package/node_modules/semver": {
   55266  			"version": "7.6.2",

   55276  		},
   55277: 		"node_modules/patch-package/node_modules/shebang-command": {
   55278  			"version": "2.0.0",

   55288  		},
   55289: 		"node_modules/patch-package/node_modules/shebang-regex": {
   55290  			"version": "3.0.0",

   55297  		},
   55298: 		"node_modules/patch-package/node_modules/slash": {
   55299  			"version": "2.0.0",

   55306  		},
   55307: 		"node_modules/patch-package/node_modules/supports-color": {
   55308  			"version": "7.2.0",

   55318  		},
   55319: 		"node_modules/patch-package/node_modules/which": {
   55320  			"version": "2.0.2",

   55333  		},
   55334: 		"node_modules/patch-package/node_modules/yaml": {
   55335  			"version": "2.4.5",

  111172  		},
  111173: 		"patch-package": {
  111174  			"version": "8.0.0",
  111175: 			"resolved": "https://registry.npmjs.org/patch-package/-/patch-package-8.0.0.tgz",
  111176  			"integrity": "sha512-da8BVIhzjtgScwDJ2TtKsfT5JFWz1hYoBl9rUQ1f38MC2HwnEIkK8VN3dKMKcP7P7bvvgzNDbfNHtx3MsQb5vA==",

package.json:
   49  		"bootstrap-npm7": "node bin/bootstrap-npm7.js",
   50: 		"postinstall": "patch-package"
   51  	},

  229  		"npm": "7.24.2",
  230: 		"patch-package": "^8.0.0",
  231  		"postcss": "^8.4.31",

package.json.new:
   49  		"bootstrap-npm7": "node bin/bootstrap-npm7.js",
   50: 		"postinstall": "patch-package"
   51  	},

  229  		"npm": "7.24.2",
  230: 		"patch-package": "^8.0.0",
  231  		"postcss": "^8.4.31",

bin/bootstrap-npm7.js:
  16  
  17: // Create a wrapper script that handles patch-package
  18  const wrapperScript = `

  29      
  30:     // Run patch-package after successful npm operations that modify dependencies
  31      const command = process.argv[2];
  32      if (['install', 'i', 'ci', 'add', 'update', 'upgrade'].includes(command)) {
  33:         execSync('node_modules/.bin/patch-package', { stdio: 'inherit' });
  34      }

  53  
  54: 	console.log( 'npm7 wrapper installed. Use `npx npm7` to run npm commands with automatic patch-package handling.' );
  55  

Untitled-1:
  1: 		"postinstall": "node -e \"if(process.env.SKIP_PATCH !== 'true') require('child_process').execSync('patch-package')\"",
  2: 		"apply-patches": "patch-package"

It's worth noting that other script/command dependencies of the root package.json will also be missing when installing via npm ci -w <workspace> - the one I noticed was husky, which is admittedly unlikely to cause an issue in real-world usage as it's only used for commit hooks and it seems quite unlikely we'd want to use npm ci -w <workspace> outside of a CI context.

To me it seems that trying to run multiple npm versions is causing more problems than it's worth. Back in my first implementation I sent to review with globally installed npm v7, I had the workspaces and E2E tests working. With #6026 in EB, what do you think about requiring npm v7 globally in the interim, all devs need to do is run npm install -g [email protected] .

I can see it's more complicated than I had imagined, but at the same time it seems that some of the issues you've come across are not related to the local/global NPM question.

On balance I'm still likely to agree with the conclusion that simply installing the required version globally is the path of least resistance, but I'm interested to get your thoughts on my findings.

@benbowler benbowler force-pushed the infrastructure/10013-npm-workspaces-e2e branch from 067e649 to d1a1de7 Compare February 6, 2025 11:54
Copy link

github-actions bot commented Feb 6, 2025

Size Change: 0 B

Total Size: 2 MB

ℹ️ View Unchanged
Filename Size
./dist/assets/css/googlesitekit-admin-css-********************.min.css 62.5 kB
./dist/assets/css/googlesitekit-adminbar-css-********************.min.css 11.8 kB
./dist/assets/css/googlesitekit-authorize-application-css-********************.min.css 846 B
./dist/assets/css/googlesitekit-wp-dashboard-css-********************.min.css 8.48 kB
./dist/assets/js/33-********************.js 2.76 kB
./dist/assets/js/34-********************.js 2.25 kB
./dist/assets/js/35-********************.js 3.64 kB
./dist/assets/js/36-********************.js 936 B
./dist/assets/js/37-********************.js 892 B
./dist/assets/js/38-********************.js 1.61 kB
./dist/assets/js/39-********************.js 1.57 kB
./dist/assets/js/40-********************.js 1.61 kB
./dist/assets/js/41-********************.js 1.59 kB
./dist/assets/js/42-********************.js 1.83 kB
./dist/assets/js/43-********************.js 3.12 kB
./dist/assets/js/analytics-advanced-tracking-********************.js 901 B
./dist/assets/js/googlesitekit-activation-********************.js 24.1 kB
./dist/assets/js/googlesitekit-ad-blocking-recovery-********************.js 54.1 kB
./dist/assets/js/googlesitekit-adminbar-********************.js 35 kB
./dist/assets/js/googlesitekit-api-********************.js 10.1 kB
./dist/assets/js/googlesitekit-components-gm2-********************.js 6.41 kB
./dist/assets/js/googlesitekit-components-gm3-********************.js 10.1 kB
./dist/assets/js/googlesitekit-consent-mode-********************.js 25.6 kB
./dist/assets/js/googlesitekit-data-********************.js 2.38 kB
./dist/assets/js/googlesitekit-datastore-forms-********************.js 8.96 kB
./dist/assets/js/googlesitekit-datastore-location-********************.js 2.09 kB
./dist/assets/js/googlesitekit-datastore-site-********************.js 20.2 kB
./dist/assets/js/googlesitekit-datastore-ui-********************.js 10 kB
./dist/assets/js/googlesitekit-datastore-user-********************.js 28.2 kB
./dist/assets/js/googlesitekit-entity-dashboard-********************.js 82.8 kB
./dist/assets/js/googlesitekit-events-provider-contact-form-7-********************.js 646 B
./dist/assets/js/googlesitekit-events-provider-easy-digital-downloads-********************.js 624 B
./dist/assets/js/googlesitekit-events-provider-mailchimp-********************.js 630 B
./dist/assets/js/googlesitekit-events-provider-ninja-forms-********************.js 712 B
./dist/assets/js/googlesitekit-events-provider-optin-monster-********************.js 675 B
./dist/assets/js/googlesitekit-events-provider-popup-maker-********************.js 634 B
./dist/assets/js/googlesitekit-events-provider-woocommerce-********************.js 657 B
./dist/assets/js/googlesitekit-events-provider-wpforms-********************.js 633 B
./dist/assets/js/googlesitekit-i18n-********************.js 3.93 kB
./dist/assets/js/googlesitekit-main-dashboard-********************.js 163 kB
./dist/assets/js/googlesitekit-metric-selection-********************.js 52 kB
./dist/assets/js/googlesitekit-modules-********************.js 22.4 kB
./dist/assets/js/googlesitekit-modules-ads-********************.js 36 kB
./dist/assets/js/googlesitekit-modules-adsense-********************.js 119 kB
./dist/assets/js/googlesitekit-modules-analytics-4-********************.js 192 kB
./dist/assets/js/googlesitekit-modules-pagespeed-insights-********************.js 22.7 kB
./dist/assets/js/googlesitekit-modules-reader-revenue-manager-********************.js 41.1 kB
./dist/assets/js/googlesitekit-modules-search-console-********************.js 69.4 kB
./dist/assets/js/googlesitekit-modules-sign-in-with-google-********************.js 32.4 kB
./dist/assets/js/googlesitekit-modules-tagmanager-********************.js 32.1 kB
./dist/assets/js/googlesitekit-notifications-********************.js 43.3 kB
./dist/assets/js/googlesitekit-polyfills-********************.js 378 B
./dist/assets/js/googlesitekit-reader-revenue-manager-block-editor-********************.js 477 B
./dist/assets/js/googlesitekit-settings-********************.js 128 kB
./dist/assets/js/googlesitekit-splash-********************.js 68.7 kB
./dist/assets/js/googlesitekit-user-input-********************.js 43.9 kB
./dist/assets/js/googlesitekit-vendor-********************.js 325 kB
./dist/assets/js/googlesitekit-widgets-********************.js 104 kB
./dist/assets/js/googlesitekit-wp-dashboard-********************.js 63.3 kB
./dist/assets/js/runtime-********************.js 1.4 kB

compressed-size-action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants