From 9ba786e433afd334303ac269bd8d2a9af0633688 Mon Sep 17 00:00:00 2001 From: William Wong Date: Mon, 23 Sep 2024 12:02:49 -0700 Subject: [PATCH] Move to npm workspaces (#5301) * Rebuild package-lock for npm workspaces * Target ESNext * Clean up start scripts * Remove prebump/postbump * Cleaner bump scripts * Clean up audit fix scripts * Remove reference to lerna * Bump Speech SDK * Update husky script * Remove tableflip * Update steps * No initial build * Add --raw * Simplify concurrently * Bump Chrome version * Add punycode * Clean up concurrently and no globalize * Add --prefix-colors * Clean up * Move to build * Build serially * Add test/web-server to workspaces * Fix tsd path * Add p-defer-es5 * Use npm install temporarily * Only import type * Add p-defer * Move away from p-defer * Add .js extension to import * Use core-js-pure for withResolvers * Use isomorphic-react * Add entry * Add entry * Fix bump scripts * Revert package-lock.json * Add --strict-peer-deps * Add more --strict-peer-deps * Add 06.e.extending-ui to npm workspaces * Fix use-state-with-ref * Add build samples --- .eslintrc.node.yml | 1 - .github/CONTRIBUTING.md | 304 +- .github/workflows/daily-release.yaml | 9 +- .github/workflows/playground.yaml | 5 +- .github/workflows/preview-branch.yml | 7 +- .github/workflows/publish-github-pages.yaml | 6 +- .github/workflows/pull-request-validation.yml | 45 +- .gitignore | 1 - CHANGELOG.md | 3 + azure-pipelines.yml | 1 - docs/BUILD_SCRIPTS.md | 9 +- lerna.json | 17 - package-lock.json | 52954 ++++++---------- package.json | 65 +- packages/api/package-lock.json | 8002 --- packages/api/package.json | 89 +- .../scripts/createPrecompiledGlobalize.mjs | 2 +- packages/bundle/package-lock.json | 14397 ----- packages/bundle/package.json | 34 +- packages/bundle/scripts/devServer.mjs | 4 +- .../src/speech/CustomAudioInputStream.ts | 23 +- packages/bundle/tsup.config.ts | 2 +- packages/bundle/webpack.config.js | 18 +- packages/component/package-lock.json | 8577 --- packages/component/package.json | 24 +- packages/core/package-lock.json | 9276 --- packages/core/package.json | 73 +- packages/core/src/createPromiseQueue.js | 14 +- packages/core/src/index.ts | 5 +- packages/core/src/tsconfig.json | 3 +- packages/core/src/utils/withResolvers.ts | 11 + .../__tests__/utilities/createTestHarness.js | 5 +- .../__tests__/utilities/waitForConnected.js | 12 +- packages/directlinespeech/package-lock.json | 19069 ------ packages/directlinespeech/package.json | 24 +- .../directlinespeech/src/createTaskQueue.js | 16 +- .../src/createTaskQueue.spec.js | 16 +- .../src/patchDialogServiceConnectorInline.js | 10 +- .../src/utils/withResolvers.js | 5 + packages/embed/package-lock.json | 16706 ----- packages/embed/package.json | 16 +- packages/fluent-theme/package-lock.json | 2179 - packages/fluent-theme/package.json | 22 +- .../isomorphic-react-dom/package-lock.json | 7068 --- packages/isomorphic-react-dom/package.json | 23 +- packages/isomorphic-react/package-lock.json | 7068 --- packages/isomorphic-react/package.json | 23 +- packages/playground/package-lock.json | 30925 --------- packages/playground/package.json | 20 +- .../cldr-data-downloader/package-lock.json | 888 - .../support/cldr-data-downloader/package.json | 15 +- packages/support/cldr-data/package-lock.json | 732 - packages/support/cldr-data/package.json | 23 +- packages/support/cldr-data/src/install.mjs | 2 +- packages/test/dev-server/package-lock.json | 2134 - packages/test/dev-server/package.json | 16 +- packages/test/fluent-bundle/package-lock.json | 3701 -- packages/test/fluent-bundle/package.json | 13 +- packages/test/fluent-bundle/src/bundle.ts | 9 +- .../test/fluent-bundle/src/reactBundle.ts | 4 + packages/test/fluent-bundle/tsup.config.ts | 36 + packages/test/harness/Dockerfile | 4 +- packages/test/harness/babel.config.json | 2 +- packages/test/harness/package-lock.json | 18075 ------ packages/test/harness/package.json | 21 +- .../harness/src/host/common/host/index.js | 16 +- packages/test/page-object/babel.config.json | 2 +- packages/test/page-object/package-lock.json | 6751 -- packages/test/page-object/package.json | 28 +- .../testHelpers/createDirectLineEmulator.js | 22 +- .../createRenderWebChatWithHook.js | 13 +- packages/test/page-object/src/index.js | 4 +- .../page-object/src/utils/withResolvers.js | 5 + packages/test/web-server/package-lock.json | 2215 - packages/test/web-server/package.json | 13 +- .../javascript/bot/package.json | 7 +- .../javascript/web/package.json | 7 +- .../l.sharepoint-web-part/package.json | 7 +- samples/04.api/e.piping-to-redux/package.json | 7 +- .../04.api/f.selectable-activity/package.json | 7 +- .../04.api/g.chat-send-history/package.json | 7 +- .../04.api/h.clear-after-idle/package.json | 7 +- .../a.minimizable-web-chat/package.json | 7 +- .../b.speech-ui/package.json | 7 +- .../c.smart-display/package.json | 7 +- .../06.recomposing-ui/d.plain-ui/package.json | 7 +- .../e.extending-ui/package-lock.json | 10638 ---- .../e.extending-ui/package.json | 7 +- .../bot/package.json | 7 +- .../web/package.json | 7 +- .../b.sso-for-enterprise/app/package.json | 7 +- .../b.sso-for-enterprise/bot/package.json | 7 +- .../rest-api/package.json | 7 +- .../c.sso-for-intranet/bot/package.json | 7 +- .../c.sso-for-intranet/web/package.json | 7 +- .../d.sso-for-teams/bot/package.json | 7 +- .../d.sso-for-teams/web/package.json | 7 +- .../app/package.json | 7 +- .../bot/package.json | 7 +- .../rest-api/package.json | 7 +- samples/package-lock.json | 20 +- samples/package.json | 3 +- 102 files changed, 19837 insertions(+), 202924 deletions(-) delete mode 100644 lerna.json delete mode 100644 packages/api/package-lock.json delete mode 100644 packages/bundle/package-lock.json delete mode 100644 packages/component/package-lock.json delete mode 100644 packages/core/package-lock.json create mode 100644 packages/core/src/utils/withResolvers.ts delete mode 100644 packages/directlinespeech/package-lock.json create mode 100644 packages/directlinespeech/src/utils/withResolvers.js delete mode 100644 packages/embed/package-lock.json delete mode 100644 packages/fluent-theme/package-lock.json delete mode 100644 packages/isomorphic-react-dom/package-lock.json delete mode 100644 packages/isomorphic-react/package-lock.json delete mode 100644 packages/playground/package-lock.json delete mode 100644 packages/support/cldr-data-downloader/package-lock.json delete mode 100644 packages/support/cldr-data/package-lock.json delete mode 100644 packages/test/dev-server/package-lock.json delete mode 100644 packages/test/fluent-bundle/package-lock.json create mode 100644 packages/test/fluent-bundle/src/reactBundle.ts delete mode 100644 packages/test/harness/package-lock.json delete mode 100644 packages/test/page-object/package-lock.json create mode 100644 packages/test/page-object/src/utils/withResolvers.js delete mode 100644 packages/test/web-server/package-lock.json delete mode 100644 samples/06.recomposing-ui/e.extending-ui/package-lock.json diff --git a/.eslintrc.node.yml b/.eslintrc.node.yml index d66b06751e..3613dfb4e8 100644 --- a/.eslintrc.node.yml +++ b/.eslintrc.node.yml @@ -37,7 +37,6 @@ rules: - jest-environment-node - jest-image-snapshot - node-fetch - - p-defer-es5 - selenium-webdriver - strip-ansi node/prefer-global/buffer: error diff --git a/.github/CONTRIBUTING.md b/.github/CONTRIBUTING.md index ada6858afa..7189b5a756 100644 --- a/.github/CONTRIBUTING.md +++ b/.github/CONTRIBUTING.md @@ -25,21 +25,19 @@ There are 2 steps to prepare the environment: [install tools](#installing-tools) Please install the following in the development environment: -- [Node.js](https://nodejs.org/) of LTS or latest version -- [NPM] with minimum version of 7.0.0 -- [Docker](https://docs.docker.com/get-docker/) -- On Windows 10, install [Windows Subsystem for Linux 2](https://aka.ms/wsl2) - - [Ubuntu 20.04](https://www.microsoft.com/en-us/p/ubuntu-2004-lts/9n6svws3rx71) +- [Node.js LTS](https://nodejs.org/) +- On Windows 11, install [Windows Subsystem for Linux 2](https://aka.ms/wsl2) + - [Ubuntu 24](https://www.microsoft.com/store/productId/9NZ3KLHXDJP5) +- [ChromeDriver](https://googlechromelabs.github.io/chrome-for-testing/) + - Extract ChromeDriver binary to project root +- [Docker Desktop](https://docs.docker.com/get-docker/) ## Preparing the repository To keep our main repository clean and easy to maintain, all changes must done in a [fork](https://github.com/microsoft/botframework-webchat/fork). -Web Chat is a [monorepo](https://en.wikipedia.org/wiki/Monorepo) and uses [`lerna`](https://lerna.js.org/). After the repository is cloned locally, run the following to install all dependencies. - ```sh -npm ci -npm run bootstrap +npm clean-install ``` # Building the project @@ -48,53 +46,53 @@ npm run bootstrap There are two ways to build: one-off and continuous build. -- For one-off building, run `npm run build` -- For continuous building, run `npm start` +- For one-off building, run `npm run build` +- For continuous building + - For the first time, run `npm run build` + - Then, run `npm start` The bundle output will be located at: -- [http://localhost:5000/webchat.js](http://localhost:5000/webchat.js) (also `webchat-es5.js` and `webchat-minimal.js`) -- `/packages/bundle/dist/webchat*.js` +- [http://localhost:5000/webchat.js](http://localhost:5000/webchat.js) (also `webchat-es5.js` and `webchat-minimal.js`) +- `/packages/bundle/dist/webchat*.js` # Trying out the build There are multiple ways to try out the build: -- Using our prebuilt environments - - Our prebuilt environments are great ways to test out the latest from Web Chat - - Using the Web Chat playground - - `cd packages/playground` - - `npm start` - - Navigate to [http://localhost:3000/](http://localhost:3000/), and click on "Official MockBot" on the upper right corner - - Using `webchat-loader` - - Navigate to [https://compulim.github.io/webchat-loader/](https://compulim.github.io/webchat-loader/) - - In the version dropdown, select [`http://localhost:5000/webchat-es5.js`](`http://localhost:5000/webchat-es5.js`) - - If you do not see this options, make sure you have run `npm start` and can access the URL - - Selecting a bot to use - - If you do not own a bot or prefer to use our bots, select "[Public] MockBot" or other bots from the presets - - If you have your own Direct Line or Direct Line App Service Extension bot: - - Check "Direct Line via Web Socket" or "Direct Line App Service Extension" in the "Protocol" section - - For "Direct Line App Service Extension", set the host name to your bot - - If you have the Direct Line secret of your bot, type it in the "Secret" box - - Otherwise, type the token in the "Token" box - - If you have a Direct Line Speech bot: - - Select the region of your resource in the "Speech region" box - - Type the subscription key in the "Speech key" box, or the authorization token in the "Speech token" box - - Selecting speech options for non-Direct Line Speech bot: - - If you do not need to use speech, clear "Speech key" and "Speech token" - - If you own a Cognitive Services Speech Services resource: - - Select the region of your resource in the "Speech region" box - - Type the subscription key in the "Speech key" box, or the authorization token in the "Speech token" box - - If you do not own Cognitive Services Speech Services resource, click "MockBot" below the "Speech key" box -- Write your own HTML page to load Web Chat - - Using a HTML page is recommended over using the playground or loader for development, for faster loading and quick reproduce ability - - `` -- Create a new React app and symlink or load tarballs from these packages, in the following order: - 1. `/packages/core` - 1. `/packages/api` - 1. `/packages/component` - 1. `/packages/directlinespeech` - 1. `/packages/bundle` +- Using ChromeDriver (recommended) + - Run `npm start`, wait until stable + - In a new terminal window, run `npm run browser` + - Browse to `simple.html` +- Using `webchat-loader` + - Navigate to [https://compulim.github.io/webchat-loader/](https://compulim.github.io/webchat-loader/) + - In the version dropdown, select [`http://localhost:5000/webchat-es5.js`](`http://localhost:5000/webchat-es5.js`) + - If you do not see this options, make sure you have run `npm start` and can access the URL + - Selecting a bot to use + - If you do not own a bot or prefer to use our bots, select "[Public] MockBot" or other bots from the presets + - If you have your own Direct Line or Direct Line App Service Extension bot: + - Check "Direct Line via Web Socket" or "Direct Line App Service Extension" in the "Protocol" section + - For "Direct Line App Service Extension", set the host name to your bot + - If you have the Direct Line secret of your bot, type it in the "Secret" box + - Otherwise, type the token in the "Token" box + - If you have a Direct Line Speech bot: + - Select the region of your resource in the "Speech region" box + - Type the subscription key in the "Speech key" box, or the authorization token in the "Speech token" box + - Selecting speech options for non-Direct Line Speech bot: + - If you do not need to use speech, clear "Speech key" and "Speech token" + - If you own a Cognitive Services Speech Services resource: + - Select the region of your resource in the "Speech region" box + - Type the subscription key in the "Speech key" box, or the authorization token in the "Speech token" box + - If you do not own Cognitive Services Speech Services resource, click "MockBot" below the "Speech key" box +- Write your own HTML page to load Web Chat + - Using a HTML page is recommended over using the playground or loader for development, for faster loading and quick reproduce ability + - `` +- Create a new React app and symlink or load tarballs from these packages, in the following order: + 1. `/packages/core` + 1. `/packages/api` + 1. `/packages/component` + 1. `/packages/directlinespeech` + 1. `/packages/bundle` # How to contribute to our code @@ -143,30 +141,30 @@ For test environment convergence and stability, Web Chat uses Docker for hosting We run test suites on every PR and require 100% pass rate. If test suites do not complete successfully, the cause may be: -- New changes are causing failure(s) in existing tests -- Intermittent services issues -- Test reliability issue, please see [#2938](https://github.com/microsoft/BotFramework-WebChat/issues/2938) -- Polluted development environment, for example: - - Outdated `node_modules` content - - Outdated Node.js or NPM +- New changes are causing failure(s) in existing tests +- Intermittent services issues +- Test reliability issue, please see [#2938](https://github.com/microsoft/BotFramework-WebChat/issues/2938) +- Polluted development environment, for example: + - Outdated `node_modules` content + - Outdated Node.js or NPM When the test suites fail: -- Identify whether the tests fail WITHOUT any code changes - - Make a fresh clone of BotFramework-WebChat and run the test suites without any changes. If the fresh clone tests pass, this means the latest code changes are causing failures. -- Identify whether the failure is intermittent - - Run the (failing) test suites again, potentially using Jest filters (F for failed tests) +- Identify whether the tests fail WITHOUT any code changes + - Make a fresh clone of BotFramework-WebChat and run the test suites without any changes. If the fresh clone tests pass, this means the latest code changes are causing failures. +- Identify whether the failure is intermittent + - Run the (failing) test suites again, potentially using Jest filters (F for failed tests) If existing test suites fail without any code changes, please determine the following: -- Test suites always fail, even after repeated runs - - The service may have been updated, causing the test suite failures. Please [file an issue](https://github.com/microsoft/BotFramework-WebChat/issues/new/choose) on the Web Chat repository. -- Test suites fail intermittently - - Service reliability problems (e.g. for DirectLine, or Mockbot) may be the cause. Please [file an issue](https://github.com/microsoft/BotFramework-WebChat/issues/new/choose) on the Web Chat repository. - - Test reliability problems may be the cause. If so, please comment on [#2938](https://github.com/microsoft/BotFramework-WebChat/issues/2938) and include: - - Failing test names/files - - Failing screencaps, if available. These images can be retrieved from `__diff_output__` - - Error messages +- Test suites always fail, even after repeated runs + - The service may have been updated, causing the test suite failures. Please [file an issue](https://github.com/microsoft/BotFramework-WebChat/issues/new/choose) on the Web Chat repository. +- Test suites fail intermittently + - Service reliability problems (e.g. for DirectLine, or Mockbot) may be the cause. Please [file an issue](https://github.com/microsoft/BotFramework-WebChat/issues/new/choose) on the Web Chat repository. + - Test reliability problems may be the cause. If so, please comment on [#2938](https://github.com/microsoft/BotFramework-WebChat/issues/2938) and include: + - Failing test names/files + - Failing screencaps, if available. These images can be retrieved from `__diff_output__` + - Error messages To debug race conditions: @@ -175,11 +173,11 @@ To debug race conditions: General tips on race conditions or intermittent test failures: -- After sending a message, wait for responses from the bot, using `await pageConditions.minNumActivitiesShown(2)` -- After a long message is shown, wait for scroll to complete, using `await pageConditions.scrollToBottomCompleted()` -- Remove or speed up animations and media progress bars - - If your screenshot is taken with a GIF animation, such as the spinner next to "Connecting..." prompt, you will want to replace it using `styleOptions`: - - `styleOptions: { spinnerAnimationBackgroundImage: 'url(/assets/staticspinner.png)' }` +- After sending a message, wait for responses from the bot, using `await pageConditions.minNumActivitiesShown(2)` +- After a long message is shown, wait for scroll to complete, using `await pageConditions.scrollToBottomCompleted()` +- Remove or speed up animations and media progress bars + - If your screenshot is taken with a GIF animation, such as the spinner next to "Connecting..." prompt, you will want to replace it using `styleOptions`: + - `styleOptions: { spinnerAnimationBackgroundImage: 'url(/assets/staticspinner.png)' }` ## Static code analysis @@ -191,74 +189,74 @@ To ignore any ESLint errors, please use `eslint-disable-next-line` instead of di There are checks that automation will not be able to capture. For example: -- Transparency - - Summarize test updates or changes as a part of the pull request - - If there are ONLY test changes, summarize those changes in `CHANGELOG.md` as well - - Fill out the pull request form with details -- Hygiene - - Make sure imports, members, variables, etc, are sorted alphabetically - - Avoid one-off variables, prefer JavaScript shorthands, shorter and faster code - - CSS - - Remove unneeded CSS styles - - Use CSS BEM and always namespace with `webchat__`, for example, `webchat__block__element--modifier` - - [Articles on CSS BEM](#articles-related-to-css-block-element-modifier-methodology) - - Only use local assets - - All assets must be self-contained and not loaded from any external URLs. This includes both Web Chat assets and test assets - - No logging to console. Only exceptions: - - Deprecation notes - - Warnings - - Errors - - No global pollution, for example: - - No `taborder` with numbers other than `0` or `-1` - - Minimize `z-index` usage - - If `z-index` is an absolute must, it must be used in a [new stacking context](https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Positioning/Understanding_z_index/The_stacking_context) - - Be mindful when using CSS styles in a component with content from end-developers. CSS styles may leak into the content, for example: - - Set CSS as high in the DOM tree as possible, for example: - - ```html -
-
Header
-

First

-

Second

-
Footer

-
- ``` - - Setting `font-family` in `
` produces fewer lines of code than setting `font-family` in `
`, `

`, and `