Skip to content

Commit

Permalink
Update E2E Documentation (#52928)
Browse files Browse the repository at this point in the history 
* Update documents 1

* Run prettier

* Additional changes

* Additional examples

* Correct misspelling

* Run prettier one last time
  • Loading branch information
@worldomonation
worldomonation authored May 17, 2021
1 parent b424a7f commit abe15f4
Show file tree
Hide file tree
Showing 6 changed files with 294 additions and 151 deletions.
13 changes: 11 additions & 2 deletions test/e2e/docs/config_values.md
View file
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
# Config Values

A full list of config values are:
This list may be out of date as configuration value are added/removed.

When in doubt, the values contained within the `config/*.json` files should be taken as authoritative.

| Name | Description | Example | Required | Store in file? |
| -------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------- | -------------- |
| accounts | A JSON list of wordpress.com account information (as JSON objects). | [ { "username": "username", "password": "password", "site": "site.wordpress.com", "features": [] }, { "email": "[email protected]", "features": [ "passwordless" ] } ] | No | **NO** |
| afterHookTimeoutMS | This is the maximum total time in milliseconds that an after test hook can take including capturing the screenshots | 20000 | Yes | Yes |
| artifacts | The directory structure for test artifacts (video, screenshots, logs). | { 'screenshots': 'screenshots'} | Yes | Yes |
| browser | The browser to use: either `firefox` or `chrome` | `chrome` | Yes | Yes |
| calypsoBaseURL | The home page for calypso | <https://wordpress.com> | Yes | Yes |
| calypsoDocker | A boolean indicating whether the tests will be run against a local Calypso Docker instance (required to ensure login works) | true | No | Yes |
Expand All @@ -16,6 +19,7 @@ A full list of config values are:
| explicitWaitMS | The explicit wait time in milliseconds to wait for an element to appear - for example a widget loading data via an API | 10000 | Yes | Yes |
| highlightElements | Boolean to indicate whether to visually highlight elements being interacted with on the page | true | No | Yes |
| inviteInboxId | This is an inbox id from mailosaur used for **invite** email testing | sad34id44ss | Yes | **NO** |
| locale | Default locale to be used unless overridden. | en | Yes | Yes |
| mailosaurAPIKey | This is the API key from mailosaur used for testing email | hsjdhsjdh7878sd | Yes | **NO** |
| mochaTimeoutMS | This is the maximum total time in milliseconds a single mocha end to end test can take to finish - otherwise it will time out. | 120000 | Yes | Yes |
| passwordForNewTestSignUps | This is the password that will be set for new sign ups created for testing purposes | alongcomplexpassword%### | Yes | **NO** |
Expand All @@ -41,10 +45,15 @@ A full list of config values are:
| testUserName | This is an existing test WordPress.com account for testing purposes - this account should have a **single** site | testuser123 | Yes | **NO** |
| testUserNameJetpack | This is an existing test WordPress.com account for testing purposes **that has a single jetpack site** | testuser123 | Yes | **NO** |
| testUserNameMultiSite | This is an existing test WordPress.com account for testing purposes **that has multiple sites** | testuser123 | Yes | **NO** |
| viewportName | Default viewport name to be used unless overridden. | desktop | Yes | Yes |
| viewportSize | Viewport size specified as width and height in pixels for each viewportName that is supported. | desktop | No | Yes |

<!-- When adding new rows, run the following command to sort the rows by alphabetical order:
cd test/e2e/docs
tail -n +7 config_values.md| sort --field-separator=\| --key=1
head -n 49 config_values.md | tail +9 | sort --field-separator=\| --key=1
Adjust the value of `head -n <x>` to be the last row of the table to be sorted.
Adjust the value of `tail +x` to be the first row of the table to be sorted.
-->
52 changes: 39 additions & 13 deletions test/e2e/docs/environment_variables.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,46 @@

Environment Variables are values that are defined at the system level to serve as configuration for programs.

## Standalone Environment Variables
## Required

| Name | Description | Example | Required | Store in file? |
| ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------ | ------- | -------- | -------------- |
| MAGELLANDEBUG | If this is set, the full mocha output is printed while running Magellan | 1 | No | **NO** |
| SAUCEDEBUG | If this is set, on test failure a breakpoint will be set in SauceLabs, enabling you to continue interacting with the browser for troubleshooting | 1 | No | **NO** |
| Name | Description | Example | Required | Store in file? |
| --------------- | --------------------------------------------------- | --------- | -------- | -------------- |
| CONFIG_KEY | Secrets decryption key obtained from a8c key store. | hunter2 | No | **NO** |
| NODE_CONFIG_ENV | Name of decrypted secrets file. | decrypted | No | **NO** |

## CircleCI Environment Variables
## Magellan

These environment variables are intended for use inside CircleCI, to control which tests are being run.
| Name | Description | Example | Required | Store in file? |
| ------------- | ------------------------------------------------------------------------ | ------- | -------- | -------------- |
| MAGELLANDEBUG | If this is set, the full mocha output is printed while running Magellan. | 1 | No | **NO** |

| Name | Description | Default | Required |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------- | -------- | ------- | -------------- | --- |
| DISABLE_EMAIL | Setting this to `true` will cause the Invite and Signup tests to be skipped | false | No |
| SKIP_TEST_REGEX | The value of this variable will be used in the `-i -g *****` parameter, to skip any tests that match the given RegEx. List multiple keywords separated by a ` | `(i.e.`Invite | Domain | Theme`) | `Empty String` | No |
| SKIP_DOMAIN_TESTS | If this value is set to `true`, the tests that attempt domain registration with be skipped. | false | No |
| SKIP_JETPACK | If this value is set to `true`, the Jetpack tests with be skipped. Useful to disable scheduled tests without modifying circleCI config | false | No |
## Test Framework (Selenium)

| Name | Description | Example | Required | Store in file? |
| --------------- | -------------------------------------------------------- | ---------------- | -------- | -------------- |
| BROWSERLOCALE | Specify the target browser locale. | en | No | **NO** |
| BROWSERSIZE | Specify the target browser window size. | desktop | No | **NO** |
| HEADLESS | Configure browser headless/headful mode. | 1 | No | **NO** |
| SCREENSHOTDIR | Override the default path for screenshots to be written. | /tmp/screenshots | No | **NO** |
| TEMP_ASSET_PATH | Override the base artifacts path. | /tmp/artifacts | No | **NO** |
| TEST_VIDEO | Configure if video recording is to be used. | true | No | **NO** |

## Test Framework (Playwright)

| Name | Description | Example | Required | Store in file? |
| --------------- | ------------------------------------------------------------- | ---------------- | -------- | -------------- |
| DISPLAYSIZE | Specify the target browser window size. | desktop | No | **NO** |
| HEADLESS | Configure browser headless/headful mode. | 1 | No | **NO** |
| LOCALE | Specify the target browser locale. | en | No | **NO** |
| SCREENSHOTDIR | Override the default path for screenshots to be written. | /tmp/screenshots | No | **NO** |
| TEMP_ASSET_PATH | Override the base artifacts path. | /tmp/artifacts | No | **NO** |
| VIDEODIR | Override the default path for video recordings to be written. | /tmp/recordings | No | **NO** |

<!-- When adding new rows, run the following command to sort the resulting sub-table in alphabetical order:
cd test/e2e/docs
head -n 38 environment_variables.md | tail +33 | sort --field-separator=\| --key=1
Adjust the value of `head -n <x>` to be the last row of the table to be sorted.
Adjust the value of `tail +x` to be the first row of the table to be sorted.
-->
Loading

0 comments on commit abe15f4

Please sign in to comment.