Skip to content

Commit

Permalink
test: Resolve Cypress Flaky fixes (#3034)
Browse files Browse the repository at this point in the history
Fixes: #2900  

This is adding fixes for the Cypress flaky specs that were caused from the [Webpack Upgrade](#2674).

This is addressing Flaky results for Cypress. A test is considered to be flaky when it can pass and fail across multiple retry attempts without any code changes. For example, a test is executed and fails, then the test is executed again, without any change to the code, but this time it passes. See more about Flaky [here](https://docs.cypress.io/cloud/features/flaky-test-management).

A Flaky result only happens in the CI and not in local since running Cypress locally only gives `pass` or `fail` results.

We had recently switched from E2E tests (end-to-end) to Component tests when upgrading [Storybook](#2674) and this had created flaky results on some of our components. We think this may be due to the nature of the Component runner as it is so much quicker than E2E. Due to it's nature of being much quicker, it resulted in flaky results due to our components not being to move as quick as the component runner. There may be times where the Component runner would move quicker than an item being focused in a menu or an item switching focus from one item to the next. 

To address the Flaky issues, there have been updates made to some Component Specs:
- Be more explicit with contexts (making sure that each line could be something that a user would do)
- Rely on the use of `findByRole()` instead of using `get()`. This allows us to be more specific about what element it is grabbing
- Removing deeply nested contexts and moving out to it's own context. This way we are not using more `beforeEach` than is needed and improves legibility.

[category:Test]
  • Loading branch information
josh-bagwell authored Nov 27, 2024
1 parent ccd5c92 commit 89347dd
Show file tree
Hide file tree
Showing 14 changed files with 911 additions and 856 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/pull-request.yml
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ jobs:
strategy:
fail-fast: false
matrix:
containers: [1, 2, 3, 4]
containers: [1, 2, 3, 4, 5]

steps:
- uses: actions/checkout@v2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/scorecard.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ on:
# To guarantee Maintained check is occasionally updated. See
# https://github.com/ossf/scorecard/blob/main/docs/checks.md#maintained
schedule:
- cron: '18 21 * * 4'
- cron: '36 23 * * 1'
push:
branches: [ "master" ]

Expand Down
Loading

0 comments on commit 89347dd

Please sign in to comment.