Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Snapshot testing - Part 3] Shared lending platform #1087

Merged
merged 37 commits into from
Dec 20, 2024

Conversation

meissadia
Copy link
Contributor

@meissadia meissadia commented Dec 11, 2024

Closes #1062
Closes #1090

Changes

This PR builds off of #1072, so includes all of those commits, but is separate to emphasize this PR's focus on adding snapshots to the shared-lending-platform tests. Once #1072 is reviewed and merged, this PR should be much easier to review.

  • Add snapshot checks to E2E Shared Lending Platform tests

How to test this PR

  1. yarn test:e2e:snapshot
  2. Run shared-filing-platform tests

Screenshots

Screenshot 2024-12-12 at 10 19 08 AM

…variable (SBL_ENABLE_PLAYWRIGHT_SNAPSHOT_TESTING)
…variable (SBL_ENABLE_PLAYWRIGHT_SNAPSHOT_TESTING)
… .snapshot-ignore to the element) to simplify the process
@meissadia meissadia force-pushed the 1062-snapshot__shared-lending-platform branch from dffce7c to 9d45947 Compare December 11, 2024 21:25
@meissadia meissadia mentioned this pull request Dec 11, 2024
8 tasks
@meissadia meissadia marked this pull request as ready for review December 11, 2024 21:59
@meissadia meissadia enabled auto-merge (squash) December 11, 2024 23:30
Copy link
Contributor

@billhimmelsbach billhimmelsbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great and thanks for remembering to add the documentation to the readme.

I did notice the flakiness you mentioned when it comes to the footer: just like you said the gray on black with the link dots cause the most issues.

My thought on it: could we have one assertion for the unauthenticated landing page that does the image diff on the full page and the footer, then mask the footer with a mask for the rest of the tests since the footer is static?

You could add a bool that's like isFlakyRegionIgnored or just isFooterIgnored to your takeSnapshot function that defaults to true. Then for your one test on the unauthenticated landing page that includes the full page and footer, you could set isFlakyRegionIgnored=false. Then for the rest of the tests, isFlakyRegionIgnored could default to true, and a mask could be applied to another class like .snapshot-ignore-flaky or you could just pick out the footer class that already exists and mask that.

I think it would cut down a lot on the flakiness hopefully, since then we won't be testing the footer over and over. 🤞

@meissadia
Copy link
Contributor Author

I've got a draft to mask the Footer on most pages but will wait until we merge #1072 to cut down on merge conflicts.

@billhimmelsbach
Copy link
Contributor

There is the option in playwright just to take a screenshot of an element, so you could just take a screenshot of the footer if you wanted to. I'm not sure though if that would make the test more/less flaky @meissadia, but might be worth testing?

@meissadia
Copy link
Contributor Author

@billhimmelsbach To avoid blocking this PR as we head into the holidays, I've created a follow-up ticket to explore the Footer flakiness.

[Snapshot] Reduce flakiness of Footer snapshot tests

Copy link
Contributor

@billhimmelsbach billhimmelsbach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Thanks for ticketing that.

@meissadia meissadia merged commit 61edf51 into main Dec 20, 2024
10 checks passed
@meissadia meissadia deleted the 1062-snapshot__shared-lending-platform branch December 20, 2024 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[e2e] Snapshot testing - Add script to easily update baseline snapshots [e2e] Add snapshot testing
2 participants