Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#8990: fix non-awaited screenshot promise #9004

Merged
merged 4 commits into from
Aug 13, 2024
Merged

#8990: fix non-awaited screenshot promise #9004

merged 4 commits into from
Aug 13, 2024

Conversation

fungairino
Copy link
Collaborator

@fungairino fungairino commented Aug 12, 2024

What does this PR do?

Discussion

  • This PR introduces screenshot testing in our e2e playwright tests. Right now the max diff is set to 0.1 (so if the image is more than 10% different, it will fail the test).
    • Opted for consistency to instead just check to see if the image has a rendered width

Checklist

  • Added jest or playwright tests and/or storybook stories

Comment on lines -44 to -47
// Set the viewport to null because we rely on the real inner window width to detect when the sidepanel is open
// See: https://github.com/microsoft/playwright/issues/11465 and https://github.com/pixiebrix/pixiebrix-extension/blob/b4b0a2efde2c3ac5e634220b555532a2875fe5da/src/contentScript/sidebarController.tsx#L78
viewport: null,
Copy link
Collaborator Author

@fungairino fungairino Aug 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should no longer need this work-around since we removed this bit of code that detects if the sidebar is open based on the window inner width.

{ platform }: BrickOptions,
): Promise<unknown> {
): Promise<{ data: string }> {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Side note: this should probably correspond to the output schema directly

Copy link

No loom links were found in the first post. Please add one there if you'd like to it to appear on Slack.

Do not edit this comment manually.

Copy link

codecov bot commented Aug 12, 2024

Codecov Report

Attention: Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Project coverage is 74.39%. Comparing base (8318d74) to head (9dfea56).
Report is 174 commits behind head on main.

Files Patch % Lines
src/bricks/transformers/screenshotTab.ts 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9004      +/-   ##
==========================================
+ Coverage   74.24%   74.39%   +0.14%     
==========================================
  Files        1332     1353      +21     
  Lines       40817    41898    +1081     
  Branches     7634     7828     +194     
==========================================
+ Hits        30306    31168     +862     
- Misses      10511    10730     +219     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@twschiller twschiller added the bug Something isn't working label Aug 12, 2024
@twschiller twschiller added this to the 2.1.0 milestone Aug 12, 2024
Copy link

github-actions bot commented Aug 12, 2024

Playwright test results

passed  121 passed
flaky  3 flaky
skipped  6 skipped

Details

report  Open report ↗︎
stats  130 tests across 44 suites
duration  1 hour, 35 minutes, 32 seconds
commit  9dfea56
info  For more information on how to debug and view this report, see our readme

Flaky tests

chrome › tests/pageEditor/addStarterBrick.spec.ts › Add starter brick to mod
msedge › tests/pageEditor/addStarterBrick.spec.ts › Add starter brick to mod
chrome › tests/regressions/welcomeStarterBricks.spec.ts › #8740: can view the starter mods on the pixiebrix.com/welcome page

Skipped tests

chrome › tests/regressions/doNotCloseSidebarOnPageEditorSave.spec.ts › #8104: Do not automatically close the sidebar when saving in the Page Editor
msedge › tests/regressions/doNotCloseSidebarOnPageEditorSave.spec.ts › #8104: Do not automatically close the sidebar when saving in the Page Editor
chrome › tests/runtime/googleSheetsIntegration.spec.ts › can activate a google spreadsheet mod with config options
msedge › tests/runtime/googleSheetsIntegration.spec.ts › can activate a google spreadsheet mod with config options
chrome › tests/runtime/insertAtCursor.spec.ts › Insert at Cursor › 8154: can insert at cursor after opening sidebar from selection menu
msedge › tests/runtime/insertAtCursor.spec.ts › Insert at Cursor › 8154: can insert at cursor after opening sidebar from selection menu

@twschiller twschiller changed the title fix non-awaited screenshot promise #8990: fix non-awaited screenshot promise Aug 12, 2024
Copy link
Collaborator

@grahamlangford grahamlangford left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't know playwright had screenshot testing built in by default. That required a 3rd party plugin in Cypress. Nice!

@grahamlangford
Copy link
Collaborator

Looks like screenshot tests might have cross-browser compatibility issues? That's not surprising, though I thought Chrome/Edge would be similar enough.

We might want to limit those to just Chrome unless you can create screenshots for both browsers?

@fungairino
Copy link
Collaborator Author

Looks like screenshot tests might have cross-browser compatibility issues? That's not surprising, though I thought Chrome/Edge would be similar enough.
We might want to limit those to just Chrome unless you can create screenshots for both browsers?

We can create screenshots for both browsers by adding the browserName to the snapshot, but for now I'd rather defer trying to handle this more generally and just check to see if there is any rendered image where expected (the actual visual look of this particular image isn't really that relevant)

@fungairino fungairino enabled auto-merge (squash) August 13, 2024 15:15
@fungairino fungairino merged commit fec20aa into main Aug 13, 2024
22 checks passed
@fungairino fungairino deleted the screenshot-bug branch August 13, 2024 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working regression
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Image crop brick does not appear in form
3 participants