Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added test tool to do UI snapshot testing #343

Closed
wants to merge 10 commits into from

Conversation

ViMaSter
Copy link
Member

@ViMaSter ViMaSter commented Feb 18, 2023

DRAFT PR UNTIL BUILD WORKFLOW IS UPDATED TO UPLOAD SCREENSHOTS OF FAILED TESTS

Summary

To prevent issues like #312 from reoccurring I added a one-line tool that can be added to do snapshot testing similar to Jest.

When initially hit, Unity creates a screenshot, which needs to be committed to the repository. For all subsequent runs, Unity creates another screenshot and compares it with the one stored inside the repository.

Before/after screenshots and/or animated gif

Adding yield return ScreenshotDiff.TakeScreenshotOrCompare() to any test causes one these things to happen:

  • if there's no "expected state" yet and the Editor is open...
    1. Unity takes a screenshot and stores it inside /Assets/Tests/PlayModeTests/Tools/Screenshots/{FullNameOfTest}/{TakeScreenshotOrCompareWasCalledInsideTest}.png
    2. Unity continues the test
    3. this file becomes the "expected state" of the UI at that point; commit this file to the repository
  • if there's no "expected state" yet and the test is running on GitHub...
    1. Unity marks the test as failed as there's no "expected" state for Unity to compare against
      image
  • if the "expected state" exists...
    1. Unity takes a screenshot as "actual state" and stores it inside a temporary location
    2. Compares the "actual" with the "expected" state
      1. if it matches
        1. the test continues
      2. if it doesn't match
        1. Unity copies the expected and actual screenshot to /TestRunArtifacts/TestContext.CurrentTestExecutionContext.StartTime/{FullNameOfTest}.{TakeScreenshotOrCompareWasCalledInsideTest}
        2. Unity marks the test as failed and uploads both expected and actual screenshots
          image
          image

Testing instructions

not yet available

Additional information

  • [ ] Changes UI
  • [ ] Introduces new feature
  • [ ] Removes existing feature
  • [ ] Has associated resource:

@codecov-commenter
Copy link

codecov-commenter commented Feb 18, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 84.34%. Comparing base (ea6a61f) to head (78ddf3d).
Report is 261 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #343      +/-   ##
===========================================
+ Coverage    84.18%   84.34%   +0.15%     
===========================================
  Files           78       76       -2     
  Lines         2586     2574      -12     
===========================================
- Hits          2177     2171       -6     
+ Misses         409      403       -6     
Flag Coverage Δ
EditMode 31.04% <ø> (+0.14%) ⬆️
PlayModePlaythrough 66.20% <ø> (+0.30%) ⬆️
PlayModeScenes 60.17% <ø> (+0.04%) ⬆️
PlayModeScripts 60.68% <ø> (-0.07%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@ViMaSter ViMaSter closed this Aug 31, 2024
@ViMaSter ViMaSter deleted the features/snapshot-testing branch August 31, 2024 08:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants