Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Percy and jest-puppeteer environment setup for visual testing #670

Merged

Conversation

KshitijThareja
Copy link
Contributor

@KshitijThareja KshitijThareja commented Jun 21, 2024

Description

This PR introduces the environment setup and configuration for introducing visual testing to the KDS repository.

Steps to test

  1. Sign up for a Percy.io account and create a new percy webapp project.
  2. Install all the dependecies by running yarn install in the root directory of your local copy of KDS.
  3. Copy the project token and set it as an environment variable.
  4. Run yarn test:visual to execute visual tests.

Implementation notes

At a high level, how did you implement this?

To implement this, several new packages like jest-puppeteer, @percy/cli, etc were added to the project's dependecies and several changes were made to the current testing environment to allow for visual tests to be run along with the unit tests.

Does this introduce any tech-debt items?

--

Testing checklist

  • Contributor has fully tested the PR manually
  • If there are any front-end changes, before/after screenshots are included
  • Critical and brittle code paths are covered by unit tests
  • The change is described in the changelog section above

Reviewer guidance

  • Is the code clean and well-commented?
  • Are there tests for this change?
  • Are all UI components LTR and RTL compliant (if applicable)?
  • Add other things to check for here

After review

  • The changelog item has been pasted to the CHANGELOG.md

Comments

Copy link
Member

@AlexVelezLl AlexVelezLl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Woooo this is super exciting! 🎉 🎉 Thank you @KshitijThareja!!! This is looking super good. I just left some minor comments/questions, but this is looking great! 😃

.eslintrc.js Outdated
// Remove linting errors for the globals defined in the jest-puppeteer package
esLintConfig.env = {
...esLintConfig.env,
'jest': true,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dont think we should add this here, because this lint config is for the whole package, including the development environment where we are not running jest, and I think this could cause unwanted behaviours. Probably we could set these settings in a specific file for tests?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @KshitijThareja! why do we need this property here in the esLintConfig env?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had followed troubleshooting guidelines as specified here: Jest-Puppeteer
This was basically to recognize jest's global variables and not flag their use while running linting tests.
But on taking a closer look now, I can see that this property has already been included in the main esLint config import (kolibri-tools/.eslintrc). I'll make the required changes.

.eslintrc.js Show resolved Hide resolved
.percy.yml Outdated Show resolved Hide resolved
jest.conf/visual.index.js Outdated Show resolved Hide resolved
jest.conf/setup.visual.js Outdated Show resolved Hide resolved
start-server-and-test.js Outdated Show resolved Hide resolved
start-server-and-test.js Outdated Show resolved Hide resolved
lib/KImg/__tests__/KImg.spec.js Outdated Show resolved Hide resolved
lib/KImg/__tests__/KImg.spec.js Outdated Show resolved Hide resolved
package.json Outdated Show resolved Hide resolved
start-server-and-test.js Outdated Show resolved Hide resolved
@KshitijThareja
Copy link
Contributor Author

Hi @bjester @AlexVelezLl :)
I've added the component rendering mechanism for visual testing. Sharing some sample test results:

Screenshot from 2024-06-30 17-03-56

For this test, I rendered the KButton component from within the tests written in KButton.spec.js.

I'd love to hear your thoughts and suggestions on this.

Copy link
Member

@AlexVelezLl AlexVelezLl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @KshitijThareja! This looks good, I ran the tests, and I didn't find any problems in Percy's dashboard. I just left some minor concerns, let me know what you think!

startVisualTests.js Show resolved Hide resolved
docs/pages/testing-playground.vue Outdated Show resolved Hide resolved
.gitignore Outdated Show resolved Hide resolved
lib/KImg/__tests__/KImg.spec.js Outdated Show resolved Hide resolved
package.json Outdated Show resolved Hide resolved
expect(error.mock.calls[0][0].message).toBe(
'Missing required prop - provide altText or indicate isDecorative'
);
expect(() =>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @KshitijThareja! I think we can let this file without modifications. In this case it seems it is the expected behaviour to pass an error listener to catch these errors. See #645 (comment).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. I'll make the changes. I was currently working on finding a way to use concurrently as discussed, so was a bit busy with that. I'll get this resolved by today at the earliest.

Copy link
Member

@AlexVelezLl AlexVelezLl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @KshitijThareja!!! 🎉 This looks good. I have run some tests and it works fine, so the setup is working as expected. Follow up about concurrently will happen in #676.

Failed checks are because of node version update and missing changelog, but we dont need to add it in this PR as is still a feature WIP. So nothing else to not from my side, but will let the final approve and merge to @bjester 👐.

@AlexVelezLl AlexVelezLl requested a review from bjester July 9, 2024 14:54
@bjester
Copy link
Member

bjester commented Jul 10, 2024

One last thing @KshitijThareja: please separate the visual tests from the existing yarn test so we can keep the GH test check passing for now. We should add a followup for the GH Actions integration which runs the visual tests and the Percy diff checks.

@bjester bjester merged commit 905e770 into learningequality:gsoc/visual-testing Jul 10, 2024
3 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants