Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Use Karma instead of custom-built mocha test-runner in the browser #1694

Draft
wants to merge 4 commits into
base: 4.x
Choose a base branch
from

Conversation

nknapp
Copy link
Collaborator

@nknapp nknapp commented Jun 9, 2020

At the moment, the saucelabs tests are failing for IE and Safari. I already had a short mail-exchange with a customer-support technician, and he suggested we make sure that the following browser-caps are used when calling the SauceLabs API.

seleniumVersion: '3.141.0', 
iedriverVersion: '3.141.0' 

I have not succeeded in doing so, so as an alternative I thought we could also use Karma.
This is a work-in-progress PR for this change, but I would like to have some feedback before merging it. What I like about Karma is:

  • It works (I've tested that).
  • Log-output is visible in the travis-ci output. You don't need to go to saucelabs to read it.

What I don't like:

  • You cannot have multiple entry-points for multiple test-setups. At the moment, we have different entry-points for the default-build, AMD and UMD, so we can use one Saucelabs browser for testing multiple environments. With Karma, we have to start the Chrome-browser multiple times.
    This may slow down the tests, and I don't know what the impacts on pricing are (it probably doesn't matter, because I think we have a free open-source account at SauceLabs).

However: If we introduce new code for SauceLabs (i.e. the Karma configs), we should also get rid of the old code, but it is stil in use when you run "grunt dev". The usual use case is: You run "grunt dev", point your browser to "http://localhost:9999/spec" and it will run all tests in the browser. I'm not sure if it re-runs tests in the browser after file-changes.

With Karma, you can either choose to let Karma start one browser (or multiple), so when you change a file and save it, a browser will pop-up and run the Karma tests. This might be disturbing.

You can also tell Karma, just to run the server. When you open the browser and point it to the server, the tests will not run until you change a file (at least with the current configuration). This seems counter-intuitive. On the other hand, if you have "grunt dev" running, and multiple browsers pointed to the Karma-server, it will run the tests in all of them, which is pretty cool.

My main concern however is, that you can only have a single entry-point. So in dev-mode, you will not be able to point your browser to "index.html", "amd.html", "umd.html" to test the different build-targets. I don't know if anyone is actually doing this. I usually only run the tests in NodeJs and leave the rest to the CI.

With all that said (and thank you for reading this far):

  • Do we want to use Karma instead of the current solution?
  • Is the new code in the PR simple enough to understand?

There is still some work to do here (remove old code, cleanup), but I'd like to have some reassurance first. I'd like to know that I am doing the right thing...

@jaylinski jaylinski marked this pull request as draft September 6, 2023 21:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant