Testing your changes

There are several groups of tests you can run on the Redash code. Any code that is submitted via a PR will need to pass all of them before it is merged into our master branch.

• Lint tests - very quick testing of the layout and structure of your code. Helps to ensure consistency through the code base
• Unit tests - Fairly quick testing of functions in the code. Helps catch problems without needing to run the time consuming full End-to-End (E2E) tests
• E2E tests - This runs the entire application locally in docker containers, doing all the needed (test) setup, then runs a fairly comprehensive test suite on the whole thing. Can take about 1/2 an hour to run, depending on the hardware its running on

The Lint and Unit tests both have backend (Python) and front end (javascript / typescript) tests available. The E2E tests only has one option, as it tests both the front and back end at the same time.

Quick testing

It's fairly quick and easy to run both the lint and unit tests together.

First, make sure you've set up a virtual environment (as per this page), then run make test:

$ make test
./bin/flake8_tests.sh
6.0.0 (mccabe: 0.7.0, pycodestyle: 2.10.0, pyflakes: 3.0.1) CPython 3.8.10 on Linux
pip 20.0.2 from /home/jc/redashvenv2/lib/python3.8/site-packages/pip (python 3.8)
0
[lots of extra stuff here]

The Python lint tests (flake) generally show a lot of very verbose output. We should probably fix most of them some day... 😁

In that same make test run, after the flake8 Python linting it will run the Python backend unit tests:

========================= test session starts =========================
platform linux -- Python 3.8.17, pytest-5.2.1, py-1.11.0, pluggy-0.13.1
rootdir: /app, inifile: pytest.ini
plugins: cov-2.8.1, anyio-3.7.0
collected 736 items

tests/test_authentication.py ...................................
[etc]
============ 736 passed, 307 warnings in 111.48s (0:01:51) ============

At the end of those backend unit tests, it should show many tests passed and probably a lot of warnings. There should be no errors.

Next, it will then automatically run the frontend (javascript / typescript) unit tests:

yarn test
yarn run v1.22.19
$ run-s type-check jest
$ tsc --noEmit --project client/tsconfig.json
$ TZ=Africa/Khartoum jest
 PASS  client/app/components/ApplicationArea/ErrorMessage.test.js
 PASS  client/app/services/auth.test.js
[etc]
 PASS  client/app/pages/users/components/ReadOnlyUserProfile.test.js
 PASS  client/app/components/queries/ScheduleDialog.test.js

Test Suites: 13 passed, 13 total
Tests:       1 skipped, 77 passed, 78 total
Snapshots:   12 passed, 12 total
Time:        3.028s
Ran all test suites.
Done in 6.23s.

Again, this will likely show several things passing, and possible some warnings. There should be no errors though.

Comprehensive E2E testing (Cypress)

We use Cypress version 5.3.0 (very old) for End to End (E2E) testing. It mostly works ok, though can take a bit of time to get used to.

When we previously ran the yarn command during the earlier setup process, it would have installed the cli based Cypress pieces but not the Cypress GUI. The Cypress GUI can be useful in development, so lets install that now:

$ ./node_modules/cypress/bin/cypress install

Before we start the Cypress GUI, or run the Cypress tests via cli, we need to start Redash in the background with things setup for testing:

$ yarn cypress build
$ yarn cypress start

With that done, we can launch the Cypress GUI, or run the Cypress tests via the cli (a bit more reliable).

To run the tests via the cli, use this:

$ yarn cypress run

Alternatively, to start the Cypress GUI use this:

$ yarn cypress open > /dev/null 2>&1 &

It should look like this:

To run the entire E2E test suite, click the "Run all specs" button on the right hand side.

If you just want to run an individual test file (eg for debugging purposes), then select it in the list. When you click on it, Cypress will open a new window to run just that one. You can pause it, and also view the state of the in-built browser at any point in time during the run.

For debugging purposes, it is often useful to open the Developer Tools -> Console. A lot of useful debugging information is displayed there.