QAShot is a Drupal 8 application which provides a UI to use BackstopJS to execute A/B testing of 2 sites.
Features:
- Test queue; Tests are put into a queue and are executed as soon as possible.
- UI to manage BackstopJS test cases.
QAShot is heavily in development. Breaking changes with no automatic upgrade paths might be introduced to it without notice. Use or update it at your own discretion.
A recent refactor (Commit a2ed30d102d) results in the following error:
The service "backstopjs.backstop" has a dependency on a non-existent service "qa_shot_test_worker.worker_factory".
This has no easy solution, so a workaround is supplied:
- cd
- git pull
- git apply patches/qa_shot.services-deployment_hotifx.patch
- git apply patches/backstopjs.services-deployment_hotifx.patch
- do a release as usual
- checkout the 2 patched service files and clear caches
QAShot will undergo big changes in the future.
We are developing a QAShot Worker project. This will serve as a remote execution stack independent from, but used with this Drupal based project. This will allow us to make the Drupal instance more lightweight and increase test execution speed.
We would like to also keep local run capabilities, but supporting both in a sustainable way is not possible. This means, a full back-end rewrite will start hopefully soon.
For information about the docker stack, see the README for the docker stack.
The tests are not run automatically, they are first put into a queue which is managed with cron. If you want the tests to run, you have to set up a cron job on the system.
Example:
# Into e.g /etc/cron.d/qashot-queue
#
# We want to restart on reboot.
@reboot <user that's set up to use docker without sudo> /bin/sh -c "cd <path to qashot project root> && ./prod-startup.sh"
# We want to execute queues automatically.
* * * * * <user that's set up to use docker without sudo> /bin/sh <path to qashot project root>/run-test-queue.sh 2>&1 | tee --append /var/log/custom/qashot.cron.log > /dev/null 2>&1 || true
COMPOSE_INTERACTIVE_NO_CLI=1
@reboot sudo -u ubuntu /bin/sh -c "cd ~/qashot && ./prod-startup.sh"
# Run queues every minute.
* * * * * sudo -u alpine-www-data /bin/sh ~/qashot/run-test-queue.sh > /dev/null 2>&1 || true
See https://www.drupal.org/node/23714 for more cron info.
The UI part is a standard Drupal installation. Drupal dependencies are managed with composer.
To import the existing configuration, you need to set the site uuid to the exported one:
drush config-set "system.site" uuid "f700763e-1289-406f-919e-98dc38728a53" -y
You also need to remove shortcuts from the fresh install, as the standard profile creates some by default:
drush ev '\Drupal::entityManager()->getStorage("shortcut_set")->load("default")->delete();'
To run the tests, you need BackstopJS installed globally.
Detailed install guide is in the INSTALL.md
Endpoints are exposed by the qa_shot_rest_api module.
For more information how to use it see API.md file.