Skip to content

Test Strategy

Jump to bottom
mweyns edited this page Apr 22, 2018 · 31 revisions

Purpose

This document details our testing approach. It's aim is to create a shared understanding of the approach and tools we are going to use with respect to testing.

Guiding Principles

The main principle of our testing strategy is shared responsibility. As everybody is going to write code at some point, it's important everybody not only writes testable code but also writes test for them. All types of tests (unit, integration, acceptance, regression, performance) should be automated. Manual testing will of course also play an important role in that your tests can not cover every possibility, and it will allow programmers to come up with new tests and improve their work.

In order not to fall behind on testing, tests should be implemented as soon as possible, which would make implementing new stuff easier without 'breaking' previously written code. Apart from testing as early as possible, it also needs to be done as often as possible, in order to find as much defects before continuing development. In general, the test cases and testing code should have the same priority as 'normal' code. This is summarized down below.

  • Shared Responsibility
  • Test Automation
  • Test Often and Early

Quality and Test Objectives

Attribute Description Measure and Target Priority
Correctness All implemented features and functions work as intended. 100% completion of all features. Very High
Integrity Prevent unauthorized access, protect privacy of entered data, prevent information loss All access will be via HTTPS over a secured connection. Very High
Maintainability How easy it is to add features, release changes or repair defects. Code Duplication < 7% *** Unit Test Coverage > 80% *** Method Length < 20 lines Very High
Availability Percentage of planned-up time that the system should be operational. 99.99% availability of web application High
Performance Expected performance of a general user request Response Time < 200ms High

Test Scope

This section specifies what is in the scope for the tests, based on the scope of the project.

In Scope

As our entire project is a client-side web application, the whole front-end and all of it's functionalities should be tested.

Test Approach

Test Types

This section describes the different test types and the tools used to execute them. A build will fail if one of the tests fails.

Unit Testing

Unit tests are tests that verify the implementation of software elements in isolation. As TypeScript is our main language, we use JasmineJS as the framework and KarmaJS as the test runner. The unit tests are automatically run every time the project is built. Building will fail if a single unit test fails.

End-to-end Testing

End-to-end testing tests the application by automating front-e

Summary

Type Definition Tools Execution/Timing
Unit Testing that verifies the implementation of software elements in isolation. Framework: JasmineJS -- Runner: KarmaJS Upon implementing new methods.
End-to-end Testing an integrated hardware and software system to verify that the system meets required functionality. Selenium Webdriver Upon implementing new functionalities.
Acceptance Testing based on acceptance criteria to enable the customer to determine whether or not to accept the system. User Acceptance Testing In week 7 and 9 (more on this later)
Performance Testing to confirm that the system meets required performance. JMeter Upon implementing new functionalities.

Functional Tests

Sprint 1

Sprint 2

  • Display information about app
Description Display information about app
Preconditions None
Postconditions Information is displayed
Normal flow
  1. User clicks on "About".
  2. User sees information about app.
Alternative flows --
  • Display contact information
Description Display contact information
Preconditions None
Postconditions Contact information is displayed
Normal flow
  1. User clicks on "Contact".
  2. User sees contact information
Alternative flows --
  • Link to gitHub repository
Description Link to UnSHACLed GitHub repository
Preconditions None
Postconditions User has been referred to UnSHACLed Github repository.
Normal flow
  1. User clicks on GitHub logo.
  2. User is referred to UnSHACLed GitHub repository.
Alternative flows --
  • Link to latest release notes
Description Link to latest release notes
Preconditions None
Postconditions User has been referred to latest release notes.
Normal flow
  1. User clicks on release button (button that indicates the latest release version).
  2. User is referred to latest release notes.
Alternative flows --
  • Login without authentication/authorization
Description Login without authentication/authorization
Preconditions None
Postconditions User is "logged in", i.e. has loaded the visual editor.
Normal flow
  1. User clicks on UnSHACLed symbol.
  2. User sees login page.
  3. User fills in random login information
  4. User clicks on "Login".
  5. User is "logged in".
Alternative flows
  1. User clicks on UnSHACLed symbol.
  2. User sees login page.
  3. User forgets to fill in a field
  4. User clicks on "Login".
  5. An error message is displayed for the corresponding field.
  • Logout without authentication/authorization
Description Logout without authentication/authorization
Preconditions User is "logged in".
Postconditions User is "logged out".
Normal flow
  1. User clicks on "Logout".
  2. User sees login page.
Alternative flows --
  • Filter building blocks
Description Filter building blocks
Preconditions User is "logged in".
Postconditions Building blocks have been filtered according to search string.
Normal flow
  1. Enters a search string in search bar.
  2. Building blocks are filtered according to search string, i.e. user sees filtered results.
Alternative flows --
  • Drag nodes
Description Drag nodes
Preconditions User is "logged in".
Postconditions Node has been dragged.
Normal flow
  1. User clicks on a node
  2. User drags this node.
Alternative flows --

Sprint 3

  • Display information about app

  • Display contact information

  • Link to UnSHACLed GitHub repository

  • Link to latest release notes

  • Login without authentication/authorization

  • Logout without authentication/authorization

  • Filter building blocks

  • Drag nodes

  • Import graph

Description Import graph
Preconditions User is "logged in".
Postconditions Graph has been imported (and displayed).
Normal flow
  1. User clicks on "Import Graph"
  2. A file picker pops up.
  3. User selects a file.
  4. User opens the file.
  5. The file is loaded.
Alternative flows --
  • Save graph
Description Save graph
Preconditions User is "logged in".
Postconditions Graph has been downloaded as file.
Normal flow
  1. User clicks on "Save Graph".
  2. User selects graph from drop down menu.
  3. User clicks save.
  4. The file containing the encoded graph is downloaded.
Alternative flows --
  • Hide the sidebar
Description Hide the sidebar
Preconditions User is "logged in".
Postconditions The sidebar has been hidden
Normal flow
  1. User clicks on the sidebar icon.
  2. The sidebar collapses.
Alternative flows --
  • Delete selected node
Description Delete selected node
Preconditions User is "logged in".
Postconditions The selected node has been deleted.
Normal flow
  1. User selects a node.
  2. User clicks on the garbage can icon.
  3. The selected node is deleted.
Alternative flows
  1. User selects nothing.
  2. User clicks on the garbage can icon.
  3. Nothing happens.
  • Undo last change
Description Undo last change
Preconditions
  1. User is "logged in".
  2. User has made a change to the graph structure (i.e. has created a new node or has changed the content of an existing node) since he/she has logged in that has not yet been undone.
Postconditions The last change has been made undone.
Normal flow
  1. User clicks on the undo button (arrow pointing left).
  2. The most recent change made to the graph structure is undone.
Alternative flows --
  • Redo last change
Description Redo last change
Preconditions
  1. User is "logged in".
  2. User has recently undone a change to the graph structure.
Postconditions The change that was most recently undone has been redone.
Normal flow
  1. User clicks on the redo button (arrow pointing right).
  2. The change that was most recently undone is redone.
Alternative flows
  1. User makes another change to the graph structure.
  2. User clicks on the redo button (arrow pointing right).
  3. Nothing happens as the change was overwritten.
  • Take snapshot
Description Take snapshot
Preconditions User is "logged in".
Postconditions A new window has opened containing a snapshot of the graph structure.
Normal flow
  1. User clicks on the camera icon.
  2. A new window has opened containing a snapshot of the graph structure.
Alternative flows --
  • Zoom in
Description Zoom in
Preconditions User is "logged in".
Postconditions The editor has zoomed in.
Normal flow
  1. User clicks on the zoom-in button.
  2. The editor zooms in where the canvas was previously focused.
Alternative flows --
  • Zoom out
Description Zoom out
Preconditions User is "logged in".
Postconditions The editor has zoomed out.
Normal flow
  1. User clicks on the zoom-out button.
  2. The editor zooms out where the canvas was previously focused.
Alternative flows --
  • Fullscreen
Description Fullscreen
Preconditions User is "logged in".
Postconditions The canvas view is zoomed out to contain the entire graph structure.
Normal flow
  1. User clicks on the fullscreen icon.
  2. The editor zooms so that the view now encompasses the entire graph structure.
Alternative flows --
  • Detail
Description Detail
Preconditions User is "logged in".
Postconditions The canvas view has assumed its original depth.
Normal flow
  1. User clicks on the small screen icon.
  2. The editor assumes the depth at which the user first entered the editor; i.e. its standard depth.
Alternative flows --
  • Create node by dragging building block
Description Create node by dragging building block
Preconditions User is "logged in".
Postconditions A new node of the respective type has been created.
Normal flow
  1. User clicks on a building block and keeps the mouse button pressed down.
  2. User drags the building block onto the canvas.
  3. User releases the mouse button.
  4. An empty node of the type represented by the building block has been created on the canvas.
Alternative flows
  1. User clicks on a building block and keeps the mouse button pressed down.
  2. User releases the building block before reaching the canvas.
  3. No new node is created.
  • Edit node field
Description Edit node field
Preconditions
  1. User is "logged in".
  2. At least one node is present on the canvas.
Postconditions A node field has been altered (for this sprint, altered is synonymous with emptied).
Normal flow
  1. User double clicks on a node field.
  2. User types anything into the field.
  3. User leaves the field.
  4. The field is emptied.
Alternative flows --

Sprint 4

  • Display information about app
  • Display contact information
  • Link to UnSHACLed GitHub repository
  • Link to latest release notes
  • Login without authentication/authorization
  • Logout without authentication/authorization
  • Filter building blocks
  • Drag nodes
  • Save graph
  • Hide the sidebar
  • Delete selected node
  • Undo last change
  • Redo last change
  • Take snapshot
  • Zoom in
  • Zoom out
  • Fullscreen
  • Detail
  • Edit node field
  • Import data graph
Description Import data graph
Preconditions User is "logged in".
Postconditions Data graph has been imported and displayed.
Normal flow
  1. User clicks on "Import Data Graph"
  2. A file picker pops up.
  3. User selects a file.
  4. User opens the file.
  5. The file is loaded and displayed.
Alternative flows --
  • Import SHACL graph
Description Import SHACL graph
Preconditions User is "logged in".
Postconditions Shapes graph has been imported and displayed.
Normal flow
  1. User clicks on "Import SHACL Graph"
  2. A file picker pops up.
  3. User selects a file.
  4. User opens the file.
  5. The file is loaded and displayed.
Alternative flows --
  • Check conformance errors
Description Check conformance errors
Preconditions User is "logged in".
Postconditions A message has been displayed containing active violations.
Normal flow
  1. User clicks on "Conformance errors" when a shapes graph and a non-conforming data graph have been loaded.
  2. A message is displayed showing the active violations.
Alternative flows
    1. User clicks on "Conformance errors" when no shapes graphs have been loaded.
    2. A message is displayed showing "No conformance errors."
    1. User clicks on "Conformance errors" when a shapes graph and a conforming data graph have been loaded.
    2. A message is displayed showing "No conformance errors."
Clone this wiki locally