Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update quality-checks.md #265

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
+1 −1
Open

Update quality-checks.md #265

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d167ea2
Update quality-checks.md
AndrewBell81 Jul 15, 2022
d3e1114
Merge branch 'main' into patch-1
andyblundell Jul 15, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion quality-checks.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ We recommend tracking progress on an Engineering Quality dashboard, for example:

| Quality check | Classification | Applicability | What it means | Why we care | Tolerances for green | Endorsed tools / configuration | Further details |
| :----------------------------------------------------- | :-------------- | :------------ | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | :------------------------------------------------------------------------------------------------------------ |
| <a name="accessibility-test"></a> Accessibility test | Other | Universal | The practice of making applications usable by as many people as possible. | It is a regulatory requirement that our applications are accessible by as many people as possible. Catching accessibility failures up front is essential to maximise the accessibility | | | |
| <a name="accessibility-test"></a> Accessibility test | Other | Universal | The practice of making applications usable by as many people as possible. | It is a regulatory requirement that our applications are accessible by as many people as possible. Catching accessibility failures up front is essential to maximise the accessibility | | BrowserStack Automate <br /> <br /> - Windows 11: Chrome, Firefox, Edge <br /> <br /> - Windows 10: Chrome, Firefox, Edge <br /> <br /> - OS X (2020+): Safari <br /> <br /> - iOS (2020+): Safari, Chrome <br /> <br /> - Android (2020+): Chrome | |
| <a name="api-test"></a> API / contract test | Functionality | Contextual | Check whether the API interface adheres to the agreed contract | Any API interface is an integration point with another component or a software systems. An extra care has to be taken to ensure compatibility and stability of that integration are maintained so that we don't break applications that depend on our APIs | Builds fail if any tests fail | Postman | [Automate Your API Tests with Postman](https://www.postman.com/use-cases/api-testing-automation/) |
| <a name="capacity-test"></a> Capacity test | Resilience | Contextual | Identify the application's breaking point in terms of an increasingly heavy load. Degradation may manifest itself as <br/>* throughput bottlenecks<br/>* increasing response times<br/>* error rates rising | Without this test, we don't know how much load the application can handle before the application breaks or degrades | | | |
| <a name="chaos-test"></a> Chaos test | Resilience | Contextual | Cause failures in a system to test the resiliency of that system and its environment, and our ability to respond to failures | Give the team confidence that failures in a given environment will not lead to unplanned downtime or a negative user experience<br/><br/>Ensures that the team has visibility (e.g. dashboards and alerts) to be able to identify issues<br/><br>Surface performance bottlenecks, weaknesses in system design and tipping points that aren't visible through other types of testing.</br><br><br/>Help the team to understand their mean time to recovery (MTTR) and to build muscle memory & confidence for recovery activities | Regular (at least every couple of months) game days, and:<br/><br/>Builds fail if any test fails - note, these tests are slow, and are likely to be part of an infrequently-triggered (e.g. overnight) build<br/><br/>The tests cover whether the system self-heals, auto-scales, and alerts as expected | [aws-fis](tools/aws-fis/) | |
Expand Down