Skip to content

Latest commit

 

History

History
65 lines (45 loc) · 2.81 KB

testing.md

File metadata and controls

65 lines (45 loc) · 2.81 KB
title tags
Testing
testing
dsv-cli

Testing currently is covered in three layers.

  • Unit Tests:
    • Regular unit testing that created for each ".go" file in every package.
  • Integration Tests:
    • The older one leaves in "cicd-integration" directory. It builds the CLI binary, prepares the environment and checks the CLI output for each set of arguments.
  • End-To-End Tests:
    • A newer approach is implemented in "tests/e2e" directory. It supports testing workflows/wizards.

CICD Integration Tests

To invoke, all the required environment vars are checked and correct flags are passed via: mage test:integration.

Two important notes for "cicd-integration" tests:

  • all test cases must be run one by one, because sometimes they depend on each other, e.g. one will create a secret and a next one will try to read it
  • names/paths for different DSV objects are hardcoded, hence you cannot run it in parallel against one DSV tenant
{"secret-soft-delete", []string{"secret", "delete", secret1Name}, outputPattern("will be removed")},

End To End Testing

A newer approach is implemented in "tests/e2e" directory. This testing suite is a bit more advanced, because among the other cool things it also supports testing workflows/wizards.

c.ExpectString("Select an option")
c.SendKeyArrowDown()
c.SendKeyArrowDown()
c.SendKeyEnter()

All the names for DSV objects will be generated by the testing suite. This allows multiple runs of the suite simultaneously. In the same way as cicd-integration, the tests/e2e testing suite also compiles the binary and collects the code coverage.

WIP: Currently, this is a work in progress for migrating all tests from cicd-integration to tests/e2e.

Code Coverage

The end2end test performs actions as a compiled cli. Using go test -c this generates a compiled binary that is able to track the code coverage for what it performs.

This is inspired by the article by Elastic Code Coverage for your Golang System Tests.

This means that if running locally, you'll have to go test -cacheclean as the test caching isn't able to detect changes outside the code in the e2e directory.

To simplify this and invocation of the tests: mage test:integration and mage test:endtoend handle this automatically for you.

Other

There is an active branch for using DSV SDK with Mage to retrieve test configuration values. This will mean eventually all that is needed to run the full suite of tests locally would be a one time initialized dsv profile.

This work is located in: refactor/469577/start-of-using-flags-instead-of-envvars which focused on using flags instead of env vars. This would allow mage test to provide the correct values directly instead of requiring more complex environment setup.