Backend Core - Testing Improvements

[jbee]

A page to collect and sketch ideas around improving the testing.

Goals

make text execution...

• fast
• hurdle free (mostly relevant for e2e)
• stable
• work with SL demo DB (always when "clean", likely when "dirty")
• uniform (API tests use same API/coding style across e2e and integration)
• easy to debug

Proposals

A. Make e2e code like integration tests

This entails two main things

• Introduce a new base test layer that is similar to what we use in integration except that it implements making request with actual HTTP client instead of springs mock MVC client
• define/establish a minimal starting point data wise and the general mechanisms around setup and tear-down

B. Move integration to e2e tests

Based on the finding that e2e tests are faster and do test the actual product there are benefits in moving away from containerized integration tests. The goal here is to establish a base test in e2e (as outlined in A) that would allow to more or less copy&paste a test from integration to e2e and it still works

C. Slim the containerized integration test setup using embedded jetty

TBD

D. Recode e2e tests based on simpler tech stack

Once A is done have a look at the "old" e2e tests and re-code the scenario in a new test. This intentionally should not try to redo the original tests exactly as it was. The implicit assumptions made by the "old" e2e tests are one of the reasons they are hard to deal with. The new test scenario not just uses another API to do the exact same thing. It should rethink in particular the setup and tear-down requirements. Ideally this means we are still using the same input files to setup and run the tests, just that there are no special assumptions made beyond what is encoded in such files. If files create objects the test is also responsible to clean up afterwards. This should not be left to some "magic" change tracking build into the abstract test.