chore: add code coverage (🏗️🚧👷‍♂️)

[stijnmoreels]

Add code coverage reporting.

Relates to arcus-azure/arcus#264

[netlify]

✅ Deploy Preview for arcus-messaging canceled.

Name	Link
🔨 Latest commit	4c1fa03
🔍 Latest deploy log	https://app.netlify.com/sites/arcus-messaging/deploys/63d21968fa12200009e1b3d4

[codecov]

Codecov Report

❗ No coverage uploaded for pull request base (main@f67914a). Click here to learn what that means.
Patch has no changes to coverable lines.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #386   +/-   ##
=======================================
  Coverage        ?   92.11%           
=======================================
  Files           ?       86           
  Lines           ?     2854           
  Branches        ?      123           
=======================================
  Hits            ?     2629           
  Misses          ?      201           
  Partials        ?       24           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

[stijnmoreels]

Excluded the pumps projects as these are tested in the integration tests and are mostly containing code to contact Azure resources a.k.a. integration stuff.

[fgheysels]

Excluded the pumps projects as these are tested in the integration tests and are mostly containing code to contact Azure resources a.k.a. integration stuff.

I understand the reasoning, but it feels a bit wrong to me. It feels like it's a way to pimp the code-coverage results and although I know that they're being tested via integration tests, I don't think we should 'fix' this in this way.

Ideally we run code-coverage during integration tests as well,and merge those results together with the unit-test results into a single report.
However, I think the problem is here that integration tests are not always being run and unit-tests are ran by every pipeline-run.

We can maybe fix this by making sure that we only publish the code-coverage reports from the nightly build pipelines where the integration-tests are run as well.

[stijnmoreels]

However, I think the problem is here that integration tests are not always being run and unit-tests are ran by every pipeline-run.

We can maybe fix this by making sure that we only publish the code-coverage reports from the nightly build pipelines where the integration-tests are run as well.

That's not entirely true. We have two build pipelines: CI and Release, and in both pipelines we run both the unit and integration tests.

The problem with merging the reports, is that the threshold functionality does not work as it still inspects the separate reports. The same problem occurred in the Web API project. I think it will be the choice to either merge the reports and let go of the threshold, or only running the unit tests.
We could also come up with a way to inspect the merged report ourselves, and look for the total code coverage percentage, but that should be investigated.
Also, when we decide to run these integration tests, we should also decide to migrate the entire CI build as it would otherwise lead to too much test runs and unnecessary interaction problems.

[fgheysels]

However, I think the problem is here that integration tests are not always being run and unit-tests are ran by every pipeline-run.

We can maybe fix this by making sure that we only publish the code-coverage reports from the nightly build pipelines where the integration-tests are run as well.

That's not entirely true. We have two build pipelines: CI and Release, and in both pipelines we run both the unit and integration tests.

The problem with merging the reports, is that the threshold functionality does not work as it still inspects the separate reports. The same problem occurred in the Web API project. I think it will be the choice to either merge the reports and let go of the threshold, or only running the unit tests. We could also come up with a way to inspect the merged report ourselves, and look for the total code coverage percentage, but that should be investigated. Also, when we decide to run these integration tests, we should also decide to migrate the entire CI build as it would otherwise lead to too much test runs and unnecessary interaction problems.

For now, I would say: let go of the threshold.

[stijnmoreels]

Waiting for arcus-azure/arcus#292