Testing Process Generator

This page is out of date! It applies only to generator but not to generator2.

Software tests

All generator test cases are defined in the project org.svis.generator.tests. Currenlty there exists at least one regression test for every workflow but only a few unit tests. Feel free to add new test cases! The projects org.svis.metrics and org.svis.hismo.extractors have a separate test folder that contains some tests as well.

Regression tests

Regression tests are simple: They calculate the checksum of two files and check for equality. This is very fast and ensures that all workflows are working. However they are not very informative in case of inequality. If you need more information create a diff manually or add a corresponding unit test.

Example: org.svis.generator.tests.rd.famix.AndroidPhoneTest.xtend

Unit tests

Unit tests are defined via XmlUnit and are executed based on the producet xml file. XmlUnit evaluates a given xpath expression. For instance it is possible to check the number of classes or methods in a file.

Example: org.svis.generator.tests.rd.famix.BankTest.xtend

Executing and skipping tests

All test cases are automatically run when using Test Generator or Build everything. However sometimes it mith be useful to skip test execution. This can be accomplished in two ways:

Skipping all Tests

Global run menu → Run Configurations… → Select the corresponding entry → Select Skip Tests

Don't commit configurations with skipped tests to master or development branch. If some tests fail, we would like to know it.

Skipping specific Tests

Open pom.xml of org.svis.generator.tests. In the configuration section of the maven-surfire-plugin it is possible to exclude test cases based on a regular expression or wild card. The file already contains some examples. However if you exclude some test cases please don't commit these changes.

Continious Integration

We use http://codeship.com/ als continious integration platform. It clones the repository and runs "Build everything" via maven. If one or more test cases fail every team member recieves an email from codeship including the corresponding log file. This process is executed automatically for every push in every branch.