How to clearly separate unit tests and integration tests?

[lppedd]

Currently the testing strategies offered by Tycho seems to be a bit of mix up. It's difficult to get a sense out of it.

• tycho-surefire-plugin:test is executed only for eclipse-test-plugin packaging, in the integration-test phase.
  This is strange for two reasons:

  1. I would expect a test mojo to run on the test phase
  2. It only applies to eclipse-test-plugin, and not to eclipse-plugin

• tycho-surefire-plugin:plugin-test is executed only for eclipse-plugin packaging, in the integration-test phase.
  Feel strange because I'd like to execute JUnit tests on the test phase, and only SWTBot tests on the integration-test phase.

Ultimately my question is, which approach is the modern approach? How do I end up with unit tests in the same source bundle executed in the test phase, and SWTBot tests in the integration-test phase?

[laeubi]

Currently the testing strategies offered by Tycho seems to be a bit of mix up. It's difficult to get a sense out of it.

Yep we track this here:

• Add a section that explains different ways of testing tycho-plugins #1502
• What kind of documentation do you need or feels incomplete? #1597

* `tycho-surefire-plugin:test` is executed only for `eclipse-test-plugin` packaging, in the `integration-test` phase.
  This is strange for two reasons:
  
  1. I would expect a `test` mojo to run on the `test` phase
  2. It only applies to `eclipse-test-plugin`, and not to `eclipse-plugin`

Because of this I do not recommend using eclipse-test-plugin packaging for new designs but we have to keep it that way for historic reasons.

* `tycho-surefire-plugin:plugin-test` is executed only for `eclipse-plugin` packaging, in the `integration-test` phase.
  Feel strange because I'd like to execute JUnit tests on the `test` phase, and only SWTBot tests on the `integration-test` phase.

The default binding was removed in Tycho 4.x because it is not flexible enough, even though you can bind "classic" JUnit Tests...

Ultimately my question is, which approach is the modern approach? How do I end up with unit tests in the same source bundle executed in the test phase, and SWTBot tests in the integration-test phase?

... see for example the following integration-tests

https://github.com/eclipse-tycho/tycho/tree/master/tycho-its/projects/surefire.combinedtests

[lppedd]

Thank you! Looking at the IT POM helped.
So, to recap:

1. include unit/integration tests in eclipse-plugin packaging
2. use maven-surefire-plugin bind to the test phase (which is by default) to execute unit tests without a workbench
   

   tycho/tycho-maven-plugin/src/main/resources/META-INF/plexus/components.xml

   Lines 39 to 41 in 42397c7

   	<test>
   	org.apache.maven.plugins:maven-surefire-plugin:${surefire-plugin.version}:test
   	</test>

3. use tycho-surefire-plugin bind to integration-test to execute SWTBot tests.

However, I then don't get why we have <useUIHarness> in tycho-surefire-plugin if we expect to run only integration tests. I would expect it to be true by default.

[laeubi]

However, I then don't get why we have in tycho-surefire-plugin if we expect to run only integration tests. I would expect it to be true by default.

Integration test run inside an OSGi framework, they do not necessarily have an UI and even if they have an UI the do not necessarily require useUIHarness (what is a special flag for Eclipse Ui tests!).

[lppedd]

@laeubi got it. This discussion helped too #1492 (comment)
Seems like I was not the only one confused at least 😆