Best Practices for Writing Eclipse Tests

[AhmetBilgin]

Dear members of the Eclipse Tycho-User community,

Our team is facing an important decision regarding the writing of Eclipse tests, and we would like to benefit from your collective experience and expertise.

Currently, due to historical reasons, we have a test plugin that contains the test code for approximately 80 Eclipse plugins (part of our Eclipse RCP prodcut). To improve our structure, we plan to split this test plugin. However, we are uncertain about which approach represents best practices:

a) Placing the test code in Maven style under src/test/java within the production code plugin.

b) Creating the test code as a test fragment separately from the production code plugin, which would result in approximately 80 new test fragments.

After reviewing the Tycho release notes and documentation, we suspect that option a) might be considered a best practice. We would like to hear your opinions on this matter.

• Tycho Release Notes
• Tycho Documentation on Testing

Some background information on our historical decisions and current structure:

• Our code base is extensive, including Eclipse plugins and numerous native libraries, which must be built before the (Int-)tests can be run. Our build consists of multiple build phases, with the Tycho build being only a small part. All tests are executed in the test phase (also the tests for our eclipse plugins).
• Many of our tests do not require the startup of the Eclipse framework. We have our own test runner that executes tests without starting the Eclipse framework. In the long term, we may reconsider this aspect if the performance of alternatives is acceptable.
• Many tests have an integration-test character and use other Eclipse plugins during testing.
• The majority of tests use a common test framework base.

We would greatly appreciate your insights and recommendations on this matter. Specifically, we would like to know what the recommended best practice is for "new Eclipse plugins + tests" and which approach (a) or (b) you would recommend for our existing code, taking into account the background information provided (the pro's and con's for each variante would help with our decision).

Thank you in advance for your support.

Best regards,
Ahmet Bilgin

[laeubi]

@AhmetBilgin you are right that option a) is recommended (I don't want to say best-practise as requirements may vary) especially if you don't need "Eclipse" and have own runners and have real Unit-Tests that are dedicated to the code in your plugin. This can be "plain" unit tests, or it could be a plugin test or even a quite new bnd-test...

It is also common in such setup to still have one (or a few) dedicated Integration test plugins (not fragments(!)) that are testing a larger story with many bundles involved that should work together.

For migration, I would pick one test and one bundle and just setup things, look how it works for you and then proceed with moving more stuff there, and if you see any issues please let us know as we want to improve this story see also relevant PDE isses/changes here:

• Add support for specify test requirements of a bundle eclipse-pde/eclipse.pde#914
• First class OSGi Testing Support eclipse-pde/eclipse.pde#877

[AhmetBilgin]

Hi laeubi, thank you for your answer. It was already very helpful!

We will try out option a)

So if I understand you correctly, until eclipse-pde/eclipse.pde#914 is available in PDE, we have to add test-dependencies in the pom.xml like in the tycho demo example for "same module tests" ?

Best regards,
Ahmet

[laeubi]

There are different ways, you can use the pom, some use "additional.bundles" (unter the automated dependency management tab), I even have seen optional require bundle/import packages for that purpose there is a lot out in the wild ...