Timeout executing test test suite when upgrading to Tycho 3.0.4 from 2.7.4

[rubenporras]

Hi,

when upgrading to Tycho 3.0.4 from 2.7.4 I get a timeout executing a test suite with Tycho Surefire (I have tested Tycho 3.0.1 with the same effect). Then the output of tests is visible for about 30 minutes and then it stops, the Jenkins console output shows that the process is still running but there is no output. I have a Jenkins parallel pipeline which runs two different test suites, one runs in about 20 minutes, and completes successfully, the other in about an hour, and finishes with a timeout with the upgrade.

I have checked the release notes and I cannot find anything relevant. The output of Maven with the command line which is used is the following:

2023-05-30 11:09:42.566  [INFO] --- tycho-surefire-plugin:3.0.1:test (default-test) @ com.avaloq.tools.asmd.testsuite ---
2023-05-30 11:09:43.134  [INFO] Could not find a java toolchain of type jdk, using java from JAVA_HOME instead (/data/jenkins-slave/tools/hudson.model.JDK/openJDK17U-jdk_x64_linux_hotspot_17.0.2_8/jdk-17.0.2+8/bin/java)
2023-05-30 11:09:43.134  [INFO] Executing Test Runtime with timeout 14400, logs (if any) will be placed at: /data/jenkins-slave/workspace/asmd-cf-bb-verify-pipeline@4/com.avaloq.tools.asmd.testsuite/target/work/data/.metadata/.log
2023-05-30 11:09:43.134  [INFO] Command line:
[/data/jenkins-slave/tools/hudson.model.JDK/openJDK17U-jdk_x64_linux_hotspot_17.0.2_8/jdk-17.0.2+8/bin/java, -Dosgi.noShutdown=false, -Dosgi.os=linux, -Dosgi.ws=gtk, -Dosgi.arch=x86_64, -Dfile.encoding=UTF-8, -Xms256m, -Xmx4g, -ea, -Dosgi.clean=true, -jar, /data/jenkins-slave/workspace/asmd-cf-bb-verify-pipeline@4/.repository/p2/osgi/bundle/org.eclipse.equinox.launcher/1.6.400.v20210924-0641/org.eclipse.equinox.launcher-1.6.400.v20210924-0641.jar, -data, /data/jenkins-slave/workspace/asmd-cf-bb-verify-pipeline@4/com.avaloq.tools.asmd.testsuite/target/work/data, -install, /data/jenkins-slave/workspace/asmd-cf-bb-verify-pipeline@4/com.avaloq.tools.asmd.testsuite/target/work, -configuration, /data/jenkins-slave/workspace/asmd-cf-bb-verify-pipeline@4/com.avaloq.tools.asmd.testsuite/target/work/configuration, -application, org.eclipse.tycho.surefire.osgibooter.uitest, -testproperties, /data/jenkins-slave/workspace/asmd-cf-bb-verify-pipeline@4/com.avaloq.tools.asmd.testsuite/target/surefire.properties, -product, com.avaloq.tools.ice.branding.product, -nouithread, -timeout, 14400000]
2023-05-30 11:09:43.134  Picked up JAVA_TOOL_OPTIONS: -Djava.io.tmpdir=/data/tmp
2023-05-30 11:09:48.403  SWT SessionManagerDBus: Failed to connect to org.gnome.SessionManager: Failed to execute child process “dbus-launch” (No such file or directory)
2023-05-30 11:09:48.403  SWT SessionManagerDBus: Failed to connect to org.xfce.SessionManager: Failed to execute child process “dbus-launch” (No such file or directory)
...
2023-05-30 11:39:17.814  09:39:17.589 [WorkbenchTestable] INFO  pse.xtext.builder.impl.XtextBuilder                      - Built TEST_856 in 2 ms

Then the output stops after the 30 minutes, the Jenkins console output shows that the process is still running but there is no output.

Any ideas about what could be the problem?

Thanks

[laeubi]

Can you reproduce this locally on your system or does it only happen on the CI?

[cdietrich]

i assume you need a jstack. i assume there is some kind of deadlock

[rubenporras]

Thanks for the replies. I am trying to get a local reproducer but I have failed so far. I will write again when I have more information.

At least it is good to know that there is no obvious change in behavior between 2.7.4 and 3.0.4.

Could it that different versions from Tycho use different versions from surefire and that there is some change in behavior there? Would it be possible to upgrade all the different tycho plugins to 3.0.4 and leave tycho-surefire-plugin on 2.7.4, or is that a bad idea?

[laeubi]

Yes different surefire versions are the most notable changes, you can also try out the current tycho snapshot build (4.0.0-SNAPSHOT) that has the most recent surefire version, maybe this already solve your issue.

[rubenporras]

Thanks for the hint, but I cannot test that in the corporate environment :(

So far I have been able to isolate the problem to a concrete test suite, and by disabling that test suite, to upgrade to Tycho 3.0.4. I am however, still trying to figure out, what is the problem with that disabled test suite, it does not look like it is doing something special.

[rubenporras]

It looks like a a test opens an XsdEditor and waits for the opening to finish:

"WorkbenchTestable" #56 prio=6 os_prio=0 cpu=314770.82ms elapsed=2693.74s tid=0x00007f201a381420 nid=0x3a62 in Object.wait() [0x00007f1fe86b1000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait([email protected]/Native Method)
- waiting on
at java.lang.Object.wait([email protected]/Object.java:338)
at org.eclipse.swt.widgets.Synchronizer.syncExec(Synchronizer.java:200)
- locked <0x0000000797d36948> (a org.eclipse.swt.widgets.RunnableLock)
at org.eclipse.ui.internal.UISynchronizer.syncExec(UISynchronizer.java:133)
at org.eclipse.swt.widgets.Display.syncExec(Display.java:5960)
at com.avaloq.tools.ddk.xtext.ui.util.UiThreadDispatcher.dispatchAndWait(UiThreadDispatcher.java:59)
at com.avaloq.tools.dsl.netwstruct.ui.test.internal.NetwStructTestUtil.openXsdEditor(NetwStructTestUtil.java:128)

On the main thread, the Xsd Editor open an UnknownContentTypeDialog an is waiting for some user input, an I guess it waits forever there:

"main" #1 prio=6 os_prio=0 cpu=139685.19ms elapsed=2712.72s tid=0x00007f2018025050 nid=0x305d runnable [0x00007f2021a5d000]
java.lang.Thread.State: RUNNABLE
at org.eclipse.swt.internal.gtk.OS.Call(Native Method)
at org.eclipse.swt.widgets.Display.sleep(Display.java:5641)
at org.eclipse.jface.window.Window.runEventLoop(Window.java:824)
at org.eclipse.jface.window.Window.open(Window.java:799)
at org.eclipse.jface.dialogs.MessageDialog.open(MessageDialog.java:609)
at org.eclipse.wst.sse.ui.StructuredTextEditor.doSetInput(StructuredTextEditor.java:2061)
at org.eclipse.ui.texteditor.AbstractTextEditor.lambda$1(AbstractTextEditor.java:3165)
at org.eclipse.ui.texteditor.AbstractTextEditor$$Lambda$1006/0x0000000802b32000.run(Unknown Source)
at org.eclipse.jface.operation.ModalContext.runInCurrentThread(ModalContext.java:434)
at org.eclipse.jface.operation.ModalContext.run(ModalContext.java:352)
at org.eclipse.ui.internal.WorkbenchWindow.lambda$7(WorkbenchWindow.java:2346)
at org.eclipse.ui.internal.WorkbenchWindow$$Lambda$1007/0x0000000802b362e8.run(Unknown Source)
at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:74)
at org.eclipse.ui.internal.WorkbenchWindow.run(WorkbenchWindow.java:2344)
at org.eclipse.ui.texteditor.AbstractTextEditor.internalInit(AbstractTextEditor.java:3182)
at org.eclipse.ui.texteditor.AbstractTextEditor.init(AbstractTextEditor.java:3207)
at org.eclipse.wst.sse.ui.StructuredTextEditor.init(StructuredTextEditor.java:2739)
at org.eclipse.ui.part.MultiPageEditorPart.addPage(MultiPageEditorPart.java:224)
at org.eclipse.ui.part.MultiPageEditorPart.addPage(MultiPageEditorPart.java:203)
at org.eclipse.wst.xsd.ui.internal.adt.editor.CommonMultiPageEditor.createSourcePage(CommonMultiPageEditor.java:555)
at org.eclipse.wst.xsd.ui.internal.adt.editor.ADTMultiPageEditor.createPages(ADTMultiPageEditor.java:171)
at org.eclipse.wst.xsd.ui.internal.editor.InternalXSDMultiPageEditor.createPages(InternalXSDMultiPageEditor.java:570)
at com.avaloq.tools.dsl.xsd.ui.XsdEditor.createPages(XsdEditor.java:266)

From the code in StructuredTextEditor this happens if the content type is not known, Any idea how upgrading Tycho from 2.7.5 or 2.6.x to 3.x could cause this behaviour? I do not think Tycho should affect any of this.

Thanks

[laeubi]

You can try to compare the bundles that are chosen for a test, maybe more/less items are selected that change the behavior of your tests.

[rubenporras]

Thanks, I will do that. I think I have enough to keep the investigation on my side, so I will close the ticket for the time being. Thanks for your support.