[Failing Test]: org.apache.beam.sdk.io.gcp.spanner.changestreams.it.SpannerChangeStreamOrderedWithinKeyIT testOrderedWithinKey

[stankiewicz]

What happened?

java.lang.AssertionError: ParDo(ToString)/ParMultiDo(ToString).output: 
Expected: iterable with items ["{\"SingerId\":\"0\"}\n{\"FirstName\":\"Inserting mutation 0\",\"LastName\":null,\"SingerInfo\":null};{};", "{\"SingerId\":\"1\"}\n{\"FirstName\":\"Inserting mutation 1\",\"LastName\":null,\"SingerInfo\":null};{\"FirstName\":\"Updating mutation 1\"};{};{\"FirstName\":\"Inserting mutation 1\",\"LastName\":null,\"SingerInfo\":null};{};", "{\"SingerId\":\"2\"}\n{\"FirstName\":\"Inserting mutation 2\",\"LastName\":null,\"SingerInfo\":null};{\"FirstName\":\"Updating mutation 2\"};{};", "{\"SingerId\":\"3\"}\n{\"FirstName\":\"Inserting mutation 3\",\"LastName\":null,\"SingerInfo\":null};{\"FirstName\":\"Updating mutation 3\"};{};", "{\"SingerId\":\"4\"}\n{\"FirstName\":\"Inserting mutation 4\",\"LastName\":null,\"SingerInfo\":null};{};", "{\"SingerId\":\"5\"}\n{\"FirstName\":\"Updating mutation 5\",\"LastName\":null,\"SingerInfo\":null};{\"FirstName\":\"Updating mutation 5\"};{};"] in any order
     but: not matched: "{\"SingerId\":\"5\"}\n{\"FirstName\":\"Updating mutation 5\"};{};"
	at org.apache.beam.sdk.testing.PAssert$PAssertionSite.capture(PAssert.java:176)
	at org.apache.beam.sdk.testing.PAssert.that(PAssert.java:461)
	at org.apache.beam.sdk.testing.PAssert.that(PAssert.java:453)
	at org.apache.beam.sdk.io.gcp.spanner.changestreams.it.SpannerChangeStreamOrderedWithinKeyIT.testOrderedWithinKey(SpannerChangeStreamOrderedWithinKeyIT.java:126)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:288)
	at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:282)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
Caused by: java.lang.AssertionError: 
Expected: iterable with items ["{\"SingerId\":\"0\"}\n{\"FirstName\":\"Inserting mutation 0\",\"LastName\":null,\"SingerInfo\":null};{};", "{\"SingerId\":\"1\"}\n{\"FirstName\":\"Inserting mutation 1\",\"LastName\":null,\"SingerInfo\":null};{\"FirstName\":\"Updating mutation 1\"};{};{\"FirstName\":\"Inserting mutation 1\",\"LastName\":null,\"SingerInfo\":null};{};", "{\"SingerId\":\"2\"}\n{\"FirstName\":\"Inserting mutation 2\",\"LastName\":null,\"SingerInfo\":null};{\"FirstName\":\"Updating mutation 2\"};{};", "{\"SingerId\":\"3\"}\n{\"FirstName\":\"Inserting mutation 3\",\"LastName\":null,\"SingerInfo\":null};{\"FirstName\":\"Updating mutation 3\"};{};", "{\"SingerId\":\"4\"}\n{\"FirstName\":\"Inserting mutation 4\",\"LastName\":null,\"SingerInfo\":null};{};", "{\"SingerId\":\"5\"}\n{\"FirstName\":\"Updating mutation 5\",\"LastName\":null,\"SingerInfo\":null};{\"FirstName\":\"Updating mutation 5\"};{};"] in any order
     but: not matched: "{\"SingerId\":\"5\"}\n{\"FirstName\":\"Updating mutation 5\"};{};"
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:18)
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:6)
	at org.apache.beam.sdk.testing.PAssert$AssertContainsInAnyOrder.apply(PAssert.java:1701)
	at org.apache.beam.sdk.testing.PAssert$AssertContainsInAnyOrder.apply(PAssert.java:1681)
	at org.apache.beam.sdk.testing.PAssert$CheckRelationAgainstExpected.apply(PAssert.java:1300)
	at org.apache.beam.sdk.testing.PAssert$CheckRelationAgainstExpected.apply(PAssert.java:1280)
	at org.apache.beam.sdk.testing.PAssert.doChecks(PAssert.java:1632)
	at org.apache.beam.sdk.testing.PAssert$GroupedValuesCheckerDoFn.processElement(PAssert.java:1589)
	at org.apache.beam.sdk.testing.PAssert$GroupedValuesCheckerDoFn$DoFnInvoker$1ce70db5.invokeProcessElement(Unknown Source)
	at org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:217)
	at org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:192)
	at org.apache.beam.runners.core.SimplePushbackSideInputDoFnRunner.processElementInReadyWindows(SimplePushbackSideInputDoFnRunner.java:80)
	at org.apache.beam.runners.direct.ParDoEvaluator.processElement(ParDoEvaluator.java:244)
	at org.apache.beam.runners.direct.DoFnLifecycleManagerRemovingTransformEvaluator.processElement(DoFnLifecycleManagerRemovingTransformEvaluator.java:54)
	at org.apache.beam.runners.direct.DirectTransformExecutor.processElements(DirectTransformExecutor.java:165)
	at org.apache.beam.runners.direct.DirectTransformExecutor.run(DirectTransformExecutor.java:129)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)

example: https://github.com/apache/beam/pull/37758/checks?check_run_id=65695009140

Issue Failure

Failure: Test is flaky

Issue Priority

Priority: 1 (unhealthy code / failing or flaky postcommit so we cannot be sure the product is healthy)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner