Report generated with wrong number of test cases executed in case a step fails

[eitzenbe]

What happened?

you can easily reproduce the problem via this minimal repo
https://github.com/zelite/serenity-cucumber-starter-master

It seems to be related to gluecode that uses the @step annotation like screenplay-rest e.g.

What did you expect to happen?

instead of 16 test cases it should have executed 2. for me it seems that every annotated step is counted as a test case

Serenity BDD version

4.2.4-SNAPSHOT from master and earlier version down to 4.1.x

JDK version

jdk 17

Execution environment

No response

How to reproduce the bug.

https://github.com/zelite/serenity-cucumber-starter-master

How can we make it happen?

Work on this myself and propose a PR (with Serenity BDD team guidance)

[eitzenbe]

taking the branch try_without_ensure_screenplay minimizes the steps further and reveals (at least from my point of view) that the structuring of steps is not consistent. e.g.in the generated json the last step "When do something" is not inside the datavariant step "1: do something with "hello" ({something="hello"})" but after it.....

[eitzenbe]

01b033a2122873d2388c1b66d68d0c06d59773290200813d70a49a8ce2b3a9cf.json

[eitzenbe]

Hmm added one more version on the branch trying to reproduce it with steps annotation but did not succeed, it might be related to screenplay then?

[eitzenbe]

@wakaleo eager to fix this ;)

[eitzenbe]

HI @wakaleo Happy to fix this but would love to get a hook on where in the code to start to work on this issue?

[wakaleo]

I would start by following the stack trace - it could be a JDK issue related to ByteBuddy:

I am not seeing any miscounted tests though...

[eitzenbe]

Interesting, which JDK are you running? And can you share the related json file? And on which OS are you testing this?

[eitzenbe]

And you did use the branch try_without_ensure_screenplay ?