-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
A passed test from a group is not included into testng-failed.xml for skipped test which is depended on this group #2940
Comments
It if not obvious but Groups are designed for test selection and order. In your sample, there is no reason to retry the succeed test. If you use dependsOnMethods instead, it should work as you expect. |
@juherr, Here is an example from the 5.7.1 section of https://testng.org/doc/documentation-main.html:
Should a fail report include |
It will depend on what groups are enabled in the runner. Maybe in the sample, testng-failed.xml should include all the groups which were enabled and were a dependency of the failing test. @krmahadevan wdyt? |
@akrigator - How are you running these tests ? I would expect TestNG to honour the group selection only if it was enabled as part of execution by filtering on groups. I remember streamlining some issues around the group selection which perhaps can explain why you see what you see. Can you please edit your question and also include the suite xml file/surefire configuration or any other relevant details which will tell us more on the execution mode ? |
@akrigator - This is the issue that I was mentioning #2664 |
@krmahadevan, i've added a suite xml example. Fill free to request anything also you need from me and thanks for quick involvement |
@akrigator - I was guessing that this would be the case :) (You are running without choosing any groups). I think we eventually ended up restoring the old behaviour in TestNG, but sounds like the issue comes back when there are failures ? I need to first check if There are two interpretations when no group is specified:
@juherr - WDYT ? The tests are being run without any groups. So should we just think that we would like to ignore |
From my points of view: TestNG provides ability to set dependencies via
in this case a rerun of fails will work as expected. But this pattern is hard to be sustained in case of the "group" growing or increasing number of test dependent on the "group". |
TestNG Version
7.8.0
Expected behavior
The 7.3.0 version produces testng-failed.xml with passed, failed and skipped tests.
Actual behavior
The 7.6.1 and 7.8.0 produces testng-failed.xml with failed and skipped tests.
Is the issue reproducible on runner?
Test case sample
Suite xml
Contribution guidelines
Incase you plan to raise a pull request to fix this issue, please make sure you refer our Contributing section for detailed set of steps.
The text was updated successfully, but these errors were encountered: