-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
@Ignore doesn't work when used on multiple child classes, and parent has @Test methods #2868
Comments
The implementation of ignore is basic with limitations. |
I've bumped into this same issue too - alphabetical ordering of which tests might be ignored seems highly unintuitive to me! |
@ignore does not work either for methods on implemented interfaces. This was not the behavior of 6.9.9. That is: import org.testng.annotations.Test;
public interface MyInterface {
@Ignore
default void ignoreMePlease()
{
System.out.println("do the hussle, but ignore it!");
}
} implemented by the test class import org.testng.annotations.Test;
@Test
public class ParentClassTest implements MyInterface {
@Test
public void testc() {
hook();
}
protected void hook() {};
} TestNG will attempt to run every default method on every implemented interface, which leads to an explosion that practically disables it in the presence of any meaningful interface(s). |
@atassaad I think your sample is not complete: The method from the interface is not supposed to be discovered as a test method, except if a class is annotated with |
@juherr Good catch, I will edit it accordingly, but we are still in a bind: 6.9.9 never considered default methods from interfaces as Test methods when the implementing class was marked as Test. And currently there is no mechanism (barring a transformer I guess) to recover the behavior of 6.9.9 without modifying thousands of test classes. |
@atassaad please share a full sample and a description of the actual and expected result. |
@juherr This sample is now reflective of our entire codebase (test classes that implement interfaces that "plug tests" into a cloud environment. I have removed the @ignore annotation from the interface method (as it is invalid). public interface MyInterface {
default void ignoreMePlease()
{
System.out.println("do the hussle, but ignore it!");
}
} implemented by the test class import org.testng.annotations.Test;
@Test
public class ParentClassTest implements MyInterface {
@Test
public void testc() {
hook();
}
protected void hook() {};
} Here is the behavior of this code under TestNG 6.9.9: |
@atassaad I don't catch why you set As a workaround, what if you annotate the default method with |
@juherr just for completeness: the use case is that we have thousands of test classes annotated for grouping purposes, whereas the class methods are annotated for enable/disable of individual tests and for method dependencies. |
@atassaad - Here's a sample annotation transformer that can help you filter out all the import org.testng.IInvokedMethod;
import org.testng.IInvokedMethodListener;
import org.testng.ITestResult;
import org.testng.SkipException;
public class MethodPruningTransformer implements IInvokedMethodListener {
@Override
public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
boolean isDefault = method.getTestMethod().getConstructorOrMethod().getMethod().isDefault();
String msg = String.format("Is %s() a default Method ? %s", method.getTestMethod().getQualifiedName(),
isDefault);
System.err.println(msg);
if (isDefault) {
throw new SkipException("Skipping default method");
}
}
} import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
import java.util.stream.Collectors;
import org.testng.IReporter;
import org.testng.IResultMap;
import org.testng.ISuite;
import org.testng.ISuiteResult;
import org.testng.ITestContext;
import org.testng.ITestNGMethod;
import org.testng.xml.XmlSuite;
public class MethodLogger implements IReporter {
private List<String> methods;
public List<String> getMethods() {
return methods;
}
@Override
public void generateReport(List<XmlSuite> xmlSuites, List<ISuite> suites,
String outputDirectory) {
methods = suites.stream().flatMap(each -> each.getResults().values().stream())
.map(ISuiteResult::getTestContext)
.flatMap(each -> extract(each).stream())
.map(ITestNGMethod::getQualifiedName)
.collect(Collectors.toList());
}
private Collection<ITestNGMethod> extract(ITestContext context) {
return new ArrayList<>(extract(context.getPassedTests()));
}
private Collection<ITestNGMethod> extract(IResultMap map) {
return map.getAllMethods();
}
} Test-case looks like below import static org.assertj.core.api.Assertions.assertThat;
import org.testng.TestNG;
public class TestCase {
public static void main(String[] args) {
TestNG testng = new TestNG();
MethodLogger logger = new MethodLogger();
MethodPruningTransformer transformer = new MethodPruningTransformer();
testng.addListener(transformer);
testng.addListener(logger);
testng.setTestClasses(new Class<?>[]{ParentClassTest.class});
testng.run();
assertThat(logger.getMethods())
.containsOnly("com.rationaleemotions.issue2868.ParentClassTest.testc");
}
} |
@krmahadevan Good catch, your workaround is clearly better! 👍 |
TestNG Version
Expected behavior
Pre-requisites: a parent class (ParentClassTest) with two child classes (ChildClassTest and SecondChildClassTest).
The inherited @test methods from the base class should be ignored in a child class decorated with the @ignore annotation and should run in the child class that doesn't have the annotation
Actual behavior
Is the issue reproducible on runner?
Test case sample
First child class
Second child class
Contribution guidelines
Incase you plan to raise a pull request to fix this issue, please make sure you refer our Contributing section for detailed set of steps.
The text was updated successfully, but these errors were encountered: