Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(testresult): add skipped message to <skipped> not <failure> #189

Merged
merged 1 commit into from
Jul 19, 2023

Conversation

gibfahn
Copy link
Collaborator

@gibfahn gibfahn commented Jul 19, 2023

Currently skipped tests with messages return
<skipped></skipped><failure message="...">, but all the other examples
of JUnit I can find return <skipped message="...">.

The inclusion of both a <skipped> and <failure> confuses parsers,
making them think that the test failed when it was actually skipped.

Skipping tests was added in
#37.

XML changes

To show the difference

git switch main
bazel test java/test/com/github/bazel_contrib/contrib_rules_jvm/junit5/... &>/dev/null
cat .bazel/testlogs/java/test/com/github/bazel_contrib/contrib_rules_jvm/junit5/AssumptionsJUnit5Test/test.xml

git switch skipped_message
bazel test java/test/com/github/bazel_contrib/contrib_rules_jvm/junit5/... &>/dev/null
cat .bazel/testlogs/java/test/com/github/bazel_contrib/contrib_rules_jvm/junit5/AssumptionsJUnit5Test/test.xml

Before

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
    <testsuite name="com.github.bazel_contrib.contrib_rules_jvm.junit5.AssumptionsJUnit5Test"
    tests="2" failures="1" errors="0" disabled="0" skipped="1"
    package="">
        <properties />
        <testcase name="shouldBeSkipped"
        classname="com.github.bazel_contrib.contrib_rules_jvm.junit5.AssumptionsJUnit5Test"
        time="0">
            <!-- We say skipped here but also failure. -->
            <skipped></skipped>
            <failure message="Assumption failed: assumption is not true"
            type="org.opentest4j.TestAbortedException">
                <![CDATA[org.opentest4j.TestAbortedException: Assumption failed: assumption is not true
                        at org.junit.jupiter.api.Assumptions.throwTestAbortedException(Assumptions.java:258)
                        ...
                        at com.github.bazel_contrib.contrib_rules_jvm.junit5.JUnit5Runner.main(JUnit5Runner.java:39)
                ]]>
            </failure>
        </testcase>
    </testsuite>
</testsuites>

After

<?xml version="1.0" encoding="utf-8"?>
<testsuites>
    <testsuite name="com.github.bazel_contrib.contrib_rules_jvm.junit5.AssumptionsJUnit5Test"
    tests="2" failures="0" errors="0" disabled="0" skipped="1"
    package="">
        <properties />
        <testcase name="shouldBeSkipped"
        classname="com.github.bazel_contrib.contrib_rules_jvm.junit5.AssumptionsJUnit5Test"
        time="0">
            <!-- We just say skipped here, and include the skipped message directly.. -->
            <skipped message="Assumption failed: assumption is not true"
            type="org.opentest4j.TestAbortedException">
                <![CDATA[org.opentest4j.TestAbortedException: Assumption failed: assumption is not true
                        at org.junit.jupiter.api.Assumptions.throwTestAbortedException(Assumptions.java:258)
                       [...]
                        at com.github.bazel_contrib.contrib_rules_jvm.junit5.JUnit5Runner.main(JUnit5Runner.java:39)
                ]]>
            </skipped>
        </testcase>
    </testsuite>
</testsuites>

Commits (oldest to newest)

2ee885a fix(testresult): add skipped message to not

Currently skipped tests with messages return
<skipped></skipped><failure message="...">, but all the other examples
of JUnit I can find return <skipped message="...">.

The inclusion of both a <skipped> and <failure> confuses parsers,
making them think that the test failed when it was actually skipped.

Skipping tests was added in
#37.


Copy link
Collaborator

@illicitonion illicitonion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Currently skipped tests with messages return
`<skipped></skipped><failure message="...">`, but all the other examples
of JUnit I can find return `<skipped message="...">`.

The inclusion of both a `<skipped>` and `<failure>` confuses parsers,
making them think that the test failed when it was actually skipped.

Skipping tests was added in
bazel-contrib#37.
@illicitonion illicitonion merged commit 59d4e1a into bazel-contrib:main Jul 19, 2023
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants