[SUREFIRE-2298] @Nested test classes are not reported properly - ASF Jira

XML

Word

Printable

JSON

Details

Type: Bug
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: 3.5.3
Fix Version/s: None
Component/s: None
Labels:
None

Description

Given this example test you can see that the @Nested test classes are not reported correctly. In 3.5.2 and earlier all executions land in the TEST-BaseNestedTest$Inner.xml and with the changes in 3.5.3 they are distributed in all files. However, the nested classes belong to the enclosing class and the reporting should reflect that, so it would be more accurate to report the nested executions as part of the enclosing tests (see below).

marcphilipp please chime in.

import org.junit.jupiter.api.*;

abstract class BaseNestedTest {
    @Test
    void outerTest() {
    }
    
    @Nested
    class Inner {

        @Test
        void innerTest() {
        }
    }
}

class FirstNestedTest extends BaseNestedTest {
}

class SecondNestedTest extends BaseNestedTest {
}

This was most likely caused by https://github.com/apache/maven-surefire/commit/5aeca19716dedc4cbcab8d68dfc74a99f50ce9ee

3.5.2

TEST-FirstNestedTest.xml

<?xml version="1.0" encoding="UTF-8"?>
<testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd" version="3.0.2" name="FirstNestedTest" time="0.019" tests="0" errors="0" skipped="0" failures="0">
</testsuite>

TEST-SecondNestedTest.xml

 <?xml version="1.0" encoding="UTF-8"?>
<testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd" version="3.0.2" name="SecondNestedTest" time="0.004" tests="0" errors="0" skipped="0" failures="0">
</testsuite>

TEST-BaseNestedTest$Inner.xml

 <?xml version="1.0" encoding="UTF-8"?>
<testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd" version="3.0.2" name="BaseNestedTest$Inner" time="0.001" tests="2" errors="0" skipped="0" failures="0">
  <testcase name="outerTest" classname="FirstNestedTest" time="0.007"/>
  <testcase name="innerTest" classname="BaseNestedTest$Inner" time="0.0"/>
  <testcase name="innerTest" classname="BaseNestedTest$Inner" time="0.0"/>
  <testcase name="outerTest" classname="SecondNestedTest" time="0.001"/>
</testsuite>

-------------------

3.5.3

TEST-FirstNestedTest.xml

 <?xml version="1.0" encoding="UTF-8"?>
<testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd" version="3.0.2" name="FirstNestedTest" time="0.019" tests="1" errors="0" skipped="0" failures="0">
  <testcase name="outerTest" classname="FirstNestedTest" time="0.007"/>
</testsuite>

TEST-SecondNestedTest.xml

 <?xml version="1.0" encoding="UTF-8"?>
<testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd" version="3.0.2" name="SecondNestedTest" time="0.003" tests="1" errors="0" skipped="0" failures="0">
  </properties>
  <testcase name="outerTest" classname="SecondNestedTest" time="0.001"/>
</testsuite>

TEST-BaseNestedTest$Inner.xml

 <?xml version="1.0" encoding="UTF-8"?>
<testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd" version="3.0.2" name="BaseNestedTest$Inner" time="0.001" tests="1" errors="0" skipped="0" failures="0">
  <testcase name="innerTest" classname="BaseNestedTest$Inner" time="0.001"/>
  <testcase name="innerTest" classname="BaseNestedTest$Inner" time="0.0"/>
</testsuite>

-------------------

Expected reporting

TEST-FirstNestedTest.xml

 <?xml version="1.0" encoding="UTF-8"?>
<testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd" version="3.0.2" name="FirstNestedTest" time="0.019" tests="1" errors="0" skipped="0" failures="0">
  <testcase name="outerTest" classname="FirstNestedTest" time="0.007"/>
  <testcase name="innerTest" classname="BaseNestedTest$Inner" time="0.001"/>
</testsuite>

TEST-SecondNestedTest.xml

 <?xml version="1.0" encoding="UTF-8"?>
<testsuite xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://maven.apache.org/surefire/maven-surefire-plugin/xsd/surefire-test-report.xsd" version="3.0.2" name="SecondNestedTest" time="0.003" tests="1" errors="0" skipped="0" failures="0">
  </properties>
  <testcase name="outerTest" classname="SecondNestedTest" time="0.001"/>
  <testcase name="innerTest" classname="BaseNestedTest$Inner" time="0.0"/>
</testsuite>

Attachments

Issue Links

is caused by

SUREFIRE-1643 JUnit 5 in parallel execution mode confuses Surefire reports

Closed

relates to

SUREFIRE-2299 Missing tests inside Surefire report for Cucumber scenarios

Open

links to

GitHub Pull Request #828

Activity

People

Assignee:: Olivier Lamy

Reporter:: Leonard Brünings

Votes:: 1 Vote for this issue

Watchers:: 8 Start watching this issue

Dates

Created:: 27/Mar/25 13:48

Updated:: 23/Jul/25 10:24