Hi,
I have weird behaviour with the following test file:
TEST_GROUP(Validation_ShouldFail)
{
};
TEST_GROUP(Validation_ShouldPass)
{
};
TEST(Validation_ShouldFail, ShouldFail1)
{
CHECK(false);
}
TEST(Validation_ShouldPass, ShouldPass1)
{
CHECK(true);
}
TEST(Validation_ShouldPass, ShouldPass2)
{
CHECK(true);
}
TEST(Validation_ShouldFail, ShouldFail2)
{
CHECK(false);
}Running these tests with -ojunit, I get the following output files:
- A
cpputest_Validation_ShouldPass.xml, which looks as expected:
<?xml version="1.0" encoding="UTF-8" ?>
<testsuite errors="0" failures="0" hostname="localhost" name="Validation_ShouldPass" tests="2" time="0.002" timestamp="*censored*">
<properties>
</properties>
<testcase classname="Validation_ShouldPass" name="ShouldPass2" assertions="1" time="0.000" file="*censored*.cpp" line="22">
</testcase>
<testcase classname="Validation_ShouldPass" name="ShouldPass1" assertions="1" time="0.000" file="*censored*.cpp" line="17">
</testcase>
<system-out></system-out>
<system-err></system-err>
</testsuite>
- A
cpputest_Validation_ShouldFail.xml, which only contains one test result:
<?xml version="1.0" encoding="UTF-8" ?>
<testsuite errors="0" failures="1" hostname="localhost" name="Validation_ShouldFail" tests="1" time="0.003" timestamp="*censored* ">
<properties>
</properties>
<testcase classname="Validation_ShouldFail" name="ShouldFail2" assertions="1" time="0.000" file="*censored*.cpp" line="27">
<failure message="*censored*.cpp:29: CHECK(false) failed" type="AssertionFailedError">
</failure>
</testcase>
<system-out></system-out>
<system-err></system-err>
</testsuite>
I tracked down the problem, and found that it lays in the test order:
TEST(Validation_ShouldFail, ShouldFail1);
TEST(Validation_ShouldPass, ShouldPass1);
TEST(Validation_ShouldPass, ShouldPass2);
TEST(Validation_ShouldFail, ShouldFail2);
Interleaving the tests leads to CppUTest not recognizing the groups correctly.
I assume that this behavior is not intended, as it also contradicts
|
TEST(TestRegistry, listTestGroupNames_shouldListBackwardsGroup1AfterGroup11AndGroup2OnlyOnce) |
It is caused by this line:
As the tests not executed in group order,
endOfGroup(test) becomes true three times (instead of two).
Hi,
I have weird behaviour with the following test file:
Running these tests with
-ojunit, I get the following output files:cpputest_Validation_ShouldPass.xml, which looks as expected:cpputest_Validation_ShouldFail.xml, which only contains one test result:I tracked down the problem, and found that it lays in the test order:
Interleaving the tests leads to CppUTest not recognizing the groups correctly.
I assume that this behavior is not intended, as it also contradicts
cpputest/tests/CppUTest/TestRegistryTest.cpp
Line 331 in 81eb8b8
It is caused by this line:
cpputest/src/CppUTest/TestRegistry.cpp
Line 67 in 81eb8b8
As the tests not executed in group order,
endOfGroup(test)becomes true three times (instead of two).