Skip to content

TestNG retries are marked as Skipped and the test itself is not marked as Flaky Test #452

Description

@marvelfrozen

I'm submitting a ...

  • bug report
  • feature request
  • support request => Please do not submit support request here, see note at the top of this template.

What is the current behavior?

Tests that failed and then success after re-run using TestNG retryAnalyzer marked as skipped and not showing the Flaky Test (bomb) icon.
image
image

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem

The retry impl class

public class RetryAnalyzerImpl implements IRetryAnalyzer {
	private int maxRetryCount = 3;
	private int retryCount = 0;

	public boolean retry(ITestResult result) {
		if (retryCount < maxRetryCount) {
			retryCount++;
			return true;
		}
		return false;
	}
}

The test method

@Test(retryAnalyzer= RetryAnalyzerImpl.class)
public void VerifyRetrievePublicaAndActiveCommunity(){
	int i = ThreadLocalRandom.current().nextInt(0, 2);
	System.out.println(i);
	Assert.assertEquals(i,0);
}

What is the expected behavior?

The retries are marked as Failed correctly. And the tests are marked as Flaky.

What is the motivation / use case for changing the behavior?

Please tell us about your environment:

| Allure version | 2.13.3 |
| Test framework | testng@7.1.0 |
| Allure integration | allure-testng@2.13.3 |
| Generate report using | allure windows cli@2.13.3 |

Other information

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions