In this Tutorial, we will Understand the Feature of Repeating a Test Multiple Times using JUnit 5 Annotation @RepeatedTest With Examples:
We learned how to set the order of test cases using specific annotations as well as specific classes in our previous tutorial.
We also explored the ways to order tests for JUnit 4 and JUnit 5, based on which the ordering strategies also changed. We saw how to create a customized sorting class for ordering the test cases during their execution.
In this tutorial, we will learn about a feature supported by JUnit 5 through which we can repeat a test for a specific number of times.
=> Read Through The Easy JUnit Training Series.
We will learn about different APIs or interfaces that would help us to fetch details about the repeated tests or about the annotation used to repeat the tests.
Table of Contents:
JUnit 5 Annotation @RepeatedTest
Here, in this section let’s discuss the meaning of the term “repeated test” and see when we could use it.
What Is Repeated Test?
The term ‘repeated test’ means to repeat the execution of specific tests or tests for a certain number of times. The same purpose can be achieved in parameterization too.
However, the only difference between the parameterized test and the repeated test is that the parameterized tests work with data variations while the repeated tests work with the same data.
Compared to the feature of parameterization, the repeated test is a feature that is rarely used.
When do we use the Repeated Test?
If you have to point out a favorable use case for implementation of the repeated tests, then you could think of a scenario where you are automating an application and on navigating to a specific page or an explorer link; results to an environmental error that tends to fail your tests.
For Example: Clicking on ‘Payment’ link for a particular banking application shows “Exception: Page cannot be displayed” and this environmental disappears after 4 clicks. This is a consistent and known issue for the client.
Now, we know that we cannot afford to have tests failed for such known environmental issues that would not be accepted by the client as real defects.
Thus, in this case, of course, either we could build the code logic within the test itself to handle the clicking of the Payment link 4 times and have our test proceed further without failing or we could better take the advantage of the repeated test feature supported by JUnit 5.
@RepeatedTest: Syntax & Associated Rules
@RepeatedTest annotation is used to repeat a test for a specific number of times.
The basic syntax of @RepeatedTest is as follows:
- The annotation @RepeatedTest is used for a test method instead of the annotation @Test.
- A number is passed as the input parameter of the annotation.
- Then that specific annotated test method repeats the execution for that many numbers of times.
We will see the usage of the annotation in the examples in the below sections.
The rules associated with @RepeatedTest is as follows:
- The methods annotated with @RepeatedTest cannot be static.
- The methods annotated with @RepeatedTest cannot be private.
- The return type of the method annotated with @RepeatedTest must be void only.
@RepeatedTest: Basic Example
Let’s have a look at the most basic example of implementing the annotation @RepeatedTest in a JUnit 5 test.
#1) With An Integer Value As In Input Parameter
The simplest way of using the annotation @RepeatedTest is by passing integer value as input parameter E.g. @RepeatedTest(5) – This runs the annotated method 5 times.
Code for RepeatedTestExample.java
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.RepeatedTest;
import org.junit.jupiter.api.RepetitionInfo;
import org.junit.jupiter.api.TestInfo;
public class RepeatedTestExample {
@DisplayName ("Testcase for Multiplication")
@RepeatedTest (5)
void multiply (TestInfo testInfo)
{
int a, b;
a=10;
b=20;
assertEquals(200, (a*b), "Matched. Test status - Passed");
}
}
Result:
The Report tab of the JUnit resultant shows that the test method multiply() is repeated 5 times during execution.
repetition 1 of 5(multiply(TestInfo))
repetition 2 of 5(multiply(TestInfo))
repetition 3 of 5(multiply(TestInfo))
repetition 4 of 5(multiply(TestInfo))
repetition 5 of 5(multiply(TestInfo))
#2) With The Display Name Patterns
In order to understand the format of usage of the annotation, we first need to understand the placeholders supported by it.
CURRENT_REPETITION_PLACEHOLDER:
This is a static String placeholder that returns the count of the current repetition of the annotation. If the annotated method is repeating for the second time, then the placeholder returns 2 as the value.
@RepeatedTest(value=2, name={currentRepetition}) is how you use this placeholder in the annotation.
TOTAL_REPETITIONS_PLACEHOLDER:
This is a static String placeholder that returns the count of the total repetitions of the annotation. If the value parameter in @RepeatedTest is set to 5, then the placeholder gives 5 as the value as this would be the count of the total number of times the test would repeat its execution.
@RepeatedTest(value=2, name={totalRepetitions}) is how you use this placeholder in the annotation.
DISPLAY_NAME_PLACEHOLDER
This is a static String placeholder that returns the String value passed as a parameter in the annotation @DisplayName for the method. If the annotation @DisplayName is not present for the method, it returns the method name for the placeholder.
@RepeatedTest(value=2, name={displayName}) is how you use this placeholder in the annotation.
Being aware of all the placeholders used for the annotation, let’s try to understand the display patterns it supports. The annotation supports two static display name patterns for the String element “name” as the input parameter.
They are as shown below:
#1) SHORT_DISPLAY_NAME
This is the placeholder set by default.
When you use the input parameter for @RepeatedTest as name=RepeatedTest.SHORT_DISPLAY_NAME, the resultant in JUnit tab displays in the below pattern:
repetition {currentRepetition} of {totalRepetitions}
Example:
public class RepeatedTestExample {
@RepeatedTest (value=3, name= SHORT_DISPLAY_NAME)
void multiply ()
{
int a, b;
a=10;
b=20;
assertEquals(200, (a*b), "Matched. Test status - Passed");
}
}
Result is as shown below:
#2) LONG_DISPLAY_NAME
When you use the input parameter for @RepeatedTest as name=RepeatedTest.LONG_DISPLAY_NAME, the resultant in JUnit tab displays in the following pattern:
{displayName}:: repetition {currentRepetition} of {totalRepetitions}
Example # 1 for LONG_DISPLAY_NAME (with @DisplayName):
public class RepeatedTestExample {
@DisplayName(“Testcase for Multiplication”)
@RepeatedTest (value=3, name= LONG_DISPLAY_NAME)
void multiply ()
{
int a, b;
a=10;
b=20;
assertEquals(200, (a*b), "Matched. Test status - Passed");
}
}
Result is shown in the below pattern {displayName}:: repetition {currentRepetition} of {totalRepetitions}
Here, {displayName} returns the name passed in the annotation @DisplayName
Example # 2 for LONG_DISPLAY_NAME (without @DisplayName):
public class RepeatedTestExample {
@RepeatedTest (value=3, name= LONG_DISPLAY_NAME)
void multiply ()
{
int a, b;
a=10;
b=20;
assertEquals(200, (a*b), "Matched. Test status - Passed");
}
}
Result is shown in the below pattern {displayName}:: repetition {currentRepetition} of {totalRepetitions}
Here, {displayName} returns the method name.
Customized Display Pattern:
We saw the static ways of the display patterns for the method annotated with @RepeatedTest.
We can also set a customized display pattern using the text values along with the supported placeholders.
Example:
public class RepeatedTestExample {
@DisplayName(“Multiplication”)
@RepeatedTest (value=3, name= “Testcase: {displayName} _ The current repetition count is {currentRepetition}
out of Total repetition count {totalRepetitions})
void multiply ()
{
int a, b;
a=10;
b=20;
assertEquals(200, (a*b), "Matched. Test status - Passed");
}
}
Result in the customized pattern as per the code is shown below:
@RepeatedTest: TestInfo Interface
TestInfo is an interface that fetches details about the test method be it method annotated with @Test, @RepeatedTest, or a parameterized test or the lifecycle call back method like the ones annotated with @BeforeEach, @AfterEach, @BeforeAll, @AfterAll.
Below are the methods that help to fetch details about the test:
#1) TestInfo.getDisplayName(): Returns the display name of the test in which it is invoked. If the @DisplayName annotation was used, the value in it is returned, else the method name is returned. The return type of the method is String.
#2) TestInfo.getTags(): The return type is a Set of String values. It returns all the tags associated with the test method.
#3) TestInfo.getTestClass(): It works like the java reflection concept. It returns the class name of the test method in which it is invoked and the return type is of Class. TestInfo.getTestClass().get().getSimpleName() returns String type class name.
#4) TestInfo.getTestMethod(): It is the container for the test method and the return type is of Method class. TestInfo.getTestMethod().get().getName() returns String type method name.
@RepeatedTest: RepetitionInfo Interface
RepetitionInfo is an interface that fetches details about the repetition of the test method.
Enlisted below are the methods that help to fetch details about the test:
- RepetitionInfo.getCurrentRepetition() – Returns the current repetition of the method annotated with @RepeatedTest. This is exactly what the placeholder {currentRepetition} does.
- RepetitionInfo.getTotalRepetitions() – Returns the total number of repetitions of the method annotated with @RepeatedTest. This is exactly what the placeholder {totalRepetitions} does.
@RepeatedTest: RepeatedTest Interface
RepeatedTest is an interface that fetches details about the annotation @RepeatedTest of the test method.
Enlisted below are the methods to help fetch details about the test:
RepeatedTest.name(): Returns the name value passed as the parameter in the annotation @RepeatedTest.
RepeatedTest.value(): Returns the value passed as the parameter in the annotation @RepeatedTest.
Example To Demo Usage Of The Interfaces
Let’s look at an example that demonstrates the implementation of the three interfaces – TestInfo, RepetitionInfo and RepeatedTest and their respective methods
@RepeatedTest(value=2, name=SHORT_NAME_DISPLAY)
@DisplayName("This is my display name")
@Tag("smoke")
public void test1(TestInfo testInfo, RepetitionInfo Rinfo, RepeatedTest RTinfo) {
System.out.println(“*** Test Details ***”);
System.out.println(“Display name of the test:”+testInfo.getDisplayName());
System.out.println(“Tag:”+testInfo.getTags());
System.out.println(“Test method name:”+testInfo.getTestMethod().get().getName());
System.out.println(“***Repetition Details***”);
System.out.println(“Current Repetition:” + Rinfo.getCurrentRepetition());
System.out.println(“Total Repetitions:” + Rinfo.getTotalRepetitions());
System.out.println(“***Annotation Details***”);
System.out.println(“Name value of @RepeatedTest annotation:” + RTinfo.name());
System.out.println(“Value of @RepeatedTest annotation:” + RTinfo.value());
}
The result on the console window is as shown below:
Assuming this is the first run of the test method
FAQs On Repeated Test
Q #1) Most of us are curious if @RepeatedTest replaces @Test, how is the lifecycle workflow that the test follows.
Answer: Using @RepeatedTest works exactly like that of the test annotated with @Test.
For instance, if a test method is annotated to repeat 3 times, then the test is considered as 3 different test methods.
In other words, the method annotated with @BeforeEach runs for each repetition of the test method. Likewise, the method annotated with @AfterEach runs after each repetition of the test method.
public class RepeatedTestExample {
@BeforeAll
public static void preClass()
{
System.out.println("@BeforeAll- This is the preClass() method that runs one time before the class");
}
@BeforeEach
public void setUp ()
{
System.out.println("@BeforeEach- This is the setUp() method that runs before each testcase");
}
@DisplayName ("Testcase for Multiplication")
@RepeatedTest (2)
void multiply (TestInfo testInfo)
{
int a, b;
a=10;
b=20;
System.out.println(“Testcase for Multiplication”);
assertEquals(200, (a*b));
}
@DisplayName ("Testcase for Division")
@RepeatedTest (3)
void divide (TestInfo testInfo)
{
int a, b;
a=200;
b=20;
System.out.println(“Testcase for Division”);
assertEquals(10, (a/b));
}
@AfterEach
public void tearDown()
{
System.out.println("@AfterEach- This is the tearDown() method that runs after each testcase");
}
@AfterAll
public static void postClass()
{
System.out.println("@AfterAll-This is the postClass() method that runs one time after the class");
}
}
Result:
The JUnit tab shows that the method multiply() repeats 2 times and divide() repeats 3 times.
The Console window shows the below:
- The method annotated @BeforeAll runs as the first method once for the class and the method under @AfterClass runs once at the end of all the test’s run.
- The method annotated with @BeforeEach runs prior to the execution of the testcase multiply() and then, the method @AfterEach runs after the execution of the test case.
- This repeats for the next instance of the method multiply() and also repeats for each run of the method @divide().
@BeforeAll – This is the preClass() method that runs one time before the class
@BeforeEach – This is the setUp() method that runs before each testcase
Testcase for Multiplication
@AfterEach – This is the tearDown() method that runs after each testcase
@BeforeEach – This is the setUp() method that runs before each testcase
Testcase for Multiplication
@AfterEach – This is the tearDown() method that runs after each testcase
@BeforeEach – This is the setUp() method that runs before each testcase
Testcase for Division
@AfterEach – This is the tearDown() method that runs after each testcase
@BeforeEach – This is the setUp() method that runs before each testcase
Testcase for Division
@AfterEach – This is the tearDown() method that runs after each testcase
@BeforeEach – This is the setUp() method that runs before each testcase
Testcase for Division
@AfterEach – This is the tearDown() method that runs after each testcase
@AfterAll – This is the postClass() method that runs one time after the class
Q #2) Is the annotation @RepeatedTest supported in JUnit 4? If not, then is there any other way, to achieve the same in JUnit 4?
Answer: The annotation @RepeatedTest is NOT supported by JUnit 4. It doesn’t have any such feature to repeat a specific test too.
There is an alternative to achieve the same. We may run all the tests in a class for a certain number of times by using the parameterization of tests, in JUnit 4. This is something we will explore in our upcoming tutorial on “Creating Parameterized Test”.
Q #3) Do other Annotations work on the top of @RepeatedTest Annotation?
Answer: Another question that lingers in the minds of many is that do other annotations that work well with @Test work good with @RepeatedTest. The answer to this is yes, it works just as good as it works with @Test.
For the method annotated with @RepeatedTest, you may use @Tag to filter the tests with specific tags, @DisplayName to add a customized name to the test method, @Order to set priority for execution. These were few annotations that go well with @RepeatedTest, however, there are many more annotations that work well.
In short, you may use any annotation that you know would work well with @Test annotation.
Q #4) Can @RepeatedTest and @ParameterizedTest annotation be combined?
Answer: Let’s try rephrasing the question. The question is – can a test be parameterized with different data variations and at the same time repeat them to have multiple runs.
Given below is a quick example to clearly understand it. The code might look something like the below snippet.
Suppose, there are 3 payment types for E.g.: “Cheque”, “ECS”, “Cash” which goes into the parameterized data of paymentType.
Then the expected result is:
- The testcase runs with variation of the payment type.
- For each payment type, the testcase also repeats 3 times.
testPayment runs for paymentType=cheque thrice, paymentType=ECS thrice and paymenttype=cash thrice.
@RepeatedTest(3)
@ParameterizedTest
@MethodSource(names=”paymentType”)
void testPayment(){
}
This was all about simplifying what the question implied. Now coming to the answer to the question. Is this kind of implementation using @RepeatedTest and @ParameterizedTest valid or workable?
The answer is certainly No. You cannot combine these two annotations together. This might confuse many of us, as it conflicts with what we discussed in the third question.
Here, we need to get our concepts clear. In question 3, we saw that whichever annotation works well with @Test works equally well with @RepeatedTest, too. So, what is different from @ParameterizedTest annotation that it does work well with @RepeatedTest. Let’s try to understand this.
@ParameterizedTest or @RepeatedTest are used in the place of @Test annotation.
@Test, @ParameterizedTest, and @RepeatedTest are all a part of an interface named TestTemplate. In other words, these three annotations belong to the annotation @TestTemplate. There is a separate algorithm associated with each of these annotations to facilitate different purposes. Hence, either of them can be used for a testcase.
JUnit doesn’t allow using them together for a test as JUnit fails to understand what algorithm the test should run on.
If you try to run such a code where @RepeatedTest and @ParameterizedTest are combined, the test would result in failure and the error message would read “No ParameterResolver registered”.
Hope, this clears the confusion among which annotations should be used with @RepeatedTest and which ones are not to be used.
Conclusion
In this tutorial, we covered every aspect of the feature of Repeating a test multiple times with the same data. We learned the various ways to implement it. We also tried to answer significant questions relating to the annotation.
=> Check ALL JUnit Tutorials Here