This Tutorial Explains the Different Types of TestNG Annotations and Listeners. You Will Also Learn How to use the TestNG Annotations & Listeners with Examples:
Here, we will run a basic TestNG program using TestNG annotations and also see the role of TestNG Listeners and how to use them in testing.
=> Read Through The Easy TestNG Training Series.
Table of Contents:
What Are TestNG Annotations?
These are the program or business logic that are used in controlling the flow of methods. They play a very important role in TestNG. These annotations differ in each project as per the requirements. The flow of annotations remains the same in every program in spite of the different requirements.
There are different types of TestNG annotations that make TestNG easier and better than JUnit. Each of these Annotations will run at a specific event in the TestNG.
Types Of Annotations In TestNG
Given below are the lists of Annotations and their attributes that are used in TestNG. Let’s explore them and their usages in detail.
Before
- @BeforeSuite: This method will be executed before all the tests in the suite are run.
- @BeforeTest: This method will be executed before each test section is declared in the suite.
- @BeforeClass: This method will be executed before the first test method in the current class.
- @BeforeMethod: This method will be executed before each test method.
- @BeforeGroups: This method will be executed before any test method of the specified group is mentioned.
- @Test: Marks a class or a method as a part of the test. Or we can say that it makes a method as the test method.
After
- @AfterSuite: This method will be executed after all the tests in the suite are run.
- @AfterTest: This method will be executed after each test section is declared in the suite.
- @AfterClass: This method will be executed after the last test method of the class.
- @AfterMethod: This method will be executed after each test method is executed.
- @AfterGroups: This method will be executed after the last test method of the specified group is executed.
Work Flow Of Annotations
When we execute the TestNG.xml file, the TestNG annotations will be executed in the following sequence:
Before Suite-> Before Test-> Before Class->Before Method-> @Test -> After Method-> After Class-> After Test-> After Suite
@BeforeSuite
@BeforeTest
@BeforeClass
@BeforeMethod
@Test
@AfterMethod
@AfterClass
@AfterTest
@AfterSuite
@Test has a lot of other attributes that help us in executing our test cases more effectively.
Below are the types of attributes and their descriptions with examples.
#1) alwaysRun: When set to true, this method will run even if it depends on a method that has failed.
Example: Test (alwaysRun = true)
#2) dataProvider: This shows the name of the data provider for this method. It has attributes only i.e. name. We can use it when we test our software with multiple sets of data at input time or run time. This is also known as Data-driven testing. We can do Data-driven testing using this attribute.
Example: @dataProvider (name=”TestData”)
#3) dataProviderClass: This attribute will allow you to specify the data provider class which will contain the data provider that the @Test method will be using.
Example: @Test(dataProvider = “Client Data Test”, dataProviderClass = ClientDetailsTest.class)
#4) dependsOnGroups: This attribute depends on the list of groups i.e. the test method will start execution only after the dependent groups are executed.
Note: If any of the tests in the groups that are dependent on fails, then it will skip that test.
Below is the Example:
@Test(groups= “homepage”)
public void homepageTest(){
System.out.println("Home Page displayed successfully");
}
@Test(groups= “transactionspage”)
Public void transactionpageTest(){
System.out.println(“Transaction Page displayed successfully”);
}
@Test(dependsOnGroups={“homepage”, “transactionspage”})
public void dependOnGroupTest1(){
System.out.println(“dependency tested successful”);
#5) dependsOnMethods: This annotation depends on the list of methods. This means the test method will start execution only after the dependent methods are executed.
Note: If any of the tests in methods that are dependent fails, it will skip that test.
Example:
@Test
public void loginTest() {
System.out.println(“Login Tested Successfully”);
}
@Test
public void homepageTest() {
System.out.println(“Home Page Tested Successfully”);
}
@Test(dependsOnMethods={“ loginTest”, “homepageTest”})
public void smokeTest() {
System.out.println(“Smoke Tests were done successfully”);
#6) alwaysRun=true: We can set the attributes on a test method to true, and this will force the test to be executed even if some of the tests in the group depend upon fail.
For Example:
@Test
public void launchAppTest() {
System.out.println(“Application launched Successfully”);
}
@Test
public void loginAppTest() {
System.out.println(“Application logged in Successfully”);
}
@Test(dependsOnMethods={“launchAppTest”, “loginAppTest”}, alwaysRun=true)
public void smokeTest() {
System.out.println(“Smoke Test were done successfully”);
}
#7) description: This provides the description for the method. Generally, it contains a one-line summary.
Example:
@Test(description = “Regression Test Summary”)
#8) enabled: This attribute helps in specifying if we want to run/execute the particular test method in the current suite/class or not. Sometimes we don’t want to run a few tests due to some reasons like the requirement/function is frequently changing and we don’t want to disturb the current run for that particular function.
In those cases, we can simply ignore/disable that particular test by setting this feature as @Test(enabled = false).
Example:
@Test(enabled = false)
public void imageTest() {//We have disabled this test by giving enabled=false
System.out.println(“Image was tested successfully()”);
}
#9) expectedExceptions: This attribute shows the list of exceptions that the test method is going to throw in the run time. If no exceptions or any other exception is thrown for the method, then the test is marked as a failure.
Example:
@Test(expectedExceptions = ArithmeticException.class)
public void numericTest() {
int i = 1 / 0;
}
#10) groups: This attribute is used to specify the groups that the test method belongs to.
@Test(groups = {“Regression”})
Public void runRegressionTest(){
System.out.println(“test runs were successful”);
}
#11) priority: This helps in prioritizing the test methods, while the default priority starts with 0 and the tests are executed in the ascending order.
Example:
@Test
Public void launchApp(){
System.out.println(“Application was launched successfully”):
}
@Test (priority = 1)
Public void loginApp(){
System.out.println(“Application logged in successfully”);
}
@Test (priority = 2)
Public void checkTrans(){s
System.out.println(“Checked Transactions successfully”);
}
Results: Below are the results as per priority, even though no number was assigned to the first test, it took its priority by default as 0 and the execution was done in ascending order.
Passed: launchApp
Passed: loginApp
Passed: checkTrans
#12) timeout: This attribute helps in specifying a timeout value for the test (generally used as milliseconds). If the test takes more than the specified time out value, then the test is marked as fail. We can use this timeout for doing a performance test, to make sure that the method is returning within a reasonable time.
@Test(timeOut = 500)
public void timeTest() throws InterruptedException {
Thread.sleep(400);
System.out.println(“Time test method successfully tested”);
}
#13) invocationCount: This attribute helps in determining the number of times a test method should be invoked.
@Test(invocationCount = 5)
public void loginTest() {
WebDriver driver = new FirefoxDriver();
driver.get("http://www.google.com");
System.out.println("Page Title is " + driver.getTitle());
driver.quit();
Output:
Page Title is Google
Page Title is Google
Page Title is Google
Page Title is Google
Page Title is Google
#14) invocationTimeOut: This is the maximum time (no. of milliseconds) that this test should take for all the invocation counts. This method has to be used along with the invocation count method or else it would be ignored.
Example:
@Test(invocationCount=4, invocationTimeOut=4000)
public void loginTest(){
Thread.sleep(1000);
System.Out.println(“login Test”);
}
The above example shows that this test will take a total of 4 seconds to execute and each time the test is invoked/run, it would take 1 second to execute.
#15) @DataProvider: This method helps in supplying data for a test method. First, we have to declare a method annotated by @DataProvider and then use this method in the required test method using the “DataProvider” attribute in the @Test annotation.
A Data provider returns an array of objects especially a two-dimensional object array [][]. The first array represents a data set and the second array contains the parameters.
@DataProvider(name = “Test”) – Here, the name represents the Data Provider’s name. If the name isn’t given, then, the DataProvider’s name will automatically set to the method name.
Example:
@DataProvider(name = “Name”)
public object[][] credentials(){
return new object [][] { { “Mohan”, “23”}, { “Shikhar”, “30”} };
}
//Now we are calling the Data Provider object by its name
@Test(DataProvider = “Name”)
Public void testData(String sName, int age) {
System.out.println(“Data is: (Name, age)”);
}
#16) @Factory: This is used to specify a method as a factory for providing objects that are to be used by TestNG for its test classes. By using @Factory, we can create dynamic tests at run time and it should return an array object.
Example:
Let’s take an example to understand it better. We will create two classes i.e. FactorySample.Java and FactoryTest.Java
FactorySample.Java
public class FactorySample {
@Test
public void googleTest() {
System.out.println(“Google was launched successfully”);
}
@Test
public void gmailLogin() {
System.out.println(“Gmail logged in successfully”);
}
FactoryTest.Java
public class FactoryTest {
@Factory()
public Object[] testFact() {
FactorySample fs = new FactorySample[2];
fs[0] = new googleTest();
fs[1] = new gmailLogin();
return fs;
}
}
Output: Google was launched successfully
Gmail logged in successfully
Difference Between @Factory And @DataProvider Annotations
There is a basic difference between both the annotations. There is a lot of confusion regarding these two annotations like where to use these and why?
Let’s find the answers.
@DataProvider: This annotation will parameterize the particular test method and execute the test in a specific no. of times based on the data provided by this method.
For example, if there are two parameters, then the test method will be executed twice. For instance, if we want to login to a site with different sets of usernames and passwords each time, then this is useful as we have to provide the parameters to test.
@Factory: This will execute all the test methods present inside the test class file while using a separate instance of that class. This is useful if we want to run the test class any number of times.
For example, if we have to test the login function of any application or website and as we have to run this test multiple times, it’s better to use @Factory where we can create multiple instances of test and run the tests.
Let’s take a look at these examples to know the difference.
@DataProvider Example:
@DataProvider
public Object[][] message(){
return new Object [][]{{“Mihir” , new Integer (145632)}, {“Kumar”, new Integer (28242)}};
}
@Test (dataProvider=”message”)
public void PrintMsg(String name, Integer id){
System.out.println(“Names are: “+name+” “+id);
}
Note: In the above program we have provided two data and the program result would be:
Names are: Mihir 145632
Names are: Kumar 28242
This shows that if we increase the number of data in the message method, then the print method will execute the same number of times.
@Factory Example:
TestNG Factory is very useful when we have to run multiple test classes using a single test class.
Let’s see an example.
For this, we have to create two test classes with few test methods inside them.
TestData 1:
public class TestData1 {
@Test
public void testData1() {
System.out.println("Test data 1 successfully tested");
}
}
TestData 2:
public class TestData2 {
@Test
public void testData2() {
System.out.println("Test data 2 successfully tested");
}
}
Now we have to define the @Factory method which returns an object array of the above-defined classes.
Factory Program:
public class TestNGFactory {
@Factory()
public Object[] getTestClass() {
Object[] tests = new Object[2];
tests[0] = new Test Data 1();
tests[1] = new Test Data 2();
return tests;
}
}
Output:
Test1 test method
Test2 test method
PASSED: test1
PASSED: test2
TestNG Listeners With Types
In simple terms, Listeners listen to the event defined in the Selenium script and behave accordingly to it. The main purpose is to create logs and customize the TestNG reports.
There are many types of listeners available in TestNG.
For example, IAnnotationTransformer, IAnnotationTransformer2, IConfigurable, IConfigurationListener, IConfigurationListener2, IExecutionListener, IHookable, IInvokedMethodListener, IInvokedMethodListener2, IMethodInterceptor, IReporter, ISuiteListener, ITestListener.
However, when it comes to testing, we only use a few of them as discussed below:
#1) ISuiteListener
This is a listener for test suites. It consists of two methods i.e. onStart() and onFinish().
Whenever we implement this listener, it will guarantee that the end-user will invoke the methods onStart() and onFinish() before and after running a TestNG suite.
Method details:
void onStart(ISuite suite): This method is invoked before the Suite Runner starts.
void onFinish(ISuite suite): This method is invoked after the Suite Runner has run all the test suites.
Example:
@Override
public void onStart(ISuite suite) {
System.out.println(“TestNG Suite Starts”);
}
@Override
public void onFinish(ISuite suite) {
System.out.println(“TestNG Suite Finishes”);
}
#2) ITestListener
This listener works just like the ISuiteListener. However, the only difference is that it makes the call before and after the Test and not the Suite. It is a listener for test running and this listener has seven methods in it.
(i) onStart(): This method is invoked after the test class is instantiated and before any configuration method is called.
Example:
@Override
public void onStart(ITestContext context) {
System.out.println(“Context Name = ” + context.getName());
}
(ii) onFinish(): This method is invoked after all the tests have run and all the configuration methods are called.
Example:
public void onFinish(ITestContext context) {
System.out.println(context.getPassedTests());
}
(iii) onTestStart(): This method is invoked each time before a test will be invoked. The ITestResult is only partially filled with the references to class, method, start millis and status.
Method: void onTestStart(ITestResult result)
Example:
@Override
public void onTestStart(ITestResult result) {
System.out.println("Test Started…"+result.getStartMillis());
}
(iv) onTestSuccess(): This method is invoked each time when a test succeeds.
Method: void onTestSuccess(ITestResult result)
Example:
@Override
public void onTestSuccess(ITestResult result) {
System.out.println("Test Success. "+result.getEndMillis());
}
(v) onTestFailure(): This method is invoked each time when a test fails.
Method: void onTestFailure(ITestResult result)
Example:
@Override
public void onTestFailure(ITestResult result) {
System.out.println("Test Failed. "+result.getTestName());
}
(vi) onTestSkipped(): This method is invoked each time when a test is skipped.
Method: void onTestSkipped(ITestResult result)
Example:
@Override
public void onTestSkipped(ITestResult result) {
System.out.println("Test Skipped. "+result.getTestName());
}
(vii) onTestFailedButWithinSuccessPercentage: This method is invoked each time when a method is failed but has been annotated with success percentage and the failure keeps it within the success percentage.
Method: void onTestFailedButWithinSuccessPercentage(ITestResult result)
Example:
@Override
public void onTestFailedButWithinSuccessPercentage(ITestResult iTestResult) {
System.out.println("Test failed but it is in defined success ratio " + getTestMethodName(iTestResult));
}
#3) IExecutionListener
It is a listener that monitors the beginning and end of a TestNG run. It has two methods i.e. onExecutionStart() and onExecutionFinish().
onExecutionStart() method is called before the TestNG starts running the suites and the onExecutionFinish() method is called after TestNG is done with the running of all the test suites.
Method:
void onExecutionStart()
void onExecutionFinish()
Example:
@Override
public void onExecutionStart() {
System.out.println("TestNG is going to start");
}
@Override
public void onExecutionFinish() {
System.out.println("TestNG is finished");
}
#4) IInvokedMethodListener
It is a listener that gets invoked before and after a method is invoked by TestNG. This listener is invoked only for configurations and test methods. It has only two methods in it i.e. afterInvocation and beforeInvocation.
- beforeinvocation: Invoke before each method.
- afterInvocation: Invoke after each method.
Method:
void beforeInvocation(IInvokedMethod method, ITestResult testResult)
void afterInvocation(IInvokedMethod method, ITestResult testResult)
Example:
@Override
public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {
System.out.println("before invocation of " + method.getTestMethod().getMethodName());
}
@Override
public void afterInvocation(IInvokedMethod method, ITestResult testResult) {
System.out.println("after invocation of " + method.getTestMethod().getMethodName());
}
#5) IMethodInterceptor
This class is used to alter the list of test methods that TestNG is going to run. By using this method we can reorder the list of test methods.
It is applicable only to those methods that have no dependents and those methods that don’t depend on any other test methods will be passed in parameters. This interface will return a list of test methods that need to be run but in a different sorted manner.
Method:
java.util.List<IMethodInstance> intercept(java.util.List<IMethodInstance> methods, ITestContext context)
Example:
@Override
public List<IMethodInstance>intercept(List<IMethodInstance>methods,
ITestContext context) {
List<IMethodInstance> result = new ArrayList<IMethodInstance>();
for (IMethodInstance m : methods) {
Test test = m.getMethod().getMethod().getAnnotation(Test.class);
Set<String>groups = new HashSet<String>();
for (String group : test.groups()) {
groups.add(group);
}
if (groups.contains("sanity")) {
result.add(m);
} else {
String testMethod = m.getMethod().getMethodName();
System.out.println(testMethod + " not a SANITY test so not included");
}
}
return result; }
#6) IReporter
This is implemented by the clients to generate a report. This method will be invoked once all the suite has run and the parameters give all the test results that happened during that run.
Method:
void generateReport(java.util.List<XmlSuite> xmlSuites, java.util.List<ISuite> suites, java.lang.String outputDirectory)
Example:
@Override
public void generateReport(List<XmlSuite> xmlSuites, List<ISuite> suites,
String outdir) {
try {
writer = createWriter(outdir);
} catch (IOException e) {
System.err.println("Unable to create output file");
e.printStackTrace();
return;
}
startHtml(writer);
writeReportTitle(reportTitle);
generateSuiteSummaryReport(suites);
generateMethodSummaryReport(suites);
generateMethodDetailReport(suites);
endHtml(writer);
writer.flush();
writer.close();
}
Conclusion
In this article, we have seen how TestNG Annotations can be useful in making our Program logic easier. Annotations are used as needed.
You can pass the parameters to the annotations and do Data-Driven Testing as well. You can run the test cases in groups and save time. With listeners, you can even generate the reports. Don’t you think this is wonderful?
TestNG.xml will be explained in detail in our upcoming tutorial. This XML file is the backbone of the TestNG framework and it will help us in executing our test cases.
=> Check Out The Perfect TestNG Training Guide Here.