JUnit Vs TestNG – What Are The Differences

A comprehensive comparison between JUnit Vs TestNG frameworks. Includes comparison of annotations and featurewise comparison with examples:

In the previous tutorial, we learned DisplayName annotation and conditional test execution based on different criteria like JRE version, environmental variables, etc. We addressed some important questions around the topic as well.

Since we have been continuously learning about JUnit in previous tutorials, this one will work as a breather for our audience as we will for a while shift our focus from JUnit as a sole agenda to the comparison between JUnit vs TestNG.

=> Check Out The Perfect JUnit Training Guide Here.

JUnit Vs TestNG : A Comparison

Comparison Of Annotations

TestNG and JUnit both are unit testing frameworks from the world of Java. Both implement very closer and similar features. In this section, we shall look at some similarities in implementing a couple of features while we would also get to see a few other features that are implemented differently in JUnit and TestNG.

#1) Test method annotation

There is no difference in the way we specify a method as a test method in both JUnit and TestNG.

#2) Suite related annotation

• A method with annotation @BeforeSuite is executed once before the current test suite runs.
• This annotation is applicable only in TestNG.

#3) Annotation for a method before Class

This is the annotation for the method to be executed once before the first test method in the class is run.

#4) Annotation for a method before Test

• This annotation executes once before the methods declared inside <test> tag of testng.xml.
• This annotation is available for TestNG only.

#5) Annotation for the method to be executed before every method with @Test invokes

#6) Annotation for the method to be executed after every method with @Test invokes

#7) Annotation for method after Test

• This annotation executes once after the methods declared inside <test> tag of testng.xml.
• This annotation is available for TestNG only.

#8) Annotation for method after Class

This is the annotation for the method to be executed once after the last test method in the class is run.

#9) Annotation to disable the execution of the Test method.

• JUnit 5 provides an annotation to disable a specific test execution.
• TestNG provides an attribute for @Test i.e. ‘enabled’ with the boolean value which decides if the execution of the method would be disabled or enabled

Refer to Tutorial 7 Skipping Execution to understand how to disable tests in JUnit4 vs JUnit 5

#10) Timeout annotation

The annotation is same for JUnit 5 and TestNG

#11) Expected Exception attribute

• Exception class states that when the test executes, the exception of the given class is thrown.
• This is supported both in JUnit and TestNG with variation in the way both are declared.

#12) Suite related annotation

• A method with annotation @AfterSuite is executed once after the current test suite runs.
• This annotation is applicable only in TestNG.

#13) Group related annotation

• The annotation is available only in TestNG.
• The method with annotation @BeforeGroups runs before the test methods belonging to a particular group runs.

• The annotation is available only in TestNG.
• The method with annotation @BeforeGroups runs after the test methods belonging to a particular group runs.

#14) Order of Execution related annotations

Both JUnit and TestNG support explicitly setting the order of tests for execution. In other words, setting the priority for test cases.

• JUnit 5 has annotation @TestMethodOrder() with MethodOrderer package’s built-in class – Alphanumeric.class or OrderAnnotation.class or Random.class as an input parameter for the annotation.

Refer to Tutorial 9 – Junit Test Execution Order for more details on setting test execution order in JUnit.

• TestNG includes the attribute ‘priority’ for @Test annotation, which accepts a numerical value.

Basic Program For TestNG and JUnit 4

#1) TestNG Code

package newtest.com;
import org.testng.annotations.Test;
import org.testng.annotations.BeforeMethod;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.DataProvider;
import org.testng.annotations.BeforeClass;
import org.testng.annotations.AfterClass;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeSuite;
import org.testng.annotations.AfterSuite;

public class NewTestng {
  @Test(dataProvider = "dp")
  public void f(Integer n, String s) {
	  System.out.println(" * * * *  * * *Parameterized method *  * * * *  * * * * ");	  
	  System.out.println("Integer "+n+" String "+s);
	  System.out.println(" * * * *  * * * *  * * * *  * * * * ");
  }
  @BeforeMethod
  public void beforeMethod() {
	  System.out.println("Before Method");
  }
@AfterMethod
  public void afterMethod() {
	  System.out.println("After Method");
  }
  @DataProvider
  public Object[][] dp() {
    return new Object[][] {
      new Object[] { 1, "a" },
      new Object[] { 2, "b"},
    };
  }
  @BeforeClass
  public void beforeClass() {
	  System.out.println("Before Class");
  }
  @AfterClass
  public void afterClass() {
	  System.out.println("After Class");
  }
  @BeforeTest
  public void beforeTest() {
	  System.out.println("Before Test");
  }
  @AfterTest
  public void afterTest() {
	  System.out.println("After Test");
  }
  @BeforeSuite
  public void beforeSuite() {
	  System.out.println("Before Suite");
  }
  @AfterSuite
  public void afterSuite() {
	  System.out.println("After Suite");
  }
}

Expected Output:

#2) JUnit 4 Code

package demo.tests;
	import static org.junit.Assert.*;
	import org.junit.After;
	import org.junit.Before;
	import org.junit.Test;
	import org.junit.BeforeClass;
	import org.junit.AfterClass;
	public class JunitTest {
		@Parameterized.Parameters
	    public static Object[][] data() {
	        return new Object[3][0];
	    }
		
		@BeforeClass
		public static void beforeClass() {
			  System.out.println("Before Class";
		  }
		
		@Before
		public void beforeMethod() {
			  System.out.println("Before Method");
		 }
		
		@Test
		public void f() {
			  System.out.println(" * * * *  * * *test *  * * * *  * * * * ");
			  int n=10;
			  System.out.println("Integer "+n);
			  System.out.println(" * * * *  * * * *  * * * *  * * * * ");
		  }

		@After
		public void afterMethod() {
			  System.out.println("After Method");
		  }

		@AfterClass
		public static void afterClass() {
			  System.out.println("After Class");
		 }
	}

Expected Output:

JUnit 5 vs TestNG: Feature Difference With Examples

#1) Test Suite

• The test suite is a collection of tests, which means we are kind of wrapping up multiple test cases from multiple classes together.
• The approach TestNG uses suite is different and powerful compared to that of JUnit.

Test suite in JUnit 5

Let us quickly look at how JUnit 5 applies the Test suite.

Refer to Tutorial 8 -JUnit Test Suites & Filtering Testcases for a better understanding of the implementation of the Test suite in JUnit 4 and in JUnit 5.

@RunWith(JUnitPlatform.class)
@SelectClasses({JUnit5TestCase1.class, JUnit5TestCase2.class })
public class JUnitTestSuite { }

Test suite in TestNG

TestNG uses the XML as the below template to wrap all the logically connecting Test classes

<!DOCTYPE suite SYSTEM "http://beust.com/testng/testng-1.0.dtd" >
<suite name="My test suite">
<test name="testing">
<classes>
<class name="com.bankfair.SuiteTest1" />
<class name="com. bankfair.SuiteTest2" />
</classes>
</test>
</suite>

#2) Parameterized Test

Both TestNG and JUnit allow parameterization of tests which is nothing but running the same tests with data variations.

Parameterized test in JUnit 4

@RunWith(value=Parameterized.class)
public class JUnitclass{
int n;
public JUnitclass (int num){
this.n=num;
}

@Parameters
public static Iterable<Object []> data(){
Object[][] objectArray =new Object[][] {{1},{2},{3}};
returnArrays.asList(objectArray);
}
@Test
public void Junittst(){
System.out.println(“Multiples of 2 are :”+ 2*n);
} 
}

Parameterized test in TestNG

There are 2 ways how you could use parameterization in TestNG

• @Parameters and passing <parameters> through TestNG XML
• Annotation @DataProvider

a) @Parameters and passing <parameters> through TestNG XML

public class testins{
@Test
@Parameters(value=”env_pd”)
public void paramEnv(str env_pd){
If(env_pd=”QA”){
url=”definetest.com”
}	
else if(env_pd=”accpt”){
url=”defineacc.com”
}}}

XML for the same

<!DOCTYPE suite SYSTEM "http://beust.com/testng/testng-1.0.dtd" >
<suite name="testsuite">
<test name="testing">
<parameter name="env_pd" value="accpt"/>    
<classes>
<class name="com.insurance.testins" />
</classes>
</test>
</suite>

b) DataProvider

The DataProvider annotation always returns Object [][] which is the array of objects.

@DataProvider(name="state")
public Object[][] getDataFromDataprovider(){
return new Object[][] 
    {
       { "Maharashtra", "Pune" },
       { "Karnataka", "Bangalore" },
       { "Kerala", "Trivandrum" }
    };

@Test(dataProvider=”state”)
public void paramMethd(str stateName, str cityName){
System.out.println(stateName+” ”+cityName);
}

#3) Timeout

If a particular test doesn’t complete in the stipulated time, then it gets a timeout. In other, the Thread gets interrupted.

Timeout in JUnit

There are different approaches to Timeout implementation in JUnit. These are:

1. Using the usual timeout with specific milliseconds
2. Using timeout with the JUnit assertion
3. Using global timeout

We will have a detailed tutorial focussing on timeout for JUnit 4 and JUnit 5.

Below is the snippet showing the usage of the usual Timeout in JUnit 5:

@Test(timeout = 5000)
public void testTimeout() throws InterruptedException 
{
    while (true)
    {
        
    }
}

The above test timeout after 5 seconds.

Timeout in TestNG

The TestNG also uses the simple way of Timeout implementation:

@Test(timeout = 5000)
public void testTimeout() throws InterruptedException 
{
    while (true)
    {
        
    }
}

#4) Exception Test

Exception test makes sure that when there is this predefined exception thrown, it is gracefully caught and notified on the logs.

Exception Test in JUnit 4

@Test (expected = NumberFormatException.class)  
public void converttoint() 
{  
	Int j=Integer.parseInt(“Four”); 
}

There will be a separate tutorial covering Exceptions for JUnit 4 and 5 in detail.

Exception Test in TestNG

There is a slight change in the declaration of Exception test in TestNG:

@Test (expectedExceptions = NumberFormatException.class)  
public void converttoint() 
{  
	Int j=Integer.parseInt(“Four”); 
}

#5) Disable Test

Both TestNG and JUnit allows disabling a test for execution.

Disabled Test in JUnit 5

@Disabled annotation when used at the top of the class, all the tests within the class are skipped for execution. The annotation when used on top of a particular @Test method, that specific test case is disabled for execution.

import org.junit.AfterClass;
@Disabled("the testcase is under development")
public class JUnitProgram {

Disabled Test in TestNG

TestNG allows a test to disable execution for a test when the attribute “enabled” of annotation @Test is set to false and it is enabled when the attribute is set to true. If all tests within a class have to be enabled then explicitly mark enabled =true for each @Test method.

Below is the snippet of code that demonstrates the skipping of a test.

@Test(enabled=false)
public void f_validate(){
// let us skip this function}

#6) Group Tests

There have been contradictory statements in multiple sites and forums where people have mentioned that JUnit never supported the grouping of tests with the specific group name.

Annotations @BeforeGroups and @AfterGroups come only with TestNG however grouping is allowed in JUnit 4 as well as in JUnit 5. Here we will demonstrate quickly the usage of group tests in JUnit 5. Group tests are referred to as Categories in JUnit 4 and Tags in JUnit 5.

You may refer to Tutorial 8 – JUnit Test Suites & Filtering Tests for details on the usage in JUnit.

Group Tests in JUnit 5

@Tag(“Regression”)	
@Test
public void junitMethod1(){}

@Tag(“SmokeTest”)
@Test
public void junitMethod2(){

Code snippet from JUnit5TestSuite.java:

The below code includes the group with the name “Regression” and excludes the group “SmokeTest” which infers that junitMethod1() will be run however junitMethod2() is excluded.

@RunWith(JUnitPlatform.class)
@SelectPackages({“demo.tests“})
@IncludeTags(“Regression”)
@ExcludeTags(“SmokeTest”)
public class JUnit5TestSuite { }

Group Tests in TestNG

If the above snippet has to be interpreted in TestNG, then below is the code for the same:

@Test(groups={“Regression” })
public void junitMethod1(){}
@Test(groups={“SmokeTest” })
public void junitMethod2(){}

TestNG XML is as follows:

Here, the Regression group methods are included in the runner while the rest of the groups including SmokeTest is excluded.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="testSuite">
  <test name="mytest">
    <groups>
     <run>
      <include name="Regression" />
     </run>
   </groups>
    <classes>
       <class name="myclass"/>
    </classes>
  </test>
</suite>

#7) Parallel Tests

This is the feature that is only available with TestNG. Usually, test cases are kind of threads that are invoked one after the other. However, if you wish to save on the execution time, you can control this in TestNG by setting the tests to be run in parallel and providing the number of threads that need to run at once.

We will demonstrate in brief the usage of depends on methods here and will not discuss depends on groups.

The dependent test on another method is set via the TestNG XML as follows:

<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd" >
<suite name = "testSuite">
   <test name = "TestNGtests" parallel = "methods">
      <classes>
         <class name = "Myclass" />
      </classes>
   </test>
</suite>

#8) Dependent Tests

Dependent tests are advanced features available only with TestNG. The dependencies could be on a test or on a group.

@Test mytest1(){
System.out.println(“My test : mytest1”);
}
@Test (dependensOnMethods={“mytest1”})
public void mytest2(){
System.out.println(“My test : mytest2”);
}

In the above program, as mytest2 depends on mytest1, first mytest1 runs and then mytest2 is run. If mytest1 fails, then mytest2 will not be invoked. This is how dependent test cases can be predefined to control a specific workflow you wish to execute.

#9) Listeners

Listeners listen to every event that occurs within tests. Listeners are supported both in JUnit as well as TestNG. So, if you wish to perform certain tasks or show a specific message in the log before the test starts, after the test finishes, when the test is skipped, when the test is passed or failed, we have these Listener functions which enable us to do this

JUnit uses the Listener class, and TestNG uses a Listener interface. TestNG write a listener class defining the Listener interface methods, and the second step is to call this Listener class name using @Listeners annotation in the main class.

JUnit also inherits the methods from the Listener parent class following which a Listener runner class is defined to apply the listener functions on one or more main class.

Listener in TestNG

There is an ITestListener interface from which TestNG is implemented.

Below are the methods that need to be defined when ITestListener is implemented –

1. OnTestStart()
2. OnTestFailure()
3. OnTestSuccess()
4. OnTestSkipped()
5. OnStart()
6. OnFinish()

Below is the snippet of code demonstrating onTestStart() and onTestSuccess()

import org.testng.ITestListener; 
import org.testng.ITestResult; 
public class TestListener implements ITestListener 
{ 
@Override 
public void onTestStart(ITestResult result) { 
System.out.println("Execution started: "+result.getName()); 
} 
@Override 
public void onTestSuccess(ITestResult result) { 
System.out.println("Test Passed "+result.getName()); }

Call this listener class in your main class like shown below, using annotation @Listener:

import org.testng.annotations.Listeners;  
import org.testng.annotations.Test;  
@Listeners(com.javatpoint.Listener.class)  
public class MymainClass   
{  
@Test  
public void sum() { 

Listener in JUnit 5

RunListener is the class that needs to be extended by your listener class in order to define the Listener functions.

We have methods as follows for JUnit:

1. testRunStarted
2. testRunFinished
3. testFailure
4. tesIgnored

import org.junit.runner.notification.Failure;
import org.junit.runner.notification.RunListener;
public class Mylistenerclass extends RunListener
{
    public void testRunStarted(Description desc) throws java.lang.Exception
    {
        System.out.println("Execution started" + desc.getMethodName());
    }

public void testRunFinished(Description desc) throws java.lang.Exception
    {
        System.out.println("Execution finished" + desc.getMethodName());
    }

There needs to be a listener execution class create to invoke the above listener class.

You may apply the listener Mylistener class to multiple classes with test methods.

public class ListenerRunner
{
    public static void main(String[] args)
    {
        JUnitCore runme = new JUnitCore();     
        runme.addListener(new ListenerRunner());
        runner.run(FirstClass.class, SecondClass.class);
    }

Conclusion

In this JUnit Vs TestNG tutorial, we have compared TestNG and JUnit frameworks. We learned the common features supported in both the frameworks as well as additional features supported only in TestNG. Apparently, there are a couple of extra features only in TestNG, like parallel execution and dependency tests. Most of the features supported by TestNG is also available in JUnit.

There are slight deviations in terms of syntax, terminologies in JUnit vs TestNG for the common features. Those who undermined the strength of JUnit over TestNG would have realized until now i.e. JUnit is also one of the powerful automation frameworks.

We will come back with many more interesting aspects of JUnit. Stick to the upcoming tutorials!!!

=> Visit Here To Learn JUnit From Scratch.