TestNG: Concepts, Methods and Examples

TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use, such as:

• JDK 5 Annotations (JDK 1.4 is also supported with JavaDoc annotations).
• Flexible test configuration.
• Support for data-driven testing (with @DataProvider).
• Support for parameters.
• Allows distribution of tests on slave machines.
• Powerful execution model (no more TestSuite).
• Supported by a variety of tools and plug-ins (Eclipse, IDEA, Maven, etc…).
• Embeds BeanShell for further flexibility.
• Default JDK functions for runtime and logging (no dependencies).
• Dependent methods for application server testing.

TestNG is designed to cover all categories of tests: unit, functional, end-to-end, integration, etc…

Not just Java you can:

• Ruby (Test Unit)
• Python (PythonUnit)
• JavaScript (JsUnit)
• C, C++, C# (NUnit)
• And many others.
• All based on SUnit concept.
• For the dynamically type-checked languages, unit testing is considered as a replacement for static-type checking.

A test class is an ordinary Java class meked up with annotations. So test classesare organised into Suite, framework identifies and runs all the tests in a suite, generates a test report.

Classical strategy is: create an object of class being tested, call method, check value returned or if void method, call another non-void method and check value returned.

TestNG example

• Write a class Person.
• Person objects store a person’s first and last names.
• A person object can return the full name of a person (first name + last name).
  • Yes, this is simple and you could probably get it right first time!
  • Don’t worry it gets more complex…

First Test

Given a person with first name “a”, last name “b”, getName should return “a b” (with space), after create and initialise Person object, call getName method, Assert return value equals “a b”.

import org.testng.annotations.BeforeMethod ;
import org.testng.annotations.Test ;
import static org.testng.Assert.assertEquals ;
public class PersonTest {
private Person aPerson ;
@BeforeMethod
public void setUp ( ) {
aPerson = new Person ( "a" , "b" ) ;
}
@Test
public void testGetName ( ) {
assertEquals ( aPerson.getName ( ) , "a b" ) ;
}
}

Using Java 5 (and now Java 6).

@Test and @BeforeMethod are Java annotations, Java 5 supports declaration of annotation types, meta-data that can be attached to methods. TestNG framework defines annotations types to mark up test classes. Test class does not need to inherit from test base class.

How the test will be run

For each test method (marked @Test), create an instance of the test class, call the @BeforeMethod method.

It will create the fixture object, call the test method and then record results.

Simplest version of Person class
To allow test class to compile.

public class Person {
public Person ( final String firstName , final String lastName ) { }
public String getName ( ) { return ""; }
}

Can now run test and it finds an error.

• RED

===============================================
Person Suite
Total tests run: 1, Failures: 1, Skips: 0
===============================================

Make test pass (i.e., remove error)

public class Person {
private final String firstName ;
private final String lastName ;
public Person ( final String firstName , final String lastName ) {
this.firstName = firstName ;
this.lastName = lastName ;
}
public String getName ( ) { return firstName + " " + lastName; }
}

• Test passes. GREEN

===============================================
Person Suite
Total tests run: 1, Failures: 0, Skips: 0
===============================================

TestNG generates a detailed test report in html format (via XML).

Annotations

Here is a quick overview of the annotations available in TestNG along with their attributes.

@BeforeSuite @AfterSuite @BeforeTest @AfterTest @BeforeGroups @AfterGroups @BeforeClass @AfterClass @BeforeMethod @AfterMethod		Configuration information for a TestNG class: @BeforeSuite: The annotated method will be run before all tests in this suite have run. @AfterSuite: The annotated method will be run after all tests in this suite have run. @BeforeTest: The annotated method will be run before the test. @AfterTest: The annotated method will be run after the test. @BeforeGroups: The list of groups that this configuration method will run before. This method is guaranteed to run shortly before the first test method that belongs to any of these groups is invoked. @AfterGroups: The list of groups that this configuration method will run after. This method is guaranteed to run shortly after the last test method that belongs to any of these groups is invoked. @BeforeClass: The annotated method will be run before the first test method in the current class is invoked. @AfterClass: The annotated method will be run after all the test methods in the current class have been run. @BeforeMethod: The annotated method will be run before each test method. @AfterMethod: The annotated method will be run after each test method.
	alwaysRun	For before methods (beforeSuite, beforeTest, beforeTestClass and beforeTestMethod, but not beforeGroups): If set to true, this configuration method will be run regardless of what groups it belongs to. For after methods (afterSuite, afterClass, …): If set to true, this configuration method will be run even if one or more methods invoked previously failed or was skipped.
	dependsOnGroups	The list of groups this method depends on.
	dependsOnMethods	The list of methods this method depends on.
	enabled	Whether methods on this class/method are enabled.
	groups	The list of groups this class/method belongs to.
	inheritGroups	If true, this method will belong to groups specified in the @Test annotation at the class level.
@DataProvider		Marks a method as supplying data for a test method. The annotated method must return an Object[][] where each Object[] can be assigned the parameter list of the test method. The @Test method that wants to receive data from this DataProvider needs to use a dataProvider name equals to the name of this annotation.
	name	The name of this DataProvider.
@Factory		Marks a method as a factory that returns objects that will be used by TestNG as Test classes. The method must return Object[].
@Parameters		Describes how to pass parameters to a @Test method.
	value	The list of variables used to fill the parameters of this method.
@Test		Marks a class or a method as part of the test.
	alwaysRun	If set to true, this test method will always be run even if it depends on a method that failed.
	dataProvider	The name of the data provider for this test method.
	dataProviderClass	The class where to look for the data provider. If not specified, the data provider will be looked on the class of the current test method or one of its base classes. If this attribute is specified, the data provider method needs to be static on the specified class.
	dependsOnGroups	The list of groups this method depends on.
	dependsOnMethods	The list of methods this method depends on.
	description	The description for this method.
	enabled	Whether methods on this class/method are enabled.
	expectedExceptions	The list of exceptions that a test method is expected to throw. If no exception or a different than one on this list is thrown, this test will be marked a failure.
	groups	The list of groups this class/method belongs to.
	invocationCount	The number of times this method should be invoked.
	successPercentage	The percentage of success expected from this method
	sequential	If set to true, all the methods on this test class are guaranteed to run sequentially, even if the tests are currently being run with parallel=”true”. This attribute can only be used at the class level and it will be ignored if used at the method level.
	timeOut	The maximum number of milliseconds this test should take.
	threadPoolSize	The size of the thread pool for this method. The method will be invoked from multiple threads as specified by invocationCount. Note: this attribute is ignored if invocationCount is not specified

[http://testng.org]

[Robert Graham UCL London]

[wikipedia.org]