TestNG is a testing framework that supports unit testing, integration testing, annotations, parameters, listeners, test groups, and dependencies. It allows specifying before and after methods, data providers, and test run configuration via XML. Key features include annotations for configuring tests, parameters to pass data to tests, listeners to customize test runs, and groups and dependencies to control test order and relationships. TestNG can be run from the command line, Ant, Eclipse, IntelliJ, and is supported in Maven and Ant builds.

1. TestNG Framework
Introduction
Levon Apreyan
QA Automation Engineer

2. Test Automation Framework
What is test automation framework?
• wikipedia - A test automation framework is an integrated system that sets the rules
of automation of a specific product.
Types of test automation frameworks exists for many languages, like Java, C#, PHP,
etc..
Examples: JUnit, Mockito, TestNG, etc..

3. TestNG is a testing framework, that supports:
•Unit Testing
•Integration Testing
•Annotations
•Annotation Transformers
•Parameters
•Listeners
•Group testing
•Dependencies
•Etc..
What is TestNG?

4. TestNG vs JUnit
Why TestNG?

5. Annotations
@BeforeSuite, @BeforeTest, @BeforeGroups, @BeforeClass, @BeforeMethod
@AfterSuite, @AfterTest, @AfterGroups, @AfterClass, @AfterMethod
@Test
@DataProvider
@Factory
@Listeners

6. Annotations
@BeforeSuite - runs before all tests in this suite have run.
@BeforeTest - runs before any test method belonging to the classes inside the <test>
tag is run.
@BeforeGroups - runs shortly before the first test method that belongs to any of these
groups is invoked.
@BeforeClass - runs before the first test method in the current class is invoked.
@BeforeMethod - run before each test method.
@AfterSuite, @AfterTest, @AfterGroups, @AfterClass, @AfterMethod - the same,
but after

7. Annotations
@Test annotation attributes
dataProvider - The name of the data provider for test method.
groups - The list of groups this class/method belongs to.
expectedExceptions - The list of exceptions that a test method is expected
to throw.
timeOut - The maximum number of milliseconds this test should take.
etc...

8. Annotation Transformers
TestNG allows to modify the content of all the annotations at runtime.
An Annotation Transformer is a class that implements IAnnotationTransformer or
IAnnotationTransformer2 interface.

9. Annotation Transformers
Example: override the attribute invocationCount but only on the test method invoke()
of one of test classes:
public class MyTransformer implements IAnnotationTransformer {
public void transform(ITest annotation, Class testClass, Constructor testConstructor, Method
testMethod)
{
if ("invoke".equals(testMethod.getName())) {
annotation.setInvocationCount(5);
}
}
}

10. Parameters
Parameters can be passed to test methods.
There are two ways to set these parameters: with testng.xml or programmatically.

11. Parameters
Example 1: Parameters from testing.xml (for simple values only)
@Parameters({ "first-name" })
@Test
public void testSingleString(String firstName) {
System.out.println("Invoked testString " + firstName);
assert "Cedric".equals(firstName);
}
This XML parameter is defined in testng.xml:
<suite name="My suite">
<parameter name="first-name" value="Cedric"/>

12. Parameters
Example 2: Parameters with DataProviders
//This method will provide data to any test method that declares that its
Data Provider is named "test1"
@DataProvider(name = "test1")
public Object[ ][ ] createData1() {
return new Object[ ][ ] {
{ "Cedric", new Integer(36) },
{ "Anne", new Integer(37)},
};
}

13. Parameters
Example 2: Parameters with DataProviders
//This test method declares that its data should be supplied by the Data
Provider named "test1"
@Test(dataProvider = "test1")
public void verifyData1(String n1, Integer n2) {
System.out.println(n1 + " " + n2);
}
Output:
Cedric 36
Anne 37

14. Listeners
"TestNG Listeners" allow to modify TestNG's behavior or implement some
actions.
There are several listener interfaces.

15. Listeners
Here are a few listeners:
● IAnnotationTransformer - allows to modify the content of @Test annotation
● IAnnotationTransformer2 - allows to modify the content of other annotations
● IHookable - allows to override and possibly skip the invocation of test methods
● IInvokedMethodListener - allows to be notified whenever TestNG is about to
invoke a test (annotated with @Test) or configuration (annotated with any of the @Before
or @After annotation) method

16. Listeners
Here are a few listeners (continuation):
● IMethodInterceptor - allows control over running order of methods that don’t have
dependencies or dependents
● IReporter - will generate test report after all the suites have been run
● ISuiteListener - listener for test suites
● ITestListener - listener for test methods

17. Listeners
After implementing one of these interfaces, you can let TestNG know about it with either of the
following ways:
● Using -listener on the command line.
● Using <listeners> with ant.
● Using <listeners> in your testng.xml file.
● Using the @Listeners annotation on any of your test classes.
● Using ServiceLoader.

18. Listeners
Example 1: declaring listener with annotation:
@Listeners({ com.example.MyListener.class, com.example.MyMethodInterceptor.class })
public class MyTest {
// …
}

19. Listeners
Example 2: declaring annotation with ServiceLoader (which Workfront uses).
Assume we have listener - test.tmp.TmpSuiteListener
Create a file at the location META-INF/services/org.testng.ITestNGListener, which will name the
implementation(s) of listeners:
$ cat META-INF/services/org.testng.ITestNGListener
test.tmp.TmpSuiteListener
$ tree
|____META-INF
| |____services
| | |____org.testng.ITestNGListener

20. Test Groups
• Test methods can be grouped together.
• Test groups can contain other groups.
This allows TestNG to be invoked and asked to include a certain set of groups (or regular
expressions) while excluding another set.

21. Test Groups
For example, we have two categories of tests:
● Check-in tests. These tests should be run before you submit new code. They should
typically be fast and just make sure no basic functionality was broken.
● Functional tests. These tests should cover all the functionalities of your software and be run
at least once a day, although ideally you would want to run them continuously.

22. Test Groups
And we have following tests:
public class Test1 {
@Test(groups = { "functest", "checkintest" })
public void testMethod1() {
}
@Test(groups = {"functest", "checkintest"} )
public void testMethod2() {
}
@Test(groups = { "functest" })
public void testMethod3() {
}
}

23. Test Groups
Invoking TestNG with the following configuration will run all the test methods:
<test name="Test1">
<groups>
<run>
<include name="functest"/>
</run>
</groups>
<classes>
<class name="example1.Test1"/>
</classes>
</test>
Invoking it with checkintest will only run testMethod1() and testMethod2().

24. Test Groups
Groups can include other groups. These groups are called "MetaGroups".
<test name="Regression1">
<groups>
<define name="functest">
<include name="windows"/>
<include name="linux"/>
</define>
<define name="all">
<include name="functest"/>
<include name="checkintest"/>
</define>
<run>
<include name="all"/>
<run>
</groups>
<classes>
<class name="test.sample.Test1"/>
</classes>
</test>

25. Test Groups
TestNG allows to include groups as well as exclude them. For example we have a test
that is broken.
@Test(groups = {"checkintest", "broken"} )
public void testMethod2() {
}
We can set the broken tests to be excluded from run in testng.xml :
…
<groups>
<run>
<include name="checkintest"/>
<exclude name="broken"/>
</run>
</groups>
…

26. Test Groups
TestNG allows to define groups at the class level and then add groups at the method
level:
@Test(groups = { "checkin-test" })
public class All {
@Test(groups = { "func-test" )
public void method1() { ... }
public void method2() { ... }
}
method2() is part of the group "checkin-test".
method1() belongs to both "checkin-test" and "func-test".

27. Dependencies
Dependencies are for test methods to be invoked in a certain order.
TestNG allows to specify dependencies either with annotations or in XML.

28. Dependencies
Two types of Dependencies:
• Hard dependencies. All the dependent methods must have run and succeeded for you to
run.
• Soft dependencies. Dependent method will always be run after the methods you depend
on, even if some of them have failed. This is done by adding "alwaysRun=true" in @Test
annotation.

29. Dependencies
Example 1: hard dependency:
@Test
public void serverStartedOk() {}
@Test(dependsOnMethods = { "serverStartedOk" })
public void method1() {}

30. Dependencies
Example 2: Soft dependency:
@Test
public void serverStartedOk() {}
@Test(dependsOnMethods = { "serverStartedOk" }, alwaysRun=true)
public void method1() {}

31. Dependencies
We can also have methods that depend on entire groups:
@Test(groups = { "init" })
public void serverStartedOk() {}
@Test(groups = { "init" })
public void initEnvironment() {}
@Test(dependsOnGroups = { "init.*" })
public void method1() {}
In this example, method1() is declared as depending on any group matching the regular
expression "init.*".

32. Running TestNG
TestNG can be invoked in different ways:
● Command line
● ant
● Eclipse
● IntelliJ IDEA

33. Running TestNG
The simplest way to invoke TestNG from command line is as follows:
java org.testng.TestNG testng1.xml [testng2.xml testng3.xml ...]
where:
org.testng.TestNG is the main class;
testng1.xml [testng2.xml testng3.xml ...] are configuration files.

34. Running TestNG
We can run tests without testng.xml:
java org.testng.TestNG -groups windows,linux -testclass org.test.MyTest
where:
org.testng.TestNG is the main class;
-groups windows,linux specify which group tests should run;
-testclass org.test.MyTest specity which test clssses should run.

35. Logging and results
The results of the test run are created in a file called index.html.
It is possible to generate own reports with TestNG listeners and reporters:
● Listeners are notified in real time of when a test starts, passes, fails, etc…
● Reporters are notified when all the suites have been run by TestNG.

36. Integrating TestNG in environment
There are TestNG plugins available for the following IDEs:
• Eclipse
• IntelliJ IDEA
Plugins help to easily to:
• configure TestNG
• convert JUnit tests to TestNG tests
• run TestNG tests
• view test results

37. How to include TestNG in Maven or Ant
• Maven: add the following to pom.xml:
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>6.9.12</version>
<scope>test</scope>
</dependency>
• Ant: define the TestNG ant task as follows:
<taskdef resource="testngtasks" classpath="testng.jar"/>

38. References
• http://testng.org/doc/index.html - TestNG site
• http://testng.org/doc/documentation-main.html
- Documentation

39. THANK YOU!
Questions?
Contact: lapreyan@gmail.com