Clarity is a test automation framework that makes testing easy, organized, relevant and optimized. It allows automating scenarios across multiple WSO2 products using different browsers, databases, operating systems and application servers. Clarity uses TestNG annotations for test execution structure and listeners for operations like server startup/shutdown, report generation and artifact deployment/clean up. Test cases in Clarity utilize utility classes and optimized APIs while following an execution template.

1. Introduction To Clarity

2. What is Clarity?
Clarity is a Test Automation Framework to
make automation
● Eazy
● Organized
● Relevant
● Optimized

3. Why Clarity?
● To automate platform-wide scenarios.
● To execute tests against stratos and private clouds.
● To adopt tooling support to automation
○ Selenium
○ SoapUI
○ Jmeter
○ Ravana
● Reporting and keeping historical records.
● Coverage.

4. WSO2 Platform Scenario

5. More On Clarity?
Automate scenarios with multiple products with multiple
users/ tenants.
● Execute tests using different DBs, OS, browsers and
app servers.
● Support for webcontext root, different port changes,
host name bindings.
● Testing Clustered environments.
● Run tests against up and running server.

6. Unit Vs Integration Tests
Unit Tests:
● Focused on defining (not testing) the behavior of a particular class,
and that class only.
● Fast (less than 0.1 second per test)
● No external implementation dependencies (filesystem, database, web
services, etc.). All dependencies are faked for the test context.
● Can easily be parallelized, since each test is atomic
Integration Tests:
● Focused on verifying the integration of one or more components
together.
● May have external dependencies. (in fact, it is likely testing the
integration with this dependency)
● A test that takes longer than a unit Test should (longer than 0.1
seconds per test)

7. Integration Tests Vs Platform Tests
Integration Tests - with in wso2 product
● Executes on product pack created by build.
● Not depending on other product environments.
● Independent.
● Lightweight.
● Covers only scenarios related to features on individual product.
Platform Tests:
● Can run on a configured ( With mounts , Clusters ...) product packs
and already up and running server.
● Can generate scenarios based on multiple product environments.
● Can execute load and UI tests

8. Architecture Overview
● Follows TestNG Annotations bases
Execution Structure.
● Operations based on TestNG Listeners.
○ Server startup/Shutdown.
○ Emma instrumentation, Report Generation And
Cleaning.
○ Artifact Deployment /Artifact cleaning.
○ Tennent Creation and Registering.
○ Report Generation.
● Test Optimized API.
Reference:- http://www.ibm.
● Utility Classes to Support test cases. com/developerworks/java/library/j-testng/

9. Execution Template for Clarity
Execution Started
@BeforeSuite
@BeforeTest
@BeforeClass
@BeforeGroups
@BeforeMethod
@AfterMethod
@AfterGroups
@AfterClass
@AfterTest
@AfterSuite
Execution Closed

10. TestNg Listeners
● Listeners are interfaces allows to modify TestNG's
behavior.
● Listeners are binned to a testNg execution.
● Listeners implement the interface org.testng.ITestListener
○ IExecutionListener
■ Triggers at start and end of any execution.
■ ISuiteListener.
○
■ Triggers at suite start and end.ITestListener
■ Triggers at Test start, Finish, Failure, Skip and partial failure.
● Reporters implement the interface org.testng.IReporter
○ Notified when all the suites have been run by TestNG.
○ The IReporter instance receives a summary of entire test run

11. Usage of Listeners in Clarity
● PlatformExecutionManager (IExecutionListener)
○ onExecutionStart()
■ Emma instrumentations
■ Server start
○ onExecutionFinish()
■ Server Sutdown
■ Emma report generation
● PlatformSuiteManager (ISuiteListener)
○ Set environment properties ex- Key Store Paths
○ Populate Users.

12. Usage of Listeners in Clarity cont.d
● PlatformTestManager (ITestListener)
○ On Start
■ Artifact Deployment
○ On Finish
■ Artifact Clean up
● PlatformReportManager (IReporter)
○ Generate TestNg Report
○ Generate sure-fire report
○ Export data for Dashboard

13. Clarity Architecture

14. Artifact Repository |-- AS
| |-- aar
● Location for keeping all artifacts | | |-- Axis2 Service.aar
| |-- car
| | `-- AxisCApp-1.0.0.car
needs for your Test |-- BPS
| `-- bpel
| |-- CleanUpTest1.zip|
● Refers by automated Artifact |-- BRS
| |-- aar
| |-- rsl
deployment process | `-- ruleservice
| `-- GreetingService.aar
|-- DSS
● Manages my file structure | |-- dbs
| | |-- csv
|-- ESB
depending on product and artifact | |-- car
| | |-- DemoProxy-1.0.0.car
| |-- endpointlookup.xml
type. |-- GREG
| |-- artistRegistry.gar
|-- GS
| `-- gadget
| `-- HelloWorldGadget.xml
|-- IS
| |-- IIA001Policy.xml
`-- MS
`-- jszip
`-- schemaTest1.zip

15. Clarity Properties
● File Location <FRAMEWORK_HOME>/core/org.wso2.automation.platform.
core/src/main/resources/framework.properties
# Testing server details
● Contains all information #-------------------------------------------------------------------------------------
--------------
stratos.test=false
about your environment. port.enable=true
carbon.web.context.enable=false
service.deployment.delay=30000
cluster.enable=false
● To configure WSO2 clarity product.list=GREG
deployment.framework.enable=false
for your environment and deployment.framework.
home=/home/krishantha/automation/deployment-framework
configure package details coverage.enable=false
greg.carbon.
home=/home/dharshana/latestPack/gregPack2/wso2greg-4.0.0-
SNAPSHOT
as.host.name=localhost
as.http.port=9764
as.https.port=9444
#as.webContext.root

16. Manage Tenants
● File Location
● <FRAMEWORK_HOME>/core/org.wso2.automation.platform.
core/src/main/resources/tenantList.csv
● <FRAMEWORK_HOME>/core/org.wso2.automation.platform.
core/src/main/resources/usertList.csv
"0","admin","admin123"
"1","admin123@wso2manualQA0002.org","admin123"
"2","admin123@wso2manualQA0004.org","admin123"
"3","admin123@wso2manualQA0006.org","admin123"
"4","admin123@manualQA0001.org","admin123"
"5","admin123@krishantha.hhh","admin123"
"6","admin123@wso2autoQA0008.org","admin123"
"7","admin123@wso2autoQA0009.org","admin123"
"8","admin123@wso2autoQA0010.org","admin123"
"9","admin123@wso2autoQA0011.org","admin123"
"10","admin123@wso2autoQA0012.org","admin123"
"11","admin123@wso2autoQA0013.org","admin123"
"12","admin123@wso2autoQA0014.org","admin123"

17. Automated Artifact Deployment
<?xml version="1.0"?>
<scenarios>
<scenario testName="org.wso2.automation.common.test.as.
service.AARServiceSpaceInFileNameTest">
<products>
<product name="AS">
<artifacts userId="1">
<artifact name="Axis2 Service.aar" type="aar"/>
</artifacts>
</product>
</products>
</scenario>

18. Write First Clarity Test - Demo

19. Reporting
● Includes TestNg reports ,Junit |-- reports
| |-- BPSStructuredSuite
Reports ( Sure Fire report), And | | |-- classes.html
| | |-- FlowClient.html
Emma Coverage report | | |-- FlowClient.properties
● Directories containing separate | | |-- FlowClient.xml
| | |-- ForEachClient.html
results for each suite ( Directory | | |-- ForEachClient.properties
| | |-- ForEachClient.xml
name is as same as the suite name). | |-- index.html
● Junit Reports ( Surefire reports). | |-- junitreports
| | |-- TEST-org.wso2.automation.
● index.html (Dashboard for executed common.test.bps.mgts.
BpelStructAct_FlowClient.xml
test). | | |-- TEST-org.wso2.automation.
common.test.bps.mgtst.
BpelStructAct_forEachClient.xml

20. Reference
OT article
http://wso2.org/library/knowledge-base/2012/06/configure-execute-tests-
clarity
TestNg documenation
http://testng.org/doc/documentation-main.html
SVN Location
https://svn.wso2.org/repos/wso2/carbon/platform/trunk/platform-
integration/system-test-framework

21. Questions?