Clarity is a test automation framework that makes testing easy, organized, relevant and optimized. It allows automating scenarios across multiple WSO2 products using different browsers, databases, operating systems and application servers. Clarity uses TestNG annotations for test execution structure and listeners for operations like server startup/shutdown, report generation and artifact deployment/clean up. Test cases in Clarity utilize utility classes and optimized APIs while following an execution template.
2. What is Clarity?
Clarity is a Test Automation Framework to
make automation
● Eazy
● Organized
● Relevant
● Optimized
3. Why Clarity?
● To automate platform-wide scenarios.
● To execute tests against stratos and private clouds.
● To adopt tooling support to automation
○ Selenium
○ SoapUI
○ Jmeter
○ Ravana
● Reporting and keeping historical records.
● Coverage.
5. More On Clarity?
Automate scenarios with multiple products with multiple
users/ tenants.
● Execute tests using different DBs, OS, browsers and
app servers.
● Support for webcontext root, different port changes,
host name bindings.
● Testing Clustered environments.
● Run tests against up and running server.
6. Unit Vs Integration Tests
Unit Tests:
● Focused on defining (not testing) the behavior of a particular class,
and that class only.
● Fast (less than 0.1 second per test)
● No external implementation dependencies (filesystem, database, web
services, etc.). All dependencies are faked for the test context.
● Can easily be parallelized, since each test is atomic
Integration Tests:
● Focused on verifying the integration of one or more components
together.
● May have external dependencies. (in fact, it is likely testing the
integration with this dependency)
● A test that takes longer than a unit Test should (longer than 0.1
seconds per test)
7. Integration Tests Vs Platform Tests
Integration Tests - with in wso2 product
● Executes on product pack created by build.
● Not depending on other product environments.
● Independent.
● Lightweight.
● Covers only scenarios related to features on individual product.
Platform Tests:
● Can run on a configured ( With mounts , Clusters ...) product packs
and already up and running server.
● Can generate scenarios based on multiple product environments.
● Can execute load and UI tests
8. Architecture Overview
● Follows TestNG Annotations bases
Execution Structure.
● Operations based on TestNG Listeners.
○ Server startup/Shutdown.
○ Emma instrumentation, Report Generation And
Cleaning.
○ Artifact Deployment /Artifact cleaning.
○ Tennent Creation and Registering.
○ Report Generation.
● Test Optimized API.
Reference:- http://www.ibm.
● Utility Classes to Support test cases. com/developerworks/java/library/j-testng/
9. Execution Template for Clarity
Execution Started
@BeforeSuite
@BeforeTest
@BeforeClass
@BeforeGroups
@BeforeMethod
@AfterMethod
@AfterGroups
@AfterClass
@AfterTest
@AfterSuite
Execution Closed
10. TestNg Listeners
● Listeners are interfaces allows to modify TestNG's
behavior.
● Listeners are binned to a testNg execution.
● Listeners implement the interface org.testng.ITestListener
○ IExecutionListener
■ Triggers at start and end of any execution.
■ ISuiteListener.
○
■ Triggers at suite start and end.ITestListener
■ Triggers at Test start, Finish, Failure, Skip and partial failure.
● Reporters implement the interface org.testng.IReporter
○ Notified when all the suites have been run by TestNG.
○ The IReporter instance receives a summary of entire test run
11. Usage of Listeners in Clarity
● PlatformExecutionManager (IExecutionListener)
○ onExecutionStart()
■ Emma instrumentations
■ Server start
○ onExecutionFinish()
■ Server Sutdown
■ Emma report generation
● PlatformSuiteManager (ISuiteListener)
○ Set environment properties ex- Key Store Paths
○ Populate Users.
12. Usage of Listeners in Clarity cont.d
● PlatformTestManager (ITestListener)
○ On Start
■ Artifact Deployment
○ On Finish
■ Artifact Clean up
● PlatformReportManager (IReporter)
○ Generate TestNg Report
○ Generate sure-fire report
○ Export data for Dashboard