What is Clarity?Clarity is a Test Automation Framework tomake automation● Eazy● Organized● Relevant● Optimized
Why Clarity?● To automate platform-wide scenarios.● To execute tests against stratos and private clouds.● To adopt tooling support to automation ○ Selenium ○ SoapUI ○ Jmeter ○ Ravana● Reporting and keeping historical records.● Coverage.
More On Clarity?Automate scenarios with multiple products with multipleusers/ tenants.● Execute tests using different DBs, OS, browsers and app servers.● Support for webcontext root, different port changes, host name bindings.● Testing Clustered environments.● Run tests against up and running server.
Unit Vs Integration TestsUnit Tests: ● Focused on defining (not testing) the behavior of a particular class, and that class only. ● Fast (less than 0.1 second per test) ● No external implementation dependencies (filesystem, database, web services, etc.). All dependencies are faked for the test context. ● Can easily be parallelized, since each test is atomicIntegration Tests: ● Focused on verifying the integration of one or more components together. ● May have external dependencies. (in fact, it is likely testing the integration with this dependency) ● A test that takes longer than a unit Test should (longer than 0.1 seconds per test)
Integration Tests Vs Platform TestsIntegration Tests - with in wso2 product ● Executes on product pack created by build. ● Not depending on other product environments. ● Independent. ● Lightweight. ● Covers only scenarios related to features on individual product.Platform Tests: ● Can run on a configured ( With mounts , Clusters ...) product packs and already up and running server. ● Can generate scenarios based on multiple product environments. ● Can execute load and UI tests
Architecture Overview● Follows TestNG Annotations bases Execution Structure.● Operations based on TestNG Listeners. ○ Server startup/Shutdown. ○ Emma instrumentation, Report Generation And Cleaning. ○ Artifact Deployment /Artifact cleaning. ○ Tennent Creation and Registering. ○ Report Generation.● Test Optimized API. Reference:- http://www.ibm.● Utility Classes to Support test cases. com/developerworks/java/library/j-testng/
Execution Template for Clarity Execution Started @BeforeSuite @BeforeTest @BeforeClass @BeforeGroups @BeforeMethod @AfterMethod @AfterGroups @AfterClass @AfterTest @AfterSuite Execution Closed
TestNg Listeners● Listeners are interfaces allows to modify TestNGs behavior.● Listeners are binned to a testNg execution.● Listeners implement the interface org.testng.ITestListener ○ IExecutionListener ■ Triggers at start and end of any execution. ■ ISuiteListener. ○ ■ Triggers at suite start and end.ITestListener ■ Triggers at Test start, Finish, Failure, Skip and partial failure.● Reporters implement the interface org.testng.IReporter ○ Notified when all the suites have been run by TestNG. ○ The IReporter instance receives a summary of entire test run
Usage of Listeners in Clarity● PlatformExecutionManager (IExecutionListener) ○ onExecutionStart() ■ Emma instrumentations ■ Server start ○ onExecutionFinish() ■ Server Sutdown ■ Emma report generation● PlatformSuiteManager (ISuiteListener) ○ Set environment properties ex- Key Store Paths ○ Populate Users.
Usage of Listeners in Clarity cont.d● PlatformTestManager (ITestListener) ○ On Start ■ Artifact Deployment ○ On Finish ■ Artifact Clean up● PlatformReportManager (IReporter) ○ Generate TestNg Report ○ Generate sure-fire report ○ Export data for Dashboard