Hybrid framework for test automation

  • 959 views
Uploaded on

 

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
959
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Hybrid Framework for Test Automationby: Vinay Srinivasanemail: vinay.srinivasan@techmahindra.com srinivasan_vinay@yahoo.comPhone: +91-9823104620 © Tech Mahindra Limited 4/23/2013
  • 2. Table of Contents Page1.INTRODUCTION.......................................................................32.WHY TEST AUTOMATION FRAMEWORK...............................4..................................................................................................... 53.GOAL OF TEST AUTOMATION FRAMEWORKS ....................54.DIFFERENT TYPES OF TEST AUTOMATION FRAMEWORKS .............................................................6 4.1Modular Framework............................................................................................................................ 6 4.2Data Driven Framework...................................................................................................................... 6 4.3Keyword Driven Framework................................................................................................................ 7 4.4Hybrid Framework............................................................................................................................... 75.HYBRID FRAMEWORK ARCHITECTURE................................76.HYBRID FRAMEWORK IMPLEMENTATION............................8 6.1Initialization......................................................................................................................................... 9 6.2Driver and Controller........................................................................................................................... 9 6.3Data Table........................................................................................................................................ 10 6.4Generic Functions............................................................................................................................. 10 6.5Utility Functions................................................................................................................................. 11 6.6Test Case Functions......................................................................................................................... 11 6.7Logger Functions.............................................................................................................................. 12 6.8Timer Functions................................................................................................................................ 137.CONCLUSION.........................................................................14 © Tech Mahindra Limited 4/23/2013
  • 3. 1. INTRODUCTION Test Automation Framework is set of assumptions, concepts and practices that provide support for automated Software testing. Basing an automated testing effort on using only a record and playback test cases has its own drawbacks. Running complex and powerful tests is time consuming and expensive when using only a capture tool. Because these tests are created ad hoc, their functionality can be difficult to track and reproduce, and they can be costly to maintain. A better choice for automation testing team is to use a test automation framework, defined as set of assumptions, concepts and practices that constitute a work platform or support for automated testing. There are few types of Framework which are suitable for Test Automation. Out of this Hybrid Framework is found to be the most suitable for Test Automation as it employs most of the features of other Frameworks and is easy to implement. In following sections we will look into some of the Frameworks used in the IT Industry and discuss about how to implement Hybrid Framework for Test Automation. © Tech Mahindra Limited 4/23/2013
  • 4. 2. WHY TEST AUTOMATION FRAMEWORK Although applications are relatively unique, the components that comprise them, in general, are not. Thus, we should focus our automation framework to deal with the common components that make up our unique applications. By doing this, we can remove all application-specific context from our framework and reuse virtually everything we develop for every application that comes through the automated test process. Nearly all applications come with some form of menu system. They also have buttons to push, boxes to check, lists to view, and so on. In a typical automation tool script there is, generally, a very small number of component functions for each type of component. These functions work with the component objects independent of the applications that contain them. Traditional, captured automation scripts are filled with thousands of calls to these component functions. So the tools already exist to achieve application independence. The problem is most of these scripts construct the function calls using application- specific, hard coded values. This immediately reduces their effectiveness as application- independent constructs. Furthermore, the functions by themselves are prone to failure unless a very specific application state or synchronization exists at the time they are executed. There is little error correction or prevention built-in to these functions. To deal with this in traditional scripts we must place additional code before andor after the command, or a set of commands, to insure the proper application state and synchronization is maintained. We need to make sure our window has the current focus. We need to make sure the component we want to select, or press, or edit exists and is in the proper state. Only then can we perform the desired operation and separately verify the result of our actions. For maximum robustness, we would have to code these state and synchronization tests for every component function call in our scripts. Realistically, we could never afford to do this. It would make the scripts huge, nearly unreadable, and difficult to maintain. Yet, where we forego this extra effort, we increase the possibility of script failure. What we must do is develop a truly application-independent framework for these component functions. This will allow us to implement that extra effort just once, and © Tech Mahindra Limited 4/23/2013
  • 5. execute it for every call to any component function. This framework should handle all the details of insuring we have the correct window, verifying the element of interest is in the proper state, doing something with that element, and logging the success or failure of the entire activity. We do this by using variables, and providing application-specific data to our application- independent framework. In essence, we will provide our completed test designs as executable input into our automation framework. Does this mean that we will never have to develop application-specific test scripts? Of course not. However, if we can limit our application-specific test scripts to some small percentage, while reusing the best features of our automation framework, we will reap the rewards project after project.3. GOAL OF TEST AUTOMATION FRAMEWORKS One of our goals should be a highly modular and maintainable framework. Generally, each module should be independent and separate from all the other modules. What happens inside one is of no concern to the others. With this modular black-box approach, the functionality available within each module can be readily expanded without affecting any other part of the system. This makes code maintenance much simpler. Additionally, the complexity of any one module will likely be quite minimal. However, modularity alone will not be enough to ensure a highly maintainable framework. Like any good software project, our design must be fully documented and published. Without adequate, published documentation it will be very difficult for anyone to decipher what it is the framework is designed to do. Any hope of maintenance will not last far beyond the departure of the original framework designers. Our test automation efforts will eventually become another negative statistic. To prevent this, we should define documentation standards and templates. Wherever possible, module documentation should be developed "in-context". That is, directly in the source code itself. Tools should be retained, or designed and developed, so that we can automatically extract and publish the documentation. This will eliminate the task of maintaining two separate sets of files: the source code, and its documentation. It will also provide those doing the code maintenance quite a ready reference. Nearly everything they need to know should exist right there in the code. We must always remember: our ultimate goal is to simplify and perpetuate a successful test automation framework. To put something in place that people will use and reuse for as long as it is technically viable and productive. © Tech Mahindra Limited 4/23/2013
  • 6. 4. DIFFERENT TYPES OF TEST AUTOMATION FRAMEWORKS There are four main types of test automation frameworks as listed below. 4.1 Modular Framework A collection of small, independent scripts represents modules/sections/functions of the application under test. The modules are then used in a hierarchical or logical fashion to construct larger test realizing an actual test case features in QTP to support Modularity Framework like Reusable Actions and Functional Libraries 4.2 Data Driven Framework Data-Driven Framework is a methodology used in Test automation where test scripts are executed and verified based on the data values stored in one or more central data sources or databases. These databases can range from data pools, ODBC sources, CSV files, Excel files, DAO objects, ADO objects, etc. Data- driven testing is the establishment of several interacting test scripts together with their related data results in a framework used for the methodology. In this framework, variables are used for both input values and output verification values: navigation through the program, reading of the data sources, and logging of test status and information are all coded in the test script. Thus, the logic executed in the script is also dependant on the data values. © Tech Mahindra Limited 4/23/2013
  • 7. 4.3 Keyword Driven Framework Keyword-Driven Framework is also known as table-driven testing, action-word testing methodology. It is a technique particularly well suited to automate testing. The advantages for automated tests are the reusability and therefore ease of maintenance of tests, which have been created at a high level of abstraction. The data table records contain the keywords that describe the actions we want to perform. They also provide any additional data needed as input to the application, and where appropriate, the benchmark information we use to verify the state of our components and the application in general. 4.4 Hybrid Framework This is the most successful automation framework and it generally accommodates both keywords driven testing as well as data driven scripts. This allows data driven scripts to take advantage of the powerful libraries and utilities that usually accompany a keyword driven architecture. The framework utilities can make the data driven scripts more compact and less prone to failure than they otherwise would have been. The utilities can also facilitate the gradual and manageable conversion of existing scripts to keyword driven equivalents when and where that appears desirable. On the other hand, the framework can use scripts to perform some tasks that might be too difficult to re-implement in a pure keyword driven approach, or where the keyword driven capabilities are not yet in place.5. HYBRID FRAMEWORK ARCHITECTURE © Tech Mahindra Limited 4/23/2013
  • 8. 6. HYBRID FRAMEWORK IMPLEMENTATION The Hybrid Framework is implemented using the following Functions: © Tech Mahindra Limited 4/23/2013
  • 9. 6.1 Initialization This function is used to initialize the initial start up variables like Application Name or Executable Name, Application Url or its path, log file names and its path, Data Table name and its path, Application Login credentials like User ID and Password. This initialization is accomplished using values from the Test Data and Environment Values. Initialization Function Diagram Begin of Initialization Get Environment Values Get Connection to the Initialization Table Set Values for: Application Name Application Path User ID Password Log File Name Log File Path Test Data File Name Test Data File Path Close Connection to the Initialization Table End of Initialization6.2 Driver and Controller This Driver Script calls initialization function to initialize the input data parameters. It then calls a controller function, which has a Keyword Routine. It reads the Keyword and Data from the Data Table and then calls the corresponding Functions and Subroutines along with the input from the data table. The controller function handles this feature of calling the functions separately. Once the functions are executed, the control is then transferred back to the Controller for the execution of next set of records based on the keyword in the table. Driver Script Diagram Begin of Driver Call Initialization Call Controller Controller Function Diagram End of Driver © Tech Mahindra Limited 4/23/2013
  • 10. Begin of Controller Get Connection to the Data Table Begin of Keyword Routine Get Keyword from Data Table Start Timer for Test Run Log Test Name and Test Start Time Initialize Boolean State to True Handle Test Case Functions Collect Test Result Write Test Result to Data Table Stop the Timer for Test Run Log the End of the Test and Test End Time Save the changes made to the Data Table End of Keyword Routine Close Connection to the Data Table End of Controller6.3 Data Table This can be an Excel based table or a Database based table. Here each record or row contains a keyword; execution condition like yes or no, set of test data and this represents each test case that needs to be executed. It also contains separate columns for updating the test case and the test result like pass or fail. Sample Data Table S.NoKeywordTest Case NameTest Data1Test Data2…ExecuteTest Result1Ky1TC1……………….……..YesPass2Ky1TC2………………. ……..NoNoRun3Ky3TC3……………….……..YesFail6.4 Generic Functions These functions are very specific to the client application and are generic in nature. The advantage here is that we can use these functions for similar client applications. These Generic Functions include built in check points, synchronization and wait property. It also initializes a Boolean State condition based on the state of the objects present in the application under test. This is useful for error and exception handling. These functions also use Logging functions. © Tech Mahindra Limited 4/23/2013
  • 11. Generic Function Diagram Begin of Generic Function G1 Check Boolean State Set the Object Check if the Object Exist Reset the Boolean State to True or False Log Object Exist or Does Not Exist End of Generic Function G16.5 Utility Functions These functions are very specific to actions that need to be carried out frequently at beginning of test case, during the test case execution and at the end of the test case. It includes specific Page, Link, Tab Navigation, Login and Logout. These functions also use Generic and Logging functions. Utility Function Diagram Begin of Utility Function U1 Call Generic Function for the Object (Say G1) If Boolean State is True then Continue Perform Utility Action on the Object Log the Performed Utility Action Else Log Utility Action could not be Performed and Exit End of Utility Function U16.6 Test Case Functions These functions are very specific to the test cases and they utilize the generic and utility functions. These functions perform the test case execution along with the data passed on to them with by the Driver/ Controller. They also call custom validation functions, which are specific to the test case/ test step validation. After completing the tasks these functions, return the Test Execution Status as return value to the Driver/ Controller. Based on this Status the Test Result is also classified and it then populated into the Data Table. © Tech Mahindra Limited 4/23/2013
  • 12. Test Case Function Diagram Begin of Test Case Function TC1 Start Test Case Execution Timer Log the Test Case Name and Test Case Execution Start Time Call Utility Functions Call to Test Case/ Logic Implementation Functions Call to Custom Validation Functions Stop Test Case Execution Timer Log the Test Case Execution End Time and Time Elapsed Return the Test Execution Status as Pass/ Fail End of Test Case Function TC16.7 Logger Functions Logging functions are specific to creating logs for each step of action performed, verification and validation points and for logging Errors. The Logs are broadly classified into Step Logs and Error Logs. All the test case activities along with the respective time stamps which include date and time of the occurrence of the events are appended to Step Logs. Only Errors are reported in append mode into the Error logs along with the respective time stamps which also include date and time of the occurrence of the events. In case if the error logs are growing then it is an indication that some thing is going wrong in the test execution. As the logs are logged for each step it gives a clear idea about what was happening on each step. Also in the event of an error, it will be easy to locate the step where the error occurred, the nature of the error and the date and time at which the error occurred. Logger Function Diagram Begin of Logging Function LF1 Get the String of Data that has to be appended to the Log File Get Handle to the Log File Append the String of Data to the Log File Save and Close the Log File and free the Handle End of Logging Function LF1 © Tech Mahindra Limited 4/23/2013
  • 13. 6.8 Timer Functions The Timer functions consist of functions for date and time. It provides the time stamp which consist of start date and time at the start and end of each Test Run (can be Batch/ Test Suite) as well as for the beginning and end of each individual Test Case Execution. In addition, they provide a summary of Total Time Duration for each Test Run as well as for each individual Test Case Execution. GetTime Function Diagram Begin of GetTime Function GT Get Current Time Return Time in Hrs:Min:Sec String format End of GetTime Function GT GetDateTime Function Diagram Begin of GetDateTime Function GDT Use GetTime Function Get Current Date Convert Date to DD-MM-YYYY String format Join the Date and Time in String format and Return the value End of GetDateTime Function GDT GetTimeDuration Function Diagram Begin of GetTimeDuration Function GTD Get Start Time in Integer format Get End Time in Integer format Calculate the Difference between Start and End Time Return the Time Difference in Hrs:Min:Sec String format End of GetTimeDuration Function GTD © Tech Mahindra Limited 4/23/2013
  • 14. 7. CONCLUSION The Conclusion is that Hybrid Framework is rugged, easy to implement, easy to use, easy to expand, easy to maintain and easy to perpetuate. It is separate from the Test Design and it is also Technology and Platform independent. Once implemented it improves the speed of automating the Test Cases with the help of reusable libraries and provides better Cost Benefits/ ROI within a short span of time. © Tech Mahindra Limited 4/23/2013