Your SlideShare is downloading. ×
Whitepaper - Automated Functional Equivalence Testing
Whitepaper - Automated Functional Equivalence Testing
Whitepaper - Automated Functional Equivalence Testing
Whitepaper - Automated Functional Equivalence Testing
Whitepaper - Automated Functional Equivalence Testing
Whitepaper - Automated Functional Equivalence Testing
Whitepaper - Automated Functional Equivalence Testing
Whitepaper - Automated Functional Equivalence Testing
Whitepaper - Automated Functional Equivalence Testing
Whitepaper - Automated Functional Equivalence Testing
Whitepaper - Automated Functional Equivalence Testing
Whitepaper - Automated Functional Equivalence Testing
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Whitepaper - Automated Functional Equivalence Testing

640

Published on

Datamatics is a global Information Technology (IT) and Knowledge Process Outsourcing (KPO) organization focused on delivering smart, next-generation business solutions that help enterprises across the …

Datamatics is a global Information Technology (IT) and Knowledge Process Outsourcing (KPO) organization focused on delivering smart, next-generation business solutions that help enterprises across the world overcome their business challenges. These solutions leverage innovations in technology, knowledge of business processes, and domain expertise to provide clients a competitive edge. Visit www.datamatics.com

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
640
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
8
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. WHITEPAPER Automated Functional Equivalence Testing A White Paper by: Sagar M. Patil December, 2012www.datamatics.com Copyrights © 2012 Datamatics Global Services Ltd. All rights reserved. 1
  • 2. Table of Contents Executive Summary ........................................................................ 3 Typical Migration Testing Challenges ......................................... 4 Automation Approach for Functional Equivalence Testing .... 4 1. Design Phase .......................................................................... 5 2. Script Development Phase ................................................... 5 3. Script Execution Phase ......................................................... 6 4. Analysis .................................................................................... 6 Functional Equivalence Framework for any Migration Project .............................................................................................. 7 Automation Framework for Enhanced ROI ................................ 7 ROI Calculation ................................................................................ 8 Complete Application Migration is Planned Over Multiple Releases ........................................................................................... 8 Conclusion ....................................................................................... 11 About the Author ............................................................................. 12 About Datamatics Global Services .............................................. 12www.datamatics.com Copyrights © 2012 Datamatics Global Services Ltd. All rights reserved. 2
  • 3. Executive Summary There is a growing need for migration from legacy Visual Basic 6.0 to VB.NET or Java technology as Microsoft has discontinued support for VB6. To leverage benefits of new technology and mitigate risk, organizations have started migration of their applications to .NET or Java technology. Migration projects typically involve extensive testing. The keyFor migration projects challenge is to ensure that functional equivalence is achieved oninvolving extensive the target application. An Automated Functional Equivalencetesting, an automated Testing (AFET) solution therefore becomes a crucial part of thefunctional equivalence migration process. This is reusable, smooth and cost effectivetesting solution is means to achieve 100% functional Equivalence testing on anycrucial migration project. This Paper focuses on methodologies and techniques that Datamatics has developed for automation of functional testing, when a source VB project is migrated to .NET.www.datamatics.com Copyrights © 2012 Datamatics Global Services Ltd. All rights reserved. 3
  • 4. Typical Migration Testing Challenges When a legacy application is migrated as- is to a newer technology, e.g. VB (Source Application) to Java (Target Application) or VB to .NET, some of the typical challenges faced during testing include: 1. Effort intensive Regression Testing 2. Time constraints for development and testing cycle 3. Manual testing - being labor intensive, contributes to higher costsDatamatics’ AFET 4. Insufficient test coverageapproach is to use 5. Data consistency verificationsingle automation 6. Lack of automation tools for functional equivalence testingscript on source andtarget platform Approach for AFET The functional equivalence tools available in the market support only bitmap comparison and have limited functionality when it comes to verification testing. Datamatics overcomes this constraint through use of a single automation script on source and target platform to achieve functional equivalence verification. Here is how it works: - The automation script is developed using QTP 11 [Functional Automation Testing Tool] - The business process transaction is scripted on the source platform and then executed on target platform - Any discrepancy from the source application is flagged as a failure on the target application execution - The verification includes comparison test both at controls (buttons, list box etc) and at functional level The high level steps in a testing process for AFET are: 1. Design 2.Script Development 3.Script Execution 4.Analysis Automated Functional Equivalence Testing Process 1 Design 2 Script Dev 3 Script Exec 4 Analysis - Understand Develop description First execute script Analyse the functionality based script for on VB application discrepancies against - Analyze the reusable for VB as and mark results as expected result and reusable well as .NET baseline report errors Figure 3.0www.datamatics.com Copyrights © 2012 Datamatics Global Services Ltd. All rights reserved. 4
  • 5. 1. Design Phase a. Source VB “Application under Test” (AUT) is analyzed for identification of reusable functions/actions that need creation of scripts b. Input and output parameters and dependences are documented c. Structure of Object Repository is analyzed QTP VB/.NET scripts Microsoft®The high level steps in Visual Basic6.0a testing process for VB ApplicationAFET are: Shared Framework Folder with reusable1. Design Microsoft® actions .NET Script Compare Report2. Script Development Execution Results Errors .NET Application3. Script Execution under Test4. Analysis Figure 3.1 2. Script Development Phase a. Based on the analysis of reusable actions/functions, either objects from source VB application are added to object repository or a unique description of each object is created b. Descriptions are created in a manner such that each object is uniquely identifiable on VB (source) as well as .NET (target) application c. Once the description of objects is created, object is added to object repository and functional logic is developed d. The process used to develop the Script ensures: i. Report displays total number of controls on form ii. The properties on VB and .NET application (e.g visibledisable), total number of items in List box etc as well as excepted result of functionality are being tested e. It is important to note that while the scripts are being created on the VB application and the development of .NET application is in progress, the developed scripts are fully compatible to run on the .NET application f. Once .NET application is available, the automated scripts on the source (VB) application after minimal modification can be used for testing on target (.Net) Applicationwww.datamatics.com Copyrights © 2012 Datamatics Global Services Ltd. All rights reserved. 5
  • 6. Microsoft® Shared Framework Visual Basic6.0 Folder with reusable actions VB QTP Tool QTP Application Scripts Figure 3.2 3. Script Execution Phase a. In script execution phase, automation scripts are executed on VB (Source) application and the results are stored in datasheets that can be later used as a baseline for test verification b. The same scripts with the data from stored datasheets are then executed on .NET application followed by comparison of results against baseline c. Any deviance from the baseline is analyzed further Script Execution on Source Application Microsoft® Visual Basic6.0 VB QTP Tool QTP VBApp Script Base Line Application Scripts Execution Results Script Execution on Destination Application Report Errors Microsoft® .NET .NET QTP Tool QTP Script Compare Application .NETApp Execution Results Under Test Scripts Figure 3.3 4. Analysis a. A Root cause analysis of scripts that failed on .NET application is performed b. If script failure is due to functional issues, then it is reported as a defect and sent to the development team for fixeswww.datamatics.com Copyrights © 2012 Datamatics Global Services Ltd. All rights reserved. 6
  • 7. Functional Equivalence Framework for any Migration Project With this approach, one can create a simple excel based framework for object identification properties and methods to be used by the tool for object identification at runtime. Here is an example of object properties for VB and .NET applicationA n automationframework helpsachieve betterproductivity, fasterturnaround time andhigher accuracy Figure 4.0 Automation Framework for enhanced ROI ROI for AFET projects can be enhanced through use of frameworks. An automation framework helps achieve better productivity, faster turnaround time and higher accuracy. For the project being discussed in this paper, Datamatics’ proprietary Automation Regression Testing (DARTS) framework was used: DARTS© Framework – Key Benefits and features: 1. Complete control over the Test Planning, Execution and Test Execution Reporting Cycle 2. Technology Specific Component functions to access technology specific features 3. Greater control over AUT using built-in feature to check state of the objectwww.datamatics.com Copyrights © 2012 Datamatics Global Services Ltd. All rights reserved. 7
  • 8. 4. Reusable Business actions built using component functions, aids to create granular test cases 5. Reusable architecture 6. Agile framework enables seamless data transfer across business scenarios and better control over execution for the tester 7. Recovery scenarios are integrated at the framework level. For example application types like web based, DotNet, VB based etc. ROI Calculation For calculating ROI on AFET projects, following factors need to be considered: 1. Cost per test automation resourceBetter ROI on projects 2. Number of test cases considered for automationthrough AFET 3. Automation tool cost 4. Time required for creating reusable scripts that work on VB as well as .NET application 5. Releases planned per cycle 6. Number of regression cycles planned Complete Application Migration is Planned Over Multiple Releases Basis of ROI Calculation has been provided below Data Product Details Source Product Technology VB VB Migration Product Technology Vb.NET Vb.NET Testing Team Size Required For Current Scope Manual 4 Automation 2 Release Details No. of Releases Planned 4www.datamatics.com Copyrights © 2012 Datamatics Global Services Ltd. All rights reserved. 8
  • 9. Test Execution Data Data No. of Test Cases Source Product Technology 500 VB Migration Cycles Technology No. of TestProductPer Release 2 Vb.NET No. of Test Cases Increase Per Cycle 50 Manual Testing Effort Parameters Total Test Cases Available 500 Cycle Duration in Working Days 10 No. of Working Days 5 Working Hours 8 Productivity 15 Automation Testing Effort Parameters Avg. Cost Manual/Automation Resource $40 US Per Hour No. of Execution Hours 8 Execution Productivity for 1 Cycle 30 Execution Productivity for 2 Cycle Onwards 45 Automation Tool License Cost $20000 US Figure 7.0www.datamatics.com Copyrights © 2012 Datamatics Global Services Ltd. All rights reserved. 9
  • 10. Note: From Release 4 onwards The Cumulative Cost of Automation Testing is lower than the cost of Only Manual Testing Figure 7.1 Cumulative Cost Graph 300000 250000 200000 Cost ($ US) 150000 100000 50000 0 1 2 3 4 5 6 7 8 9 10 11 12 Cycles Cumulative Manual Testing Total Cost Cumulative Automation Cost Figure 7.2www.datamatics.com Copyrights © 2012 Datamatics Global Services Ltd. All rights reserved. 10
  • 11. Conclusion For projects involving migration of legacy applications to newer technologies, the discussed process for automated functionalBenefits of AFET: equivalence testing provide following benefits:single script which is 1. Single script can be used to test Functional Equivalence onconsistent, repeatable Source and Target Applicationand accurate; reduced 2. Script execution is consistent , repeatable and accuratecost, time and efforts 3. Reduction in development and maintenance efforts for automation scripts. The automation strategy and frameworks can be used for any migration project 4. Reduced cost due to faster QA cycles and reduced manual testing efforts 5. This process enables the flexibility of choosing any automation tool for any migration testing projectwww.datamatics.com Copyrights © 2012 Datamatics Global Services Ltd. All rights reserved. 11
  • 12. About the Authors Sagar M. Patil Manager/Architect-Test Automation At Datamatics Sagar is responsible for providing automation solutions for diverse technologies and platforms. He has 12 years of experience in IT industry and his core expertise lies in Manual Testing, Test Automation, performance testing & Test Management. He has handled multiple complex testing projects using automation tools like QTP, Selenium, TestCompelte, CodedUI, Ranorex which includes framework development. Software Testing is his passion and he likes to address each challenge with an innovation but with simple solution. Sagar is PMP, CSTE, HP Functional test, Six sigma green belt certified professional. His white paper has been awarded in Testride Mumbai conference.About Datamatics Global Services Global Information Technology (IT) & Knowledge Process Outsourcing (KPO) organization Delivers smart, next-generation business solutions Trusted partner to several Fortune 500 companies Capabilities built around technology, domain expertise & knowledge of business processes Featured amongst the Global Services 100 List in 2010 & 2011 Decades of global experience having executed projects across 60 countries 2 Alliances with global technology leaders such as Microsoft, IBM & EMC Certified for SEI CMMI Level 3 V1.3, ISO 27001:2005 ISO 9001:2008 SSAE 16 compliant processes Global presence: U.S, UK, Germany, Switzerland, Bosnia, Australia, Singapore & IndiaTo know more, connect with us on business@datamatics.comwww.datamatics.com Copyrights © 2012 Datamatics Global Services Ltd. All rights reserved. 12

×