• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Software Testing: Application And Script Independent Automation Framework: The Need For Data Normalization
 

Software Testing: Application And Script Independent Automation Framework: The Need For Data Normalization

on

  • 5,568 views

 

Statistics

Views

Total Views
5,568
Views on SlideShare
5,355
Embed Views
213

Actions

Likes
8
Downloads
0
Comments
2

17 Embeds 213

http://at4qa.blogspot.com 85
http://www.geektester.blogspot.com 60
http://geektester.blogspot.com 36
http://www.slideshare.net 11
http://at4qa.blogspot.in 6
http://at4qa.blogspot.co.il 3
http://www.at4qa.blogspot.com 2
http://at4qa.blogspot.de 1
http://at4qa.blogspot.dk 1
http://at4qa.blogspot.no 1
http://at4qa.blogspot.se 1
http://at4qa.blogspot.pt 1
http://at4qa.blogspot.fr 1
http://at4qa.blogspot.com.au 1
http://at4qa.blogspot.ie 1
http://at4qa.blogspot.com.br 1
http://at4qa.blogspot.it 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

12 of 2 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • super....
    Are you sure you want to
    Your message goes here
    Processing…
  • cool !

    can u send it to ajophilip@gmail.com?
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Software Testing: Application And Script Independent Automation Framework: The Need For Data Normalization Software Testing: Application And Script Independent Automation Framework: The Need For Data Normalization Presentation Transcript

  • Application & Script Independent Framework: The Need For Data Normalization ASIF Date: Wednesday, 14 th November, 2007 Venue: Orlando, Florida Presenter: Raj Kamal Company: Microsoft, India
    • Name: Raj
    • Indian citizen
    • 25 years
    • First time in US
    • Flew over 7718 + miles to attend QAI
    • Basically from a village in Haryana , India.
    • Sports – Ping Pong, Cricket and Pool
    • Like learning new technologies and gadgets
    Who am I ? Born in Delhi Work @ MS Hyderabad
  • Research
  • Past Research
    • Iterative Automation Process using RUP
    • ART (Automation Review Tool )
    • Mapping RUP to Rational Testing Tools
      • Presented at http://ifstep.org/
    • ROI of Testing
    http://geektester.blogspot.com
  • Big Question
  • Agenda
    • Industry Quotes
    • Problems
    • Solution
    • Architecture
    • Implementation
    • Benefits
  • Survey “ Requirement Changes are inevitable”
  • “ In QAI survey w hen asked, “ To what degree are the test products maintained and reused ?” 33% said always, 59% sometimes, and 8% never ” ” Only 19.4% of respondents said they used some type of automated testing tool, the remainder relying upon manual testing methods.” “ The prevailing reasons given for not automating test processes were: Time (35%); cost (33%); script/data maintenance (26%); and complexity (18%) “ Survey: Big Spends, Little Comfort
  • “ requirements; the real joke is that the requirements end up being what the source code does.” “ 62 % of respondents stated they could be driving a new Mercedes  or retired if they had a dollar ($) for every time they had to rewrite a test case because requirements changes weren't effectively communicated "it is often the case that development can slip on their timelines, but testing cannot.“ Survey: Automation is everywhere
  • Available Automation Frameworks
  • What’s Available
    • Record / Playback
    • Data Driven Framework
    • Modular Architecture
    • Keyword Driven Framework
    • Hybrid Framework
    • http://en.wikipedia.org/wiki/Test_Automation_Framework
  • Low ROI & High Maintenance Effort ITERATIVE DEVELOPMENT REDUNDANCY INCONSISTENCY PROBLEM: CAUSES OF AUTOMATION FAILURE Frequent updates Multiple versions Difficult to Debug Common Test Data Automation Users Complexity Grows Change Requests Phases Implementation Frequent Script updates SKILLS People Issues Common Flow Frequent Failures Coding
  • Problem #1: Redundancy
    • Scenario:
    • Test Data Sets : 500 Tables / Sheets
    • Test Cases Covered: 1000
    • Testers: 4
    • Modules: 4
    Unintentional Repetition of Test Data & Multiple copies of the same data maintained
  • Problem #1: Redundancy
    • Future Release Changes
    • - Object Type
    • - Object Name
    • - Object Data
    • Impact / Loss [ Domino effect ]
    • - Multiple Script Failure
    • - Rework updating multiple instances
    • - Loss of Data Integrity
  • Problem 2#: Inconsistency
    • Future Release Changes
    • - Object Type
    • - Object Name
    • - Object Data
    • Impact / Loss
    • Inconsistency – Different versions of Data
    • Tedious Process
    • Misleading Result / Invalid Bugs
  • Problem #3: Evolving Development Models
    • Agile Modeling
    • SCRUM
    • XP
    • RUP
    • Iterative Development
    • Microsoft Solutions Framework
    • And Many More …..
  • Problem #3: Evolving Development Models
    • Evolutionary Dev Models can handle changes gracefully and are becoming better every day….
    But Is our automation evolving / adapting with the same pace ?
  • Solution
  • Suggested Solution : ASIF
    • Data Normalization
    • User defined Views
    • Other RDBMS concepts
    • 100 % Reusable Plug and Play Library
    • Simplified Scripts (just to invoke library)
    “ ASIF, just like Rome, wasn't built in a day”
  • Solution #1: Normalization
    • Minimize duplication by storing only ONE copy of the Test Data
      • Object Type
      • Object Name
      • Object Data
    Have Test Data in Normal Form – 1 NF, 2 NF, 3 NF Eliminate Redundancy, Inconsistency
  • Solution #2: User Defined Views
    • 1 View ~ 1 Test Case
    • View = Table 1 + Table 2 + .. (JOIN)
    • View contain “FLOW” of Test Execution
    • Table Updated ~ Views get updated
    • Views get Updates ~ Tables get updated
    • Easy to create / drop / modify
    • Contains BUSINESS LOGIC
  • Solution #3: Reusable Library
    • ASIF Library
      • Talks with Views NOT Tables
      • Can Read / Understand / Update Views
      • Perform ACTION / VERIFICATION
      • Handles Errors & Exceptions
      • Can write logs / results (PASS / FAIL)
      • 100 % Reusable – DON’T CONTAIN BUSINESS LOGIC
      • Interface between Scripts & Test Data
  • Solution #4: Over Simplified Scripts
    • Scripts
      • Invoke ASIF Reusable Library
      • Passes “Test Case Id” as Parameter
      • DON’T CONTAIN BUSINESS LOGIC
      • DON’T CONTAIN ACTION CODE
      • DON’T CONTAIN VERIFICATION CODE
      • 90 % Reduction in LOC (Lines of Code)
  • Implementation
  • ASIF Automation Architecture Name Age Views Tables Scripts
  • Implementation: Automation Process Setup
  • Implementation: Database Setup
  • Implementation: Automation Test Tool
  • General Flow PK Object Name / Type Object Name / Type Object Name / Type 0R Userid~EditBox UPass~EditBox UName~EditBox TC_01 User1 Pwd1 Raj TC_02 User2 Pwd2 Sam
  • Database Design User Id Password XYZ Login Page OK Cancel User name : Age : XYZ User Information Raj 25 PK Object Name / Type Object Name / Type 0R Userid~EditBox UPass~EditBox TC_01 User1 Pwd1 TC_02 User2 Pwd2 PK Object Name / Type Object Name / Type 0R UName~EditBox UAge~EditBox TC_01 Raj 25 TC_02 Sam 28 PK Object Name / Type Object Name / Type 0R Userid~EditBox UName~EditBox TC_01 User1 Raj TC_02 User2 Sam
  • Example #1 – Change in Requirements
    • Requirement changes the business flow or logic
    • Updated only views written for that test case
    • Tables get updated automatically
    • Scripts need not to be touched.
    • Run scripts and validate the changes
  • Example #2 – Change in Design/Code
    • Object name / Object type is added / changed by a developer
    • Updated base table only at one place
    • Views get updated automatically
    • Scripts need not to be touched.
    • Run scripts and validate the changes
  • Cost
  • Cost – Manual Testing
    • Cost of manual Testing
    • = time to develop test cases
    •     + time to maintain test cases x number of times tests are executed
    •    + time to execute manual testing x number of times tests are executed
  • Cost- Automation using ASIF
    • Cost of automation Testing
    • = price of hardware
    • + price of software
    •    + time to develop scripts
    •    +   time to maintain scripts x number of times scripts are executed
    •    + time to execute scripts x number of times scripts are executed
  • Return on Investment
  • ASIF Return on Investment
    • ROI = (cost of manual – cost of automation) / cost of automation
    ASIF gives ROI of 72 % over the course of the year
  • Benefits
  • Benefits
    • Easy maintainability
    • Test Data Storage and Retrieval Efficiency
    • Better test coverage
    • Modularization
    • Gracefully handles scalability
  • ASIF Break-even Point Test Runs Cost of Testing n En = (Va + n * Da )/ (Vm + n * Dm ) V - Expenditure for Implementation D - Expenditure for single test execution
  • Benefits
    • Accelerated time to Market
    • Abstract and decouple test data from scripts
    • Provides 100 % Reusable Plug & Play Library
    • Reduced cost of quality
    • Decouple complex business function
  • Effort Reduction Releases Effort Application Complexity - Test Effort Reduction over Releases
  • Overall Performance Total Effort (%) Complexity Total Effort Reduction (%)
  • Why ASIF and not any other automation framework ?
    • Almost 90 % reduction in the script code
    • Handles requirement changes gracefully
    • ASIF Automation framework don’t take breaks, don’t need to eat, don’t take calls from their family 
  • Conclusion: When to Use ?
    • Requirements are Dynamic
    • Ideal for Iterative / Agile / SCRUM Model
    • At least 3 Test Runs to obtain ROI
    • Testers lack Coding skills
    • Ideal to run Regression and BVT Suite
    • Permanent long term solution for Client
  • Future Avenues of Research
    • Think New
    • Leverage artificial intelligence
    • Simplify ( Reduce Complexity )
    • - Adoption
    • - Use
    • - Maintenance
    • People independent
    • Automation Process Improvements
    Future Avenues of Research
  • Questions & Answers
  • References
    • http://www.borland.com/us/company/news/press_releases/2006/10_31_06_borland_qa_professionals_survey.html
    • http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=ART&ObjectId=8502
    • http://books.nap.edu/openbook.php?record_id=10695&page=137
    • http://www.logigear.com/newsletter/test_automation_top_5_suggested_best_practices.asp
    • http://findarticles.com/p/articles/mi_m3230/is_n12_v28/ai_19040722
    • http://www.itwales.com/998515.htm
    • http://www.borland.com/us/company/news/press_releases/2006/10_31_06_borland_qa_professionals_survey.html
    • http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=ART&ObjectId=8502
  • Suggested Readings
    • http://www.developsense.com/presentations/emotions%20and%20test%20oracles.ppt - Michael Bolton
    • http://www.satisfice.com/blog/ - James Bach
    • http://testertested.blogspot.com/ - Pradeep Soundarajan
    • Thank you 
    • [email_address]