Software Testing: Application And Script Independent Automation Framework: The Need For Data Normalization

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

4 comments

Comments 1 - 4 of 4 previous next Post a comment

  • + arunbharathi Anna 4 months ago
    super....
  • + guest3d9d4b guest3d9d4b 11 months ago
    I was wondering i you or anyone you know might be interested in a 2-3 month contract in Detroit. I am looking for 18 Action Script enterprise web developers who have experience dealing with flash based complicated Action Script code and retrieving data files from XML and incorporating it into the flash files. The candidates need to understand flash timelines on teh web site and be able to open the flash files and extract text as well as find the text strings in the code to make enhancements so it runs smother using the external data.

    I really need help. I am only finding UI Action Script Graphics people. Also, do you know of a site I could find these people at?

    Please e-mail me.

    Thanks
    Tiffany Allen
    tallen@judge.com
    248-253-9500 Ext. 1228
  • + guest3d9d4b guest3d9d4b 11 months ago
    I was wondering i you or anyone you know might be interested in a 2-3 month contract in Detroit. I am looking for 18 Action Script enterprise web developers who have experience dealing with flash based complicated Action Script code and retrieving data files from XML and incorporating it into the flash files. The candidates need to understand flash timelines on teh web site and be able to open the flash files and extract text as well as find the text strings in the code to make enhancements so it runs smother using the external data.

    I really need help. I am only finding UI Action Script Graphics people. Also, do you know of a site I could find these people at?

    Please e-mail me.

    Thanks
    Tiffany Allen
    tallen@judge.com
    248-253-9500 Ext. 1228
  • + ajophilip ajophilip 2 years ago
    cool !

    can u send it to ajophilip@gmail.com?
Post a comment
Embed Video
Edit your comment Cancel

5 Favorites

Software Testing: Application And Script Independent Automation Framework: The Need For Data Normalization - Presentation Transcript

  1. Application & Script Independent Framework: The Need For Data Normalization ASIF Date: Wednesday, 14 th November, 2007 Venue: Orlando, Florida Presenter: Raj Kamal Company: Microsoft, India
    • Name: Raj
    • Indian citizen
    • 25 years
    • First time in US
    • Flew over 7718 + miles to attend QAI
    • Basically from a village in Haryana , India.
    • Sports – Ping Pong, Cricket and Pool
    • Like learning new technologies and gadgets
    Who am I ? Born in Delhi Work @ MS Hyderabad
  2. Research
  3. Past Research
    • Iterative Automation Process using RUP
    • ART (Automation Review Tool )
    • Mapping RUP to Rational Testing Tools
      • Presented at http://ifstep.org/
    • ROI of Testing
    http://geektester.blogspot.com
  4. Big Question
  5. Agenda
    • Industry Quotes
    • Problems
    • Solution
    • Architecture
    • Implementation
    • Benefits
  6. Survey “ Requirement Changes are inevitable”
  7. “ In QAI survey w hen asked, “ To what degree are the test products maintained and reused ?” 33% said always, 59% sometimes, and 8% never ” ” Only 19.4% of respondents said they used some type of automated testing tool, the remainder relying upon manual testing methods.” “ The prevailing reasons given for not automating test processes were: Time (35%); cost (33%); script/data maintenance (26%); and complexity (18%) “ Survey: Big Spends, Little Comfort
  8. “ requirements; the real joke is that the requirements end up being what the source code does.” “ 62 % of respondents stated they could be driving a new Mercedes  or retired if they had a dollar ($) for every time they had to rewrite a test case because requirements changes weren't effectively communicated "it is often the case that development can slip on their timelines, but testing cannot.“ Survey: Automation is everywhere
  9. Available Automation Frameworks
  10. What’s Available
    • Record / Playback
    • Data Driven Framework
    • Modular Architecture
    • Keyword Driven Framework
    • Hybrid Framework
    • http://en.wikipedia.org/wiki/Test_Automation_Framework
  11. Low ROI & High Maintenance Effort ITERATIVE DEVELOPMENT REDUNDANCY INCONSISTENCY PROBLEM: CAUSES OF AUTOMATION FAILURE Frequent updates Multiple versions Difficult to Debug Common Test Data Automation Users Complexity Grows Change Requests Phases Implementation Frequent Script updates SKILLS People Issues Common Flow Frequent Failures Coding
  12. Problem #1: Redundancy
    • Scenario:
    • Test Data Sets : 500 Tables / Sheets
    • Test Cases Covered: 1000
    • Testers: 4
    • Modules: 4
    Unintentional Repetition of Test Data & Multiple copies of the same data maintained
  13. Problem #1: Redundancy
    • Future Release Changes
    • - Object Type
    • - Object Name
    • - Object Data
    • Impact / Loss [ Domino effect ]
    • - Multiple Script Failure
    • - Rework updating multiple instances
    • - Loss of Data Integrity
  14. Problem 2#: Inconsistency
    • Future Release Changes
    • - Object Type
    • - Object Name
    • - Object Data
    • Impact / Loss
    • Inconsistency – Different versions of Data
    • Tedious Process
    • Misleading Result / Invalid Bugs
  15. Problem #3: Evolving Development Models
    • Agile Modeling
    • SCRUM
    • XP
    • RUP
    • Iterative Development
    • Microsoft Solutions Framework
    • And Many More …..
  16. Problem #3: Evolving Development Models
    • Evolutionary Dev Models can handle changes gracefully and are becoming better every day….
    But Is our automation evolving / adapting with the same pace ?
  17. Solution
  18. Suggested Solution : ASIF
    • Data Normalization
    • User defined Views
    • Other RDBMS concepts
    • 100 % Reusable Plug and Play Library
    • Simplified Scripts (just to invoke library)
    “ ASIF, just like Rome, wasn't built in a day”
  19. Solution #1: Normalization
    • Minimize duplication by storing only ONE copy of the Test Data
      • Object Type
      • Object Name
      • Object Data
    Have Test Data in Normal Form – 1 NF, 2 NF, 3 NF Eliminate Redundancy, Inconsistency
  20. Solution #2: User Defined Views
    • 1 View ~ 1 Test Case
    • View = Table 1 + Table 2 + .. (JOIN)
    • View contain “FLOW” of Test Execution
    • Table Updated ~ Views get updated
    • Views get Updates ~ Tables get updated
    • Easy to create / drop / modify
    • Contains BUSINESS LOGIC
  21. Solution #3: Reusable Library
    • ASIF Library
      • Talks with Views NOT Tables
      • Can Read / Understand / Update Views
      • Perform ACTION / VERIFICATION
      • Handles Errors & Exceptions
      • Can write logs / results (PASS / FAIL)
      • 100 % Reusable – DON’T CONTAIN BUSINESS LOGIC
      • Interface between Scripts & Test Data
  22. Solution #4: Over Simplified Scripts
    • Scripts
      • Invoke ASIF Reusable Library
      • Passes “Test Case Id” as Parameter
      • DON’T CONTAIN BUSINESS LOGIC
      • DON’T CONTAIN ACTION CODE
      • DON’T CONTAIN VERIFICATION CODE
      • 90 % Reduction in LOC (Lines of Code)
  23. Implementation
  24. ASIF Automation Architecture Name Age Views Tables Scripts
  25. Implementation: Automation Process Setup
  26. Implementation: Database Setup
  27. Implementation: Automation Test Tool
  28. General Flow PK Object Name / Type Object Name / Type Object Name / Type 0R Userid~EditBox UPass~EditBox UName~EditBox TC_01 User1 Pwd1 Raj TC_02 User2 Pwd2 Sam
  29. Database Design User Id Password XYZ Login Page OK Cancel User name : Age : XYZ User Information Raj 25 PK Object Name / Type Object Name / Type 0R Userid~EditBox UPass~EditBox TC_01 User1 Pwd1 TC_02 User2 Pwd2 PK Object Name / Type Object Name / Type 0R UName~EditBox UAge~EditBox TC_01 Raj 25 TC_02 Sam 28 PK Object Name / Type Object Name / Type 0R Userid~EditBox UName~EditBox TC_01 User1 Raj TC_02 User2 Sam
  30. Example #1 – Change in Requirements
    • Requirement changes the business flow or logic
    • Updated only views written for that test case
    • Tables get updated automatically
    • Scripts need not to be touched.
    • Run scripts and validate the changes
  31. Example #2 – Change in Design/Code
    • Object name / Object type is added / changed by a developer
    • Updated base table only at one place
    • Views get updated automatically
    • Scripts need not to be touched.
    • Run scripts and validate the changes
  32. Cost
  33. Cost – Manual Testing
    • Cost of manual Testing
    • = time to develop test cases
    •     + time to maintain test cases x number of times tests are executed
    •    + time to execute manual testing x number of times tests are executed
  34. Cost- Automation using ASIF
    • Cost of automation Testing
    • = price of hardware
    • + price of software
    •    + time to develop scripts
    •    +   time to maintain scripts x number of times scripts are executed
    •    + time to execute scripts x number of times scripts are executed
  35. Return on Investment
  36. ASIF Return on Investment
    • ROI = (cost of manual – cost of automation) / cost of automation
    ASIF gives ROI of 72 % over the course of the year
  37. Benefits
  38. Benefits
    • Easy maintainability
    • Test Data Storage and Retrieval Efficiency
    • Better test coverage
    • Modularization
    • Gracefully handles scalability
  39. ASIF Break-even Point Test Runs Cost of Testing n En = (Va + n * Da )/ (Vm + n * Dm ) V - Expenditure for Implementation D - Expenditure for single test execution
  40. Benefits
    • Accelerated time to Market
    • Abstract and decouple test data from scripts
    • Provides 100 % Reusable Plug & Play Library
    • Reduced cost of quality
    • Decouple complex business function
  41. Effort Reduction Releases Effort Application Complexity - Test Effort Reduction over Releases
  42. Overall Performance Total Effort (%) Complexity Total Effort Reduction (%)
  43. Why ASIF and not any other automation framework ?
    • Almost 90 % reduction in the script code
    • Handles requirement changes gracefully
    • ASIF Automation framework don’t take breaks, don’t need to eat, don’t take calls from their family 
  44. Conclusion: When to Use ?
    • Requirements are Dynamic
    • Ideal for Iterative / Agile / SCRUM Model
    • At least 3 Test Runs to obtain ROI
    • Testers lack Coding skills
    • Ideal to run Regression and BVT Suite
    • Permanent long term solution for Client
  45. Future Avenues of Research
    • Think New
    • Leverage artificial intelligence
    • Simplify ( Reduce Complexity )
    • - Adoption
    • - Use
    • - Maintenance
    • People independent
    • Automation Process Improvements
    Future Avenues of Research
  46. Questions & Answers
  47. References
    • http://www.borland.com/us/company/news/press_releases/2006/10_31_06_borland_qa_professionals_survey.html
    • http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=ART&ObjectId=8502
    • http://books.nap.edu/openbook.php?record_id=10695&page=137
    • http://www.logigear.com/newsletter/test_automation_top_5_suggested_best_practices.asp
    • http://findarticles.com/p/articles/mi_m3230/is_n12_v28/ai_19040722
    • http://www.itwales.com/998515.htm
    • http://www.borland.com/us/company/news/press_releases/2006/10_31_06_borland_qa_professionals_survey.html
    • http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=ART&ObjectId=8502
  48. Suggested Readings
    • http://www.developsense.com/presentations/emotions%20and%20test%20oracles.ppt - Michael Bolton
    • http://www.satisfice.com/blog/ - James Bach
    • http://testertested.blogspot.com/ - Pradeep Soundarajan
    • Thank you 
    • [email_address]

+ guest0efb5eguest0efb5e, 3 years ago

custom

2903 views, 5 favs, 2 embeds more stats

More info about this document

© All Rights Reserved

Go to text version

  • Total Views 2903
    • 2807 on SlideShare
    • 96 from embeds
  • Comments 4
  • Favorites 5
  • Downloads 0
Most viewed embeds
  • 60 views on http://www.geektester.blogspot.com
  • 36 views on http://geektester.blogspot.com

more

All embeds
  • 60 views on http://www.geektester.blogspot.com
  • 36 views on http://geektester.blogspot.com

less

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

Cancel
File a copyright complaint
Having problems? Go to our helpdesk?

Categories