C:\documents and settings\selvam.mc\my documents\automation testing process

836 views

Published on

Automation Testing framework document for QTP, LoadRunner

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
836
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
46
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

C:\documents and settings\selvam.mc\my documents\automation testing process

  1. 1. Large - Scale Test Automation Production<br />Date :09th July’2010<br />Place :Chennai Olympia Tech Park, India<br />Anbhu Selvam.MC…MS(CS).MBA.<br />(Senior Software Quality Engineer, EDS. India)<br />
  2. 2. Common Challenges<br />Automation Technology<br />–Multiplatform (cross-platform) support<br />•Ability to interface with multiple platforms<br />•Ease of extensibility to new platforms<br />•Reusability of test suites/scripts across multiple platforms<br />–Output/Object validation support<br />•High rate-of-change<br />–Interface changes<br />–Functionality changes<br />
  3. 3. Large-scale Automations Method-Centric<br />Automation is Method-Centric—it is more than about the technology; it is about the effective application of technology<br />–Maintainability—handling high rate-of-change<br />–Reusability—ability to reuse common test components without having to program/code<br />–Scalability—ability to automate large volume of tests through reusability (and team-based cost efficiency)<br />–Visibility<br />•Tests auditable by management and non-programmer staff<br />•Productivity is structurally measurable<br />
  4. 4. Efficiency is Key!<br />Efficiency Key Contributors<br />•Test design<br />Definition: <br />A test (case) creation activity with intent to serve one or both of the following objectives: <br />1.Exposing bugs/errors (or show otherwise, it works as intended), and <br />2.Optimizing maintainability and scalability (especially for automation-readiness).<br />•Test automation (large-scale)<br />–Method-centric, not tool-centric<br />–Team-based, not individual coders<br />
  5. 5. Automation Strategy Must Include <br />Efficient Practices<br />–Methodology<br />•High maintainability (low maintenance cost)<br />•High reusability<br />–Technology<br />•High extensibility<br />•Object validation capability<br />•People and process<br />–High scalability (large-volume and team-based methodology)<br />–Just-in-time automation—test engineers are skilled at test design and automation can be done before code-complete and/or concurrent to exploratory testing<br />–High manageability (high visibility)<br />
  6. 6. "Agile" System Development<br />Establish objectives (global)<br />•Develop tests<br />•Develop the system <br />–in iterations, one subsystem at the time<br />–test-driven<br />•Release<br />Agile ―Home Ground*‖<br />•Low mission-critical<br />•Senior developers<br />•Requirements change often<br />•Small number of developers<br />•Culture that thrives on chaos<br />
  7. 7. ReservedRole of Automation<br />Effective automation can allow tests to support agile system development<br />•Automation should not dominate. Don't make it into an "agile automation" project<br />•A keyword driven method is, in my view, essential for successful test development w/ automation in agile development environment<br />–Automation is separate from test development<br />–Usually doesn’t need user interaction (only some with testers)<br />–Automation focuses on action-keywords, not on tests<br />
  8. 8. DEVELOPMENT<br />Stakeholders <br />Testers <br />Agile Test Development Process<br />AUTOMATION<br />AUTOMATION<br />AUTOMATION<br />System Development <br />Level –II Development <br />
  9. 9. Recommendations<br />Have a global test-design scheme<br />–Keep it short and simple: focused on breakdown of the tests<br />–This global test design can (and usually should) be revisited throughout the project cycle<br />•Separate (1) test objectives and (2) test cases<br />–Test objectives are easier to create/describe than test cases<br />•Use your stakeholders (specialists, power-users, etc.) wisely<br />–Focus the efforts on the relevant tests<br />–Don’t bother them more than needed<br />•Avoid involving them in low-and medium-level tests<br />–Observe various types of input: <br />•How does this work?<br />•What do we need to test?<br />•What is interesting to test/How can we break the system?<br />•Keep automation separate<br />–Test developers are not consumed by automation<br />–Use keyword-driven approach (e.g., Action Based Testing)<br />
  10. 10. EAS: Extended Automation Support<br />
  11. 11. 6000<br /> 3000<br />250<br />200<br /> 22<br /> 20<br />Application (High-level)Actions<br />TESTS<br />Built-in/System(Low-level)Actions<br />Scalability Illustration3000<br />
  12. 12. Division of Work<br />
  13. 13. Key Takeaways<br />1. Fully understand automation cost-of-ownership<br />2.Don’t underestimate the challenge of keeping maintenance costs low<br />3.You need to get efficient—optimize your volume of test to exceed 50% coverage<br />4.Efficiency is key, and it will come from excellent test design and automation methodology (e.g., action-driven), and a well architected framework technology<br />5.Minimize programming tests<br />6.High scalability comes from high reusability of common ―actions‖ and team-based staffing model <br />7.High maintainability comes from keeping maintenance activities at the lowest level<br />8.High visibility in your automation program to give you control and measurability, which ultimately leads to manageability<br />9.Practice just-in-time automation<br />10.Have a global test-design scheme that separate test objectives from test cases; and automation from test development<br />
  14. 14. Automation Cost of Ownership&The Need for Large-Volume Automation<br />
  15. 15. Automation Cost of Ownership<br />1.Technology = Tool Licensing + Development/Customization Cost<br />2.Production Cost = *One-time Cost + **Recurring Cost<br />*NEW–One-time Cost of Design, Implementation & Execution<br />–Test Case or Test Script Creation <br />»Test line creation <br />»Sequencing test logic or order of test lines (test steps) <br />»Testing and debugging test script <br />–Function or Keyword Creation <br />–Interface Capturing or Mapping <br />–Test Dataset Creation <br />**EXISTING—Recurring Cost Test Execution & Maintenance Cost <br />–Monitoring test suit execution <br />–Investigating and troubleshooting false negatives <br />–Maintaining Test Case or Test Script<br />»Test line modification <br />»Updating test logic or re-sequencing order of test lines (test steps) <br />»Testing and debugging updated test script <br />»Function or Keyword Modification <br />»Interface Recapturing or Remapping <br />»Test Dataset Modification <br />
  16. 16. The Cost of Automation<br />The Cost of Owning an Automated Test<br />Cost of an Automated Test*= **Cost of Ownership / Volume (#) of Tests<br />* All test cases are not equal—a clear and structured definition of a test case is required. <br />Furthermore, this cost will be a moving target over time<br />** Cost of Ownership = Technology Cost + Production Cost<br />
  17. 17. The Need for Large Volume <br />Cost of an Automated Test*<br />=**Cost of Ownership/Volume (#) of Tests <br />You need to get efficient by optimizing the Volume of Tests! <br />* All test cases are not equal—a clear and structured definition of a test case is required. <br />Furthermore, this cost will be a moving target over time <br />** Cost of Ownership = Technology Cost + Production Cost <br />
  18. 18. Rules of Automation Built-to-Last<br />1.Tests are treated as product asset, along with the source code.<br />2.Tests, good or bad are dependent on the design.<br />3.Tests, manual or automated must be optimized for visibility, reusability, scalability and maintainability.<br />4.Tests must be automation-ready.<br />5.Tests, if they are worth automating, should follow the 5% rule:<br />No more than 5% of all tests should be executed manually<br />No more than 5% of all efforts around testing should involve automating the tests<br />No more than 5% of coded test scripts against non-coded test scripts.<br />
  19. 19. Action Based Testing Example<br />
  20. 20. Example with Test Architect™ (TA)<br />AT Computer <br />Mobile API<br />TCP/IP<br />Wireless<br />AT Agent<br />
  21. 21. Reserved Test Example<br />
  22. 22. Action Definition for ―send SMS message and check‖ action<br />
  23. 23. TA Connects to Window Mobile via TCP/IP<br />TCP/IP<br />Wireless<br />
  24. 24. Interface Definition for ―message editor‖ Entity<br />
  25. 25. Questions ?<br />

×