Testing Tools Management Methodology
How to Manage
your Testing Automation Project


Lior Katz - Testing Automation & perf...
Who am I ?




2   Footer Goes Here
“Test Automation is probably the most
     misunderstood concept in the field of
     software testing. The automation too...
“In general, the efforts to implement
     Automated Tools and realize ROI have been
     marginally successful. One of th...
Testing Automation projects
            Testing Automation project life cycle

        1      Purchase the tool     2   Re...
Testing Tools Management methodology




7   Footer Goes Here
TTM methodology - motivation

A methodology developed by myself after years
of experience, to understand the market need a...
Why Do We Fail?


    We have the best people and the best tools
        but still most projects are failing
           ma...
Why Do We Fail
How to succeed with your testing automation project using the TTM

– management support (not enough)
– high...
TTM process - 4 steps
1. Define the project scope
2. Follow up
3. Testing Automation methodology
4. closer




11   Footer...
1. Define the Project Scope
– High Level Design (Test Plan)
     The document contains the Set we are about to automate, t...
1. Define the Project Scope
– Design (Test cases QC)
     Test cases developed in QC cover all required area in the applic...
1. Define the Project Scope
– Estimation module
     A tool can assist to understand how many working days we need for the...
1. Define the Project Scope
– Work plan
     Following the estimation module solution the exact work plan by days




15  ...
2. Follow Up - Test Cases list




16   Footer Goes Here
2. Follow Up - Work plan




17   Footer Goes Here
3. Testing Automation Methodology

– QTP Standard and Guidelines
     Documents introduces basic guidelines and code metho...
3. Testing Automation Methodology

– Code inspection
     All of the inspectors to reach consensus on a work product and a...
4. Closer activities

– Final Report - A document contains all project steps include:
          – Test case list
         ...
4. Closer activities

– Code Freeze
     A process to save all developed code under one place. (With a limited access)



...
Testing Automation Stages
– Sanity SET
     Set of ~50 QTP scripts, verify that new build (version) ready for massive test...
TTM methodology - ROI example

– Each execution was reduced by ~20 working days

– No. of execution per version - ~7

– To...
Thank you all!!!




24   Footer Goes Here
How to manage your testing automation project   ttm methodology
Upcoming SlideShare
Loading in...5
×

How to manage your testing automation project ttm methodology

3,256

Published on

מנהלים רבים וארגונים רבים מיישמים אוטומציה בתהליך הבדיקות שלהם אבל עדיין מרגישים שההחזר על ההשקעה נמוך ואף שלילי. מחקרים רבים מראים כי הבעיה נובעת מחוסר תיאום ציפיות, זיהוי לא נכון של הבעיות שהכלים באים לפתור, בחירת כלי לא מתאים ותהליך הטמעה שגוי.
מתודולוגיית TMM (Testing tools management) באה לתת מענה בדיוק לבעיות שהוצגו. המתודולוגיה כוללת הגדרת השלבים השונים בפרויקט אוטומציה, החל מהגדרת הבעיה, דרך בחירת הכלי, בחינת הכלי, הטמעה ומדידת האפקטיביות שלו לכל אורך הפרויקט

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,256
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
82
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

How to manage your testing automation project ttm methodology

  1. 1. Testing Tools Management Methodology How to Manage your Testing Automation Project Lior Katz - Testing Automation & performance leader, Tact testing (Matrix) Jul 14th 2010 1 ©2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice
  2. 2. Who am I ? 2 Footer Goes Here
  3. 3. “Test Automation is probably the most misunderstood concept in the field of software testing. The automation tool implementation in testing is a costly process and requires careful planning and execution. Test Automation has to be treated like a full blown software development project as the final key deliverable, is in fact a software code ” –Software test practitioners 4 Footer Goes Here
  4. 4. “In general, the efforts to implement Automated Tools and realize ROI have been marginally successful. One of the main reasons for this failure has been the lack of planning and inadequate funding needed to deliver key initial services around the deployment of tools, training, and mentoring of resources by the purchasing parties. Many IT managers still believe that the tools will and should work by themselves.” 2009 WORLD QUALITY REPORT 5 Footer Goes Here
  5. 5. Testing Automation projects Testing Automation project life cycle 1 Purchase the tool 2 Recruit developers 3 Understand ROI 4 Fail to implement Should be - Testing Automation projects life cycle Analyze the systems Recruit developers Follow TTM 1 2 3 Understand ROI 4 Purchase the tools and managers methodology 6 Footer Goes Here
  6. 6. Testing Tools Management methodology 7 Footer Goes Here
  7. 7. TTM methodology - motivation A methodology developed by myself after years of experience, to understand the market need and the breaking points of testing automation projects and load testing projects. 8 Footer Goes Here
  8. 8. Why Do We Fail? We have the best people and the best tools but still most projects are failing management expectations 9 Footer Goes Here
  9. 9. Why Do We Fail How to succeed with your testing automation project using the TTM – management support (not enough) – high expectations for immediate ROI – Unrealistic expectations from a tool – Underestimating the time, cost and effort for the initial introduction and implementation of a tool (e.g. training, external expertise and changes to the testing process). – Underestimating the effort required to maintain the test scripts and tool updates – Over-reliance on the tool results – robust scripting –Understand the application and the development process – How to divide our automation project in order to succeed? A MUST is to decide first regarding Testing automation packages progress (Sanity, Mini Regression, Regression, divide by modules) 10 Footer Goes Here
  10. 10. TTM process - 4 steps 1. Define the project scope 2. Follow up 3. Testing Automation methodology 4. closer 11 Footer Goes Here
  11. 11. 1. Define the Project Scope – High Level Design (Test Plan) The document contains the Set we are about to automate, the concept of automation, verification points, parameters, etc… 12 Footer Goes Here
  12. 12. 1. Define the Project Scope – Design (Test cases QC) Test cases developed in QC cover all required area in the application 13 Footer Goes Here
  13. 13. 1. Define the Project Scope – Estimation module A tool can assist to understand how many working days we need for the project, include all required steps: • Testing priority • All Automation steps • Risk factor 14 Footer Goes Here
  14. 14. 1. Define the Project Scope – Work plan Following the estimation module solution the exact work plan by days 15 Footer Goes Here
  15. 15. 2. Follow Up - Test Cases list 16 Footer Goes Here
  16. 16. 2. Follow Up - Work plan 17 Footer Goes Here
  17. 17. 3. Testing Automation Methodology – QTP Standard and Guidelines Documents introduces basic guidelines and code methods for scripting via QTP. The document describes all acceptable practices to develop in QTP: 18 Footer Goes Here
  18. 18. 3. Testing Automation Methodology – Code inspection All of the inspectors to reach consensus on a work product and approve it for use in the project 19 Footer Goes Here
  19. 19. 4. Closer activities – Final Report - A document contains all project steps include: – Test case list – Testing automation coverage – Functions – Folder where script – How to execute the SET – Result example 20 Footer Goes Here
  20. 20. 4. Closer activities – Code Freeze A process to save all developed code under one place. (With a limited access) 21 Footer Goes Here
  21. 21. Testing Automation Stages – Sanity SET Set of ~50 QTP scripts, verify that new build (version) ready for massive testing process – Quick ROI Developments Packages can retrieve the investment immediately (e.g. API, data driven, data inflation) – Regression by modules - Do Not to develop all regression tests in one SET, “Before moving forward with the development stage, we need to analyze our system and divide it by modules, for each module to develop it’s own automation SET” – Data Inflation • Using our GUI interface can help with massive data changes 22 Footer Goes Here
  22. 22. TTM methodology - ROI example – Each execution was reduced by ~20 working days – No. of execution per version - ~7 – Total = a reduction of 140 working days per version – One working day = ~225$ Saved for the organization ~31500$ per version 23 Footer Goes Here
  23. 23. Thank you all!!! 24 Footer Goes Here
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×