Successful Automation KDT as a Test Language Aviram Shotten AP Program Manager QualiTest
Today’s Menu <ul><li>Brief introduction to KDT </li></ul><ul><li>Practical KDT – how KDT needs to be implemented – 3 years...
Introduction to KDT <ul><li>KDT is 3 rd  generation automation approach (after record & replay and functional decompositio...
Introduction to KDT <ul><li>In order to better mitigate the drawbacks of automation that were mentioned before, KDT sugges...
Introduction to KDT <ul><li>Developing infrastructure: </li></ul><ul><ul><li>Mapping AUT components </li></ul></ul><ul><ul...
Requirements - KDT  <ul><li>We, as functional testers, requires from a KDT based solution: </li></ul><ul><ul><li>Simplicit...
Generic KDT Suite <ul><li>Experience led us to build 4 components to support a generic KDT implementation process among te...
AP Architecture Logical Layer  Infrastructure Layer   SUT Testing Tool AP  ENGINE VBS Or
A Test Case  Traditional Test Case Step # Action Expected Result 1 Execute the “MyApplic” application  Log in window appea...
KDT Syntax <ul><li>Keywords in use: </li></ul><ul><ul><li>Component Function (CF)  – perform a specific operation on a giv...
Test Cases Syntax KDT Test Case Step # Step Type Component Operation Value 1 SF - RunCommand “ C:Program FilesMyApplic.exe...
Test Cases Syntax KDT Test Case - Optimized Step # Step Type Component Operation Value 1 SF - RunCommand “ C:Program Files...
KDT Test Design Traditional  description KDT syntax Test applicable data KW type –  CF, BF or Sequence KW specific ID Step...
KDT -  Metrics <ul><li>Common RoI calculations are commonly defined as: </li></ul><ul><ul><li>RoI (Return on Investment)= ...
Time to Test –  where KDT covered 35%
Defect Detection
Breakeven Calculation <ul><li>KDT test case implementation takes 3 times more than traditional manual TC </li></ul><ul><li...
Rules of Thumb for KDT Imp. <ul><li>SW system under test should be: </li></ul><ul><ul><li>Has to be at least medium size d...
Conclusion <ul><li>Where thumb rules apply, KDT can be implemented with much better chance to succeed </li></ul><ul><li>Wh...
Personal Experience <ul><li>Life is now better for me as a functional tester </li></ul><ul><li>Defects are being unveiled ...
10x! [email_address] The future of KDT testers...
Upcoming SlideShare
Loading in …5
×

Qa Successful Automation Kdt As A Test Language

913 views

Published on

Qa Successful Automation Kdt As A Test Language

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
913
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • תרחיש ידני תרחיש KDT
  • תרחיש ידני תרחיש KDT
  • תרחיש ידני תרחיש KDT
  • Qa Successful Automation Kdt As A Test Language

    1. 1. Successful Automation KDT as a Test Language Aviram Shotten AP Program Manager QualiTest
    2. 2. Today’s Menu <ul><li>Brief introduction to KDT </li></ul><ul><li>Practical KDT – how KDT needs to be implemented – 3 years of experience tells… </li></ul><ul><li>KDT supporting tool – the AutomationPlanner ® </li></ul><ul><li>Measuring KDT – RoI and other metrics </li></ul><ul><li>Thumb rules for KDT implementation </li></ul><ul><li>Bad Humor </li></ul>
    3. 3. Introduction to KDT <ul><li>KDT is 3 rd generation automation approach (after record & replay and functional decomposition) </li></ul><ul><li>Previous approaches had very little success due to the following reasons: </li></ul><ul><ul><li>Automation experts required to be also good functional testers </li></ul></ul><ul><ul><li>Automation became a bottleneck on the way to increase coverage </li></ul></ul><ul><ul><li>Double (manual and automated) maintenance of test cases </li></ul></ul><ul><ul><li>Test management (test leader) didn’t necessarily knew how to manage the automation experts </li></ul></ul>
    4. 4. Introduction to KDT <ul><li>In order to better mitigate the drawbacks of automation that were mentioned before, KDT suggests the following: </li></ul><ul><ul><li>Automation experts will develop KDT infrastructure for the functional testers </li></ul></ul><ul><ul><li>Functional testers will use infrastructure for test cases implementation </li></ul></ul><ul><ul><li>Functional testers will mainly investigate failures and incidents rather than execute the test cases </li></ul></ul>
    5. 5. Introduction to KDT <ul><li>Developing infrastructure: </li></ul><ul><ul><li>Mapping AUT components </li></ul></ul><ul><ul><li>Imp. components operations </li></ul></ul><ul><ul><li>Imp. system operations </li></ul></ul><ul><ul><li>Imp. business functions </li></ul></ul><ul><li>Test engineering tasks: </li></ul><ul><ul><li>Test planning </li></ul></ul><ul><ul><li>Test implementation </li></ul></ul><ul><ul><li>Definition of commonly used sequences for test cases </li></ul></ul><ul><ul><li>Test execution </li></ul></ul><ul><ul><li>Debrief and report </li></ul></ul><ul><ul><li>Continuous improvement </li></ul></ul>Automation Discipline Testing Discipline
    6. 6. Requirements - KDT <ul><li>We, as functional testers, requires from a KDT based solution: </li></ul><ul><ul><li>Simplicity - test cases can be defined without development skills </li></ul></ul><ul><ul><li>Functionality – maintain the option to perform any functional UI operation </li></ul></ul><ul><ul><li>Readability – test cases and test reports will continue to be reviewed by other stakeholders </li></ul></ul><ul><ul><li>Maintenance – test cases will be maintained by functional tester and not by automation experts </li></ul></ul>
    7. 7. Generic KDT Suite <ul><li>Experience led us to build 4 components to support a generic KDT implementation process among test teams: </li></ul><ul><ul><li>KDT Engine – component that converts KDT test cases to executable format </li></ul></ul> – allowing easy test cases implementation for the functional testers – executes the test batches <ul><ul><li>- integrates the KDT tests to the </li></ul></ul><ul><ul><li>standard test process in HP Quality Center ® </li></ul></ul>
    8. 8. AP Architecture Logical Layer Infrastructure Layer SUT Testing Tool AP ENGINE VBS Or
    9. 9. A Test Case Traditional Test Case Step # Action Expected Result 1 Execute the “MyApplic” application Log in window appears 2 Enter user “JohnB” in username field and “123456” in the password field and press “OK” Error window appear 3 Enter user “JohnB” in username field and “QA2010” in the password field and press “OK” Login Successful, application’s main window appear
    10. 10. KDT Syntax <ul><li>Keywords in use: </li></ul><ul><ul><li>Component Function (CF) – perform a specific operation on a given UI component – click, set value, verify exist etc. </li></ul></ul><ul><ul><li>Service Function (SF) – scripts that enables assisting methods for the test implementation - wait X seconds, write to report etc. </li></ul></ul><ul><ul><li>Business Function (BF) – execute a certain functional operation, that is hard non effective to implement by component service function </li></ul></ul><ul><ul><li>Sequence (Seq) – a set of keywords (CF, SF, BF, or another Seq) that can be called from different test cases with different parameters. </li></ul></ul>
    11. 11. Test Cases Syntax KDT Test Case Step # Step Type Component Operation Value 1 SF - RunCommand “ C:Program FilesMyApplic.exe” 2 CF UserName (Edit) Set “ JohnB” 3 CF Password (Edit) Set “ 123456” 4 CF OK (Button) Click - 5 CF LoginErrorDialogue VerifyExist - 6 CF UserName (Edit) Set “ JohnB” 7 CF Password (Edit) Set “ QA2010” 8 CF OK (Button) Click - 9 CF WelcomeWindow VerifyExist -
    12. 12. Test Cases Syntax KDT Test Case - Optimized Step # Step Type Component Operation Value 1 SF - RunCommand “ C:Program FilesMyApplic.exe” 2 SEQ Login - “ JohnB”, “123456” 3 CF LoginErrorDialogue VerifyExist - 4 SEQ Login - “ JohnB”, “QA2010” 5 CF WelcomeWindow VerifyExist -
    13. 13. KDT Test Design Traditional description KDT syntax Test applicable data KW type – CF, BF or Sequence KW specific ID Steps parameters
    14. 14. KDT - Metrics <ul><li>Common RoI calculations are commonly defined as: </li></ul><ul><ul><li>RoI (Return on Investment)= BENEFIT/COST </li></ul></ul><ul><ul><ul><li>Automation Cost = Price Of HW + Price of SW + Development Cost + Maintenance Cost + Execution Cost </li></ul></ul></ul><ul><ul><ul><li>Manual Testing Cost = Development Cost + Maintenance Cost + Execution Cost </li></ul></ul></ul><ul><ul><li>RoI = manual testing cost - automation cost </li></ul></ul><ul><ul><li>automation cost </li></ul></ul><ul><li>We suggest otherwise… </li></ul>
    15. 15. Time to Test – where KDT covered 35%
    16. 16. Defect Detection
    17. 17. Breakeven Calculation <ul><li>KDT test case implementation takes 3 times more than traditional manual TC </li></ul><ul><li>This “extra effort” can be replaced with 3 manual executions of a given TC </li></ul><ul><li>In KDT era, test execution time is not a matter of constraint. A given TC can now be executed numerously with no effort invested </li></ul>
    18. 18. Rules of Thumb for KDT Imp. <ul><li>SW system under test should be: </li></ul><ul><ul><li>Has to be at least medium size dev. effort (12000 dev. Hours) </li></ul></ul><ul><ul><li>Has already began development and it is possible to execute test on </li></ul></ul><ul><ul><li>At least 50% of the TC can be done by a certain UI (the SUT doesn’t necessarily has UI, can be simulators, etc.) </li></ul></ul><ul><li>Test team organization should consider: </li></ul><ul><ul><li>KDT is simple yet requires good and talented functional testers </li></ul></ul><ul><ul><li>All beginnings are hard </li></ul></ul>
    19. 19. Conclusion <ul><li>Where thumb rules apply, KDT can be implemented with much better chance to succeed </li></ul><ul><li>Where traditional automation applies KDT can increase coverage, effectiveness and reduce cost of automation </li></ul><ul><li>We propose a risk free KDT imp. process using generic tools debugged in many projects of many kinds </li></ul><ul><li>KDT improves the “quality of life” of the functional testers - less tedious execution, more planning and analysis </li></ul>
    20. 20. Personal Experience <ul><li>Life is now better for me as a functional tester </li></ul><ul><li>Defects are being unveiled during the KDT test case implementation much earlier than in traditional TC definition </li></ul><ul><li>New options are now available for us i.e 24 hours scenario of functional capabilities </li></ul><ul><li>Assets of the KDT diffused to other teams such as SW integrators build tests, developer unit tests, algorithms test etc </li></ul>
    21. 21. 10x! [email_address] The future of KDT testers...

    ×