© Affiliated Computer Services, Inc. (ACS) 2007, 2008


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Webex technician to explain the Q % A feature, he will then turn the call over to you.
  • © Affiliated Computer Services, Inc. (ACS) 2007, 2008

    1. 1. Facilitating the Move from Predominantly Manual Testing to an Automated Test Environment Tim Leach, Systems Consulting Manager
    2. 2. Planning the move to Automation <ul><li>First Step; Understand Resources: </li></ul><ul><ul><li>Personnel </li></ul></ul><ul><ul><ul><li>Skills with coding background </li></ul></ul></ul><ul><ul><ul><li>Availability </li></ul></ul></ul><ul><ul><li>The right tools </li></ul></ul><ul><ul><li>Enough equipment </li></ul></ul><ul><li>Next choose the Project: </li></ul><ul><ul><li>Project that has the best fit </li></ul></ul><ul><ul><li>Ready to move and step up </li></ul></ul><ul><ul><li>Project with trained resources </li></ul></ul><ul><li>Last Step; Plan and Execute: </li></ul><ul><ul><li>Align resources </li></ul></ul><ul><ul><li>Train everyone from management to developers </li></ul></ul><ul><ul><li>Plan each step </li></ul></ul><ul><ul><li>Last step execute the plan </li></ul></ul>
    3. 3. Understand the Resources <ul><li>Personnel </li></ul><ul><ul><li>Select resources that will make the move successful </li></ul></ul><ul><ul><li>Make sure they are available, then assign them to the project </li></ul></ul><ul><li>Tools </li></ul><ul><ul><li>Test tools from HP, Segue, IBM or other tool possibilities </li></ul></ul><ul><ul><ul><li>http://www.softwareqatest.com/qatweb1.html </li></ul></ul></ul><ul><li>Equipment </li></ul><ul><ul><li>PCs for development and execution </li></ul></ul><ul><ul><li>Align enough test environments to accomplish tasks </li></ul></ul><ul><ul><li>Gathering the correct tools and developing the infrastructure </li></ul></ul>
    4. 4. Select Project <ul><ul><li>Project that has the best fit </li></ul></ul><ul><ul><ul><li>Important step to find a project that is willing take on additional work </li></ul></ul></ul><ul><ul><ul><li>Be aware that they might be under the microscope </li></ul></ul></ul><ul><ul><ul><li>Be able to add resources both equipment and personnel </li></ul></ul></ul><ul><ul><li>Ready to move and step up </li></ul></ul><ul><ul><ul><li>Can start early in the develop cycle </li></ul></ul></ul><ul><ul><ul><li>Add development resources to develop tests </li></ul></ul></ul><ul><ul><ul><li>A motivated team </li></ul></ul></ul><ul><ul><li>Project with trained resources </li></ul></ul><ul><ul><ul><li>Have resources on the team that understands test methods an test development </li></ul></ul></ul>
    5. 5. <ul><li>Pull together resources: </li></ul><ul><ul><li>Align equipment </li></ul></ul><ul><ul><li>Align personnel </li></ul></ul><ul><ul><li>Prep Project team </li></ul></ul><ul><li>Build Plan: </li></ul><ul><ul><li>Develop plan step by step </li></ul></ul><ul><ul><li>Analyze and document current testing state </li></ul></ul><ul><ul><li>Review plan and get approval </li></ul></ul><ul><ul><li>Set realistic expectations </li></ul></ul><ul><li>Document ROI and Sell: </li></ul><ul><ul><li>Gather ROI information and report </li></ul></ul><ul><ul><li>Sell the Plan </li></ul></ul><ul><li>Execute Plan: </li></ul><ul><ul><li>Be ready to change </li></ul></ul>Plan Development Execution Business Case Development Concept Analysis Concept Creation
    6. 6. <ul><li>Execute: </li></ul><ul><ul><li>Execute plan </li></ul></ul><ul><ul><li>Stay on plan </li></ul></ul><ul><ul><li>Resist scope creep </li></ul></ul><ul><li>Monitor: </li></ul><ul><ul><li>Monitor progress of development and test coverage </li></ul></ul><ul><ul><li>Status of meeting timeframe objective </li></ul></ul><ul><li>Adjust: </li></ul><ul><ul><li>If problems occur modify plan and report change </li></ul></ul><ul><ul><li>Reset expectations early </li></ul></ul><ul><li>Report: </li></ul><ul><ul><li>Report status using standard reports </li></ul></ul><ul><ul><li>This enables you to show progress </li></ul></ul>Plan Execution Report Adjust Monitor Execute
    7. 7. Some keys to success <ul><ul><li>Make sure that the correct expectations are set with team and management. </li></ul></ul><ul><ul><li>Make sure that the plan to automation is not in the critical path of the project. </li></ul></ul><ul><ul><li>Make sure that the effort to automate does not negatively affect the projects goals. </li></ul></ul><ul><ul><li>Make sure everyone is informed on the status of the process at each step. </li></ul></ul><ul><ul><li>Make sure the plan is not over ambitious. Best to plan several mini goals. </li></ul></ul><ul><ul><li>Make sure the automation architecture is reusable. This will win over management and prove a good ROI. </li></ul></ul>
    8. 8. Questions
    9. 9. Test Automation – Definition & Quotes <ul><ul><li>Test automation is the use of software to control the execution of tests, the comparison of actual outcomes to predicted outcomes, the setting up of test preconditions, and other test control and test reporting functions. Commonly, test automation involves automating a manual process already in place that uses a formalized testing process. - Wikipedia </li></ul></ul><ul><ul><li>In today’s environment of plummeting cycle times, test automation becomes an increasingly critical and strategic necessity. Assuming the level of testing in the past was sufficient (which is rarely the case), how do we possibly keep up with this new explosive pace of web-enabled deployment while retaining satisfactory test coverage and reducing risk? The answer is either more people for manual testing, or a greater level of test automation. After all, a reduction in project cycle times generally correlates to a reduction of time for test. - Carl J. Nagle </li></ul></ul>
    10. 10. Test Automation – Definition & Quotes Continued <ul><ul><li>Test automation raises our hopes yet often frustrates and disappoints us. Although automation promises to deliver us from a tough situation, implementing automated tests can create as many problems as it solves . - Bret Pettichord </li></ul></ul><ul><ul><li>Over the past several years, tools that help programmers quickly create applications with graphical user interfaces have dramatically improved programmer productivity. This has increased the pressure on testers, who are often perceived as bottlenecks to the delivery of software products. Testers are being asked to test more and more code in less and less time. They need to dramatically improve their own productivity. Test automation is one way to do this. - Bret Pettichord </li></ul></ul>
    11. 11. Automation Myths <ul><li>Test automation is record and playback. </li></ul><ul><li>Test automation does not need specialized skills. </li></ul><ul><li>Tools do all the work. </li></ul><ul><li>Purchasing a tool gains you automation. </li></ul><ul><li>Once you have automation you can remove manual testing. </li></ul><ul><li>Automation will find all new bugs in test cycles. </li></ul><ul><li>Automation will reduce testing resources. </li></ul><ul><li>Once you have automation developed, you’re done! </li></ul>
    12. 12. Automation Problems <ul><li>Test automation is seen as a spare time project </li></ul><ul><li>Lack of clear goals and objectives </li></ul><ul><li>Lack of expertise </li></ul><ul><li>High resource turnover </li></ul><ul><li>To much focus on tools and technology </li></ul><ul><li>Lack of long term planning and goals in automation </li></ul><ul><li>Lack of understanding </li></ul><ul><li>Unrealistic expectations </li></ul><ul><li>Over simplification of development </li></ul><ul><li>Limited resources </li></ul>
    13. 13. Automation Planning <ul><li>Test Automation development </li></ul><ul><ul><li>Before you start, take account of your current situation </li></ul></ul><ul><ul><ul><li>Develop a complete detailed metric baseline of your current state, great for comparison later </li></ul></ul></ul><ul><ul><ul><li>How long does your testing effort take? </li></ul></ul></ul><ul><ul><ul><li>How many resources are you using? </li></ul></ul></ul><ul><ul><li>Resource planning and utilization </li></ul></ul><ul><ul><ul><li>Automation will increase the need for resources, we need to manage them closely, schedule usage, assign by specialty and skill levels </li></ul></ul></ul>
    14. 14. Automation Planning <ul><ul><li>Then develop a strategy and goal (long and short term) </li></ul></ul><ul><ul><ul><li>Plan should complement the afore mentioned metrics - track your progress, report status </li></ul></ul></ul><ul><ul><ul><li>Define Requirements for testing </li></ul></ul></ul><ul><ul><ul><li>Prove the automation concept </li></ul></ul></ul><ul><ul><ul><li>Champion product testability </li></ul></ul></ul><ul><ul><ul><li>Design for sustainability </li></ul></ul></ul><ul><ul><ul><li>Plan for deployment and movement through the SDLC </li></ul></ul></ul><ul><ul><ul><li>Face the Challenges of Success </li></ul></ul></ul><ul><ul><ul><li>Improve the Testing Process </li></ul></ul></ul>
    15. 15. Automation Planning <ul><ul><li>Then develop an architecture and design for test development </li></ul></ul><ul><ul><ul><li>Plan for reliable and easy maintenance </li></ul></ul></ul><ul><ul><ul><li>Develop reusable scripts for usage on other projects </li></ul></ul></ul><ul><ul><ul><li>Plan validation goals and objective </li></ul></ul></ul><ul><ul><ul><li>Develop a standard detailed reporting status </li></ul></ul></ul><ul><ul><ul><li>Metrics will help show success and make it easy to compare incremental success </li></ul></ul></ul><ul><ul><li>Test Automation is a development process </li></ul></ul><ul><ul><ul><li>Need to have change control, release management, and testing </li></ul></ul></ul>
    16. 16. Automation Planning <ul><ul><li>Test data management </li></ul></ul><ul><ul><ul><li>Need to have plan for collecting, developing and resetting test data </li></ul></ul></ul>
    17. 17. Report Success <ul><li>Compare the difference to overall test coverage </li></ul><ul><li>Report differences to test response time </li></ul><ul><li>Report consistent and reproducible results </li></ul><ul><li>Quicker turnaround time for test cycles </li></ul><ul><li>Add reports on retesting, regression, integration and bug verification </li></ul>
    18. 18. Testing Tools <ul><li>Benefits of a testing development and management tool </li></ul><ul><ul><li>Test execution management and results reporting </li></ul></ul><ul><ul><ul><li>Can run tests and collect results in tool </li></ul></ul></ul><ul><ul><ul><li>Report of execution status using management reporting functionality </li></ul></ul></ul><ul><ul><ul><li>Manage traceability and test coverage </li></ul></ul></ul><ul><ul><ul><li>Manage development of scripts </li></ul></ul></ul><ul><ul><li>Single point of testing artifacts </li></ul></ul><ul><ul><li>Help in developing tests </li></ul></ul>
    19. 19. Tool Planning <ul><li>What tools are needed? </li></ul><ul><li>What is the automation framework that best fits the project? </li></ul><ul><li>What skills do we have? </li></ul><ul><li>What is the time frame to finish? </li></ul><ul><li>What architecture will we use? Keyword scripting, test development scripting, and manual to automation? </li></ul><ul><li>What approach will we use: focused requirements base, scenario testing, or regression testing? </li></ul>
    20. 20. Simple Place to Start <ul><li>Keyword Scripts </li></ul><ul><ul><li>Gives you both manual and automation scripting </li></ul></ul><ul><ul><li>Focus resources on areas of expertise </li></ul></ul><ul><ul><li>Easy maintenance and reuse </li></ul></ul><ul><ul><li>Works with or without testing tools </li></ul></ul><ul><li>Regression testing </li></ul><ul><ul><li>Start automation from manual tests scripts that are proven </li></ul></ul><ul><li>Defect Validation </li></ul><ul><ul><li>Great use of documented tests and a method of verifying progress </li></ul></ul>
    21. 21. Keyword Scripting <ul><li>A method that can speed up automation and maintain both manual and automation code in one format. </li></ul><ul><li>Use a keyword driver to tie the UI with the actions - this is the one area that actual code is needed and maintained. </li></ul>Run – Tests 1-3 Application under test Driver Script - QTP 3 KeyWord Script - Excel 2 KeyWord Script - Excel 1 KeyWord Script - Excel
    22. 22. Keyword Script explained <ul><li>Keyword Script Layout </li></ul><ul><ul><li>The file layout for this can be done using various formats: Excel files, database files, and if you’re using Quality Center you can develop to the Test Plan script tables. </li></ul></ul><ul><ul><li>The approach I have taken is to use Excel - most people have it and are familiar with using it so training is less (and it can be used outside of any tool) </li></ul></ul><ul><ul><li>To run the test you’re basically executing the driver which loads the keyword script and executes the steps. </li></ul></ul><ul><ul><li>The driver ties the application via the Applications UI to the Keyword actions. </li></ul></ul>
    23. 23. Keyword Script explained <ul><ul><li>Functional_Area used to group all the actions and Test_Steps to a page or screen </li></ul></ul><ul><ul><li>These groups will be selectable from a dropdown list of available actions </li></ul></ul><ul><ul><li>The dropdown can be controlled to keep only the item in the list to be used or you can open it up for any keywords </li></ul></ul><ul><ul><li>Skip_Step allows Pseudo script development. (script development that is done before the application is working in the application or keyword driver) </li></ul></ul>
    24. 24. Keyword Driver Code <ul><li>Automation Layout </li></ul><ul><li>The code snipit below explains how the process works for tying the UI of an application to test actions in the keyword script. </li></ul><ul><li>Driver scripting </li></ul><ul><ul><li>This is the heart of the keyword functionality. The driver is the tie to the application and the scripts. </li></ul></ul><ul><ul><li>The driver basically uses a Case statement to handle the action in the script. </li></ul></ul><ul><ul><ul><li>Note the first Case below, “Case ‘Enter From Date’“, that ties to the action in the Excel script “Enter From Date”. This ties the UI in the application to the action. </li></ul></ul></ul>
    25. 25. Keyword Driver and Script Compare
    26. 26. Questions