Building a Test Automation Strategy for Success


Published on

Choosing an appropriate tool and building the right framework are typically thought of as the main challenges in implementing successful test automation. However, long term success requires that other key questions must be answered including:
- What are our objectives?
- How should we be organized?
- Will our processes need to change?
- Will our test environment support test automation?
- What skills will we need?
- How and when should we implement?

In this workshop, Lee will discuss how to assess your test automation readiness and build a strategy for long term success. You will interactively walk through the assessment process and build a test automation strategy based on input from the group. Attend this workshop and you will take away a blue print and best practices for building an effective test automation strategy in your organization.
• Understand the key aspects of a successful test automation function
• Learn how to assess your test automation readiness
• Develop a test automation strategy specific to your organization

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Building a Test Automation Strategy for Success

  1. 1. © Utopia Solutions Building a Strategy for Test Automation Success Lee Barnes, CTO Utopia Solutions
  2. 2. © Utopia Solutions Agenda •What is the end goal? •Why do we need a strategy? •What do we need to consider? •What should a strategy look like? Test Automation Strategy Overview Interactive Case Study
  3. 3. © Utopia Solutions SECTION 1 What is Our End Goal?
  4. 4. © Utopia Solutions4 Test Automation Success Criteria • Issue detection and recovery • Accurate verification • Unattended execution Reliable • Minimum sensitivity to application and test case changes • Test case definition separate from automation code Maintainable • Test coverage expanded efficiently • Automated test cases created by non- automation resources Scalable
  5. 5. © Utopia Solutions5 Test Automation Approach Test Automation Framework Automated Test Cases Test Results Test Definition Test Execution Non-technical / Test Analysts & Subject Matter Experts Technical / Automation Engineers
  6. 6. © Utopia Solutions SECTION 2 Why Do We Need a Strategy?
  7. 7. © Utopia Solutions8 Why Test Automation Fails Organizational • Lack of support • Unrealistic expectations • Disjointed org structure • Missing skill sets Process • Immature testing process • Segregated processes • Lack of test environment / data control Technical • Poor tool support for AUT • Under or over design automation approach
  8. 8. © Utopia Solutions9 Are We Ready?
  9. 9. © Utopia Solutions SECTION 3 What Do We Need to Know First?
  10. 10. © Utopia Solutions11 Test Automation Readiness Assessment Org • Structure • Support • Goals and Expectations • Resources and Skills Process • Test Maturity • Test Assets • Process Integration • Test Environment / Data Tech • AUT Technology • AUT / Tool Integration Assessment Topics
  11. 11. © Utopia Solutions Where Do We Get the Information? Assessment Input Stakeholders Existing Assets AUTs
  12. 12. © Utopia Solutions Ideal Organizational Attributes Structure • Centralized testing function • Testing is a peer of development • Automation roles are defined Goals and Expectations • Automation is an investment • 100% automation is not likely • Automation requires specialized skills • Automation will free up resources to work on high- value tasks Support • Executive management has visibility to automation • Automation has a dedicated budget Resources and Skills • Resources are dedicated to automation • Resources are skilled in automation as a process AND technology
  13. 13. © Utopia Solutions Ideal Process Attributes Test Process • Test objectives are clear and measurable • Test cases are defined and documented • Expected results are predictable SDLC Integration • AUT in a frequent release state with visibility to future roadmap • Visibility to system changes as early as possible in release cycle Test Environment • Test environment dedicated to automation • Test data controlled
  14. 14. © Utopia Solutions Ideal Technology Attributes AUT Technology • Multiple AUTs built on common technology and/or development framework • Consistent use of UI objects • Consistent navigation • Consistent presentation of user feedback (warnings, errors, etc.) Tool / AUT Integration • All UI objects supported by tool • Any alternative interfaces (web services, API, DB, etc.) supported by tool • All target end user environments supported by tool
  15. 15. © Utopia Solutions SECTION 4 Common Assessment Findings
  16. 16. © Utopia Solutions17 Organizational Distributed Testing / Automation Efforts App App App App • Uneven automation resource demand • Inconsistent approach • Duplicated efforts • Restricted skill growth and knowledge sharing Leads to…
  17. 17. © Utopia Solutions18 Organizational AUT A AUT B AUT C AUT D Centralized Test Automation Team Common Resources, Processes & Technology • Flatten demand curve • Accelerate skills development • Leverage reusable knowledge and technology Benefits Centralize Automation Activities
  18. 18. © Utopia Solutions19 Process Analyze Design Build Test Deploy No ability to identify impact of app changes on automated test suite Unexpected / Unpredictable Maintenance
  19. 19. © Utopia Solutions20 Process Analyze Design Build Test Deploy Capture potential impacts as early as possible to enable proactive maintenance Capture App Changes Analyze Impact Maintain Execute Automation Activities Application SDLC
  20. 20. © Utopia Solutions21 Environment Shared Test Environment Uncontrolled Test Data Integration Functional Performance Integration Test • Schedule conflicts • Unreliable execution • Unnecessary “hardening” of the test suite Leads to… Automation
  21. 21. © Utopia Solutions22 Environment Shared Test Environment Controlled Test Data Integration Functional Performance Integration Test Dedicated Environment for Automation Test Data Controls Dedicated Environment Controlled Test Data Automation • Restore to known state • Partition data related to specific test activities • Create required data each execution
  22. 22. © Utopia Solutions SECTION 5 What Does a Strategy Look Like?
  23. 23. © Utopia Solutions24 Mobile Test Strategy Topics Test Automation Strategy Process & Organization Environment TechnicalResources Scope & Roadmap
  24. 24. © Utopia Solutions25 Automation Strategy – Process and Organization • Political and financial support • Realistic expectations • Centralized automation function Key Organizational Components • Test objectives and coverage goals • Integration with SDLC → proactive maintenance Key Process Components
  25. 25. © Utopia Solutions26 Automation Strategy – Environment • Dedicated environment for execution • Controlled test data • Controlled code migration Key Environment Components
  26. 26. © Utopia Solutions27 Automation Strategy – Resources • Roles • Skills • Time Key People Components • AUT test environment(s) • Test suite development • Test suite execution Key Hardware Components • Test tool • Supporting utilities Key Software Components
  27. 27. © Utopia Solutions28 Automation Strategy – Scope • Test data setup • AUT / environment preparation • Complex / tedious results verification Test Support Tasks • Applications built on common technology • Applications with frequent releases & friendly UI’s Application Selection • Frequently tested • High test case to UI ratio • Predictable results • Easy to automate Test Case Selection
  28. 28. © Utopia Solutions29 Automation Strategy – Roadmap • Begin with required organizational and process changes • Overlap with base framework and minimal coverage • Enhance and add test coverage over small iterations Create Implementation Timeline Phase nPhase 1 Continuous Process & Organizational Improvements Base Framework Initial Coverage AUT 1 Framework Enhancements Expand Coverage Initial Coverage Phase 2 AUT 2 Expand Coverage Expand Coverage Initial Coverage Phase 3 AUT 3
  29. 29. © Utopia Solutions SECTION 6 Case Study
  30. 30. © Utopia Solutions31 Key Assessment Findings Organization • All enterprise testing activities currently performed centrally • Management believes automation is a tool implementation Process • Manual tests are well structured, but not maintained consistently • Rely on SME’s to execute tests Environment • Manual regression test performed in a isolated environment • Test data refreshed in all environments once per quarter Technology • All AUTs have browser based UI • Developed on disparate technologies • Test tool supports all AUT components Resources • Test organization consists of non- technical SME’s • System performance (mobile) • Device performance
  31. 31. © Utopia Solutions32 Key Summary Points • DO NOT automate without a clear vision and plan • Understand your readiness from these perspectives • Organization • Process & Environment • Technical • Resources • Implement in small phases
  32. 32. © Utopia Solutions33 Questions… …. and Answers! Direct future questions to: Lee Barnes Founder and CTO Utopia Solutions, Inc. Email: Twitter: LinkedIn: Blog: