Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Use Automation to Assist -Not Replace- Manual Testing

Learn how to best integrate automated testing into your existing workflows and get more return with SmartBear's own Jeff Martin.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all
  • Login to see the comments

Use Automation to Assist -Not Replace- Manual Testing

  1. 1. Use Automation to Assist —Not Replace— Manual Testing
  2. 2. 22 Ideal Automation Return E f f o r t Return The Shark Fin Manual Automated Time
  3. 3. 33 Reality of Most Automation E f f o r t Return? Time Mountains of No Return Manual Automated
  4. 4. 44 Why the Gap? 1. Failure to align different goals across the team 2. Automated testing viewed as a replacement for existing processes 3. Automated testing approached like development 4. Not enough focus on high-return automation
  5. 5. 55 Goals - The Automation Dream • Less Time • No faulty tests / false defects • Less people • More like manufacturing The “Tester”The “Boss” • Less repetition • No faulty tests / false defects • More time to Test • More like craftsmanship
  6. 6. 66 Goals - The Agreement 1. Less Time & Repetition 2. No faulty tests / false defects Focus on these two goals
  7. 7. 77 Replacing Existing Processes What Testers and the Business roles are familiar with: Workflows Business Processes Test Cases User Stories So this is what they automate
  8. 8. 88 Example Test 1 Step 1 Step 2 Step 3 Step 4 Step 5 Step 1 Step 2 Step 3 Step 4 Step 5
  9. 9. 99 Misleading Early Results The obvious workflows get automated first Less Time & Repetition No (ok, less) faulty tests / false defects Success breeds repetition What gets overlooked is efficiency Break down workflows into component tasks
  10. 10. 1010 Task Efficiency over 5 years Credit: XKCD https://xkcd.com/1205/
  11. 11. 1111 Automate like a Dev? Ok, so all Tasks in a process are not created equal One popular alternative: Think like a developer and break processes down 2 Developer approaches with the same problem when applied to testing Object Oriented Functional Stable operations Stable “things” New “things” New operations
  12. 12. 1212 Why Dev-focused approaches fail Developers are used to building “the whole thing” They can’t choose not to code something! Development and Tests are based on the same Requirements and therefore do not capture errors in design Additionally, Developers can only interact and check for expected results Interact Check Click Values Enter text Presence Drag Layouts / Coordinates
  13. 13. 1313 An aside: Checking vs. Testing Automation when done right is more valuable than “Checking” Many non-checking functions are better done by computers Checks are based on design, Tests are based on the much messier Usage. Fully Automated Regression “Testing” is a screening tool that only Checks
  14. 14. 1414 Where are we with the Gap? 1. Failure to align different goals across the team Find the areas of agreement, focus on those 2. Automated testing viewed as a replacement for existing processes Break workflows into Tasks 3. Automated testing approached like development Recognize that development and testing, even automated testing, are different activities 4. Not enough focus on high-return automation
  15. 15. 1515 Task Automation 1. Testing is hard work, automation should free up time for that hard work 2. Processes are made of tasks 3. Tasks do not always equal tests 4. Tasks have different returns on automation investment
  16. 16. 1616 Good Task Automation Candidates1. Environment Setup / Teardown 2. Data Entry a) Form Filling b) Varying data inputs in a repetitive process 3. Exposing backend data (APIs, DB tables, etc….) 4. Repetitive / boring tasks that are prone to inattention errors 5. Tasks with high reuse value across many workflows 6. Tests with timing or screen responsiveness as a criteria for success 7. Many non-functional test types, such as Performance testing 8. Smoke Test Suites – key user workflows as a screening tool 9. Result Capture
  17. 17. 1717 Good Task Automation Candidates Ask what your Team hates Doing!
  18. 18. 1818 Example Test 1 Step 1 Step 2 Step 3 Step 4 Step 5 Automation 1 Automation 2 Manual 1 Manual 2
  19. 19. 1919 Where are we with the Gap? 1. Failure to align different goals across the team Find the areas of agreement, focus on those 2. Automated testing viewed as a replacement for existing processes Break workflows into Tasks 3. Automated testing approached like development Recognize that development and testing, even automated testing, are different activities 4. Not enough focus on high-return automation Deconstruct Tests in to Tasks Rate your tasks on Return
  20. 20. 2020 The Hard Part – Combining Execution Combining Automation and Manual execution of Tasks is difficult Many tools do exist to help combine execution and results: 1. Automation tools that pause and wait for manual testing either in a form or via a button 2. Test Management or Manual Testing tools that run scripts on demand Caution: Switching between execution methods has a cost that should be factored into Task selection.
  21. 21. Questions?

×