Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

[Thao Vo] Deadly Traps of Automation Testing

1,472 views

Published on

Test Automation significantly contributes advantages and benefits to software testing success. However, test automation projects, to some extent, have been not succeeded as stakeholders' expectation. This topic aims to suggest solutions for the following problems to prevent from automated testing mistakes.

Published in: Software
  • Login to see the comments

  • Be the first to like this

[Thao Vo] Deadly Traps of Automation Testing

  1. 1. Deadly Traps of Test Automation STC JULY 2014 THAO VO
  2. 2. A landscape 2 1. Automation idol 2. UI addiction 3. Over misusing on commercial tools 4. Poor collaboration 5. Lazy maintenance 6. Intensive ROI Source: http://www.lifehack.org/articles/productivity/how-to-manage- common-productivity-traps-for-improved-productivity.html
  3. 3. 3 Trap #1: Adore Automation Source: http://www.theguardian.com/media/shortcuts/2014/mar/16/could-robots-be-journalist-of-future
  4. 4. Trap #1: Adore Automation Symptoms • Depending on automation for all testing activities • Absence of code reviews • Absence of exploratory testing • Absence of UX testing • All tests are built by developers 4 Source: http://www.calgolftech.com/1-problem-with-buying-golf-clubs/
  5. 5. Trap #1: Adore Automation 5 Treatments Prevent comparison between manual and automation Use automation in conjunction with manual techniques Use automation to provide a baseline – a sanity check Source: http://cureforhives.org/chronic- idiopathic-urticaria-treatments
  6. 6. 6 Trap #2: Love UI so much
  7. 7. A Pyramid 7 GUI Tests Acceptance Tests Integration Tests Unit Tests
  8. 8. Trap #2: Love UI so much Symptoms • Limited technical skills • Cannot understand the application or system architecture • Huge proportion of tests are being run via the UI • No collaboration with developers • Difficult to setup test context • Test results are slow & fragile 8 Source: http://www.bostonsearchgroup.com/blog/ceo- search-break-through-medical-devices-for-wound-closure/
  9. 9. Trap #2: Love UI so much Treatments • Limit the investment in UI automation • Invest more at lowest possible level with clear test purposes • Collaborate with developers • Make sure automation gives a fast feedback 9
  10. 10. 10 Trap #3: Over misusing on commercial testing tools Source: http://www.fastcoexist.com/1679718/your-bad-eating-habits-are-contagious
  11. 11. Trap #3: Over misusing on commercial testing tools Symptoms 11 A commercial tool forms the basis a testing strategy Only certain teams or individuals can access the tool or run tests Developers and testers have not been consulted in tool selections Source: http://www.sfgate.com/news/article/Apple- exploring-cars-medical-devices-to-reignite-5239850.php
  12. 12. Trap #3: Over misusing on commercial testing tools Treatments • Use open source tools wherever possible • Use tools that can be extended or integrated into existing development tool chain • Use tools that match specified test intent • Use clear matrix tool evaluation • Keep educating!!! 12 Source: http://stugalactic.co.uk/student-shopping- medicine.aspx
  13. 13. 13 Trap #4: Too pride to collaborate when creating tests Source: http://www.jonathankettleborough.com/tag/collaboration-2/
  14. 14. Trap #4: Poor collaboration Symptoms • Automated testers are isolated from team: duplicated test scripts, too much tests but inefficiency • Poor automation design • Maintainability or compatibility issues • No definition of quality: unclear, not match the desired system quality 14 Source: http://thaineuro.org/category/pharma-and- healthcare/healthcare/
  15. 15. Trap #4: Poor collaboration Treatments • Collaborate to create good tests and prevent duplication • Ensure good technical practices and design • Collaborate to improve quality definition and verification • Limit investment in UI based automated tests 15 Source: http://medicine1.info/medicine/
  16. 16. 16 Trap #5: Too lazy maintenance Source: http://www.tastyhuman.com/the-illusion-of-8-hour-workday-are-you-hard-working-or-just-lazy/
  17. 17. Trap #5: Too lazy maintenance Symptoms • Test suite has not been run recently – state is unknown • Test suite requires manual intervention • Duplication within automation code • Small changes trigger a cascade of failures 17 Source: http://orangedocfamilymedicine.com/
  18. 18. Trap #5: Too lazy maintenance Treatments • Automated tests should be executed using a CI environment • Ensure tests are always executable – even if the system is not being actively developed • Ensure collaboration between developers and testers 18 Source: http://www.psmag.com/blogs/the-101/the- spectrum-of-color-response-take-your-medicine-51195/
  19. 19. 19 Trap #6: Cut costs through automation Source: http://insuranceknock.com/questions/155/saving-money-with-2-phrases
  20. 20. Trap #6: Cut costs through automation Symptoms • Using a predicted ROI to reduce testing budget • Hidden investments are not counted 20 Source: http://bloomintegrativehealth.com/news-updates/
  21. 21. Trap #6: Cut costs through automation Treatments • Clear objectives and reasons for automation • Cost-benefit analysis should include ongoing maintenance • Quality is the key 21 Source: https://today.duke.edu/2013/03/operationmedicinedrop
  22. 22. Summary 22
  23. 23. 23 Q&A
  24. 24. References • [1]. Testing Experience – published No. #21, #23 - http://www.testingexperience.com/ • [2]. 7 Deadly Sins of Agile Software Test Automation – Adrian Smith http://www.slideshare.net/adrianlsmith/7- deadly-sins-of-software-test-automation • [3]. Manual Test cannot be automated http://www.satisfice.com/blog/archives/58 • [4]. Testing Club Vietnam 2014 Trends in SW Testing – Paul Holland 24
  25. 25. Profile • Thao Vo – QA Architect at KMS Technology Inc. (http://www.kms-technology.com/); Board member of HCMC Software Testing Club (http://www.hcmc-stc.org/). • Email: thaovo@kms-technology.com or thaovo@hcmc-stc.org or vvthao@yahoo.com • LinkedIn: http://vn.linkedin.com/pub/thao- van-vo/24/31a/b35 25
  26. 26. © 2014 HCMC Software Testing Club THANK YOU

×