# L F M F 	                            (and others                                        who shall r                     ...
Fail	  Pa;ern	  #0:	  	  “No	  6me	  to	  automate”	                                         Copyright	  ©	  2012	  Elisab...
Velocity!Copyright	  ©	  2012	  Elisabeth	  Hendrickson,	  Quality	  Tree	  So7ware,	  Inc.	  
Fail	  Pa;ern	  #1:	  	  Automa6on	  for	  the	  sake	                                                   Copyright	  ©	  2...
Image	  of	  prinFng	  office	  from	  hJp://www.shorpy.com/node/12652	  Copyright	  ©	  2012	  Elisabeth	  Hendrickson,	  Q...
Fail	  Pa;ern	  #2:	  	  All	  automa6on	  is	  end-­‐to-­‐                                              Copyright	  ©	  2...
Copyright	  ©	  2012	  Elisabeth	  Hendrickson,	  Quality	  Tree	  So7ware,	  Inc.	  Flooded	  basement	  image	  courtesy...
Fail	  Pa;ern	  #3:	  	    Dispropor6onate	                                           Copyright	  ©	  2012	  Elisabeth	  H...
Copyright	  ©	  2012	  Elisabeth	  Hendrickson,	  Quality	  Tree	  So7ware,	  Inc.	  Image	  courtesy	  of	  cdrussorusso	...
Fail	  Pa;ern	  #4:	  	           Fragile	  Tests	                                                Copyright	  ©	  2012	  E...
Hey, the                                                                              build is red.!                      ...
Fail	  Pa;ern	  #5:	  	  Inter-­‐dependent	  Tests	                                           Copyright	  ©	  2012	  Elisa...
Run #1!                           Run #3!                                                  Run #2!       Run #4!Copyright	...
Fail	  Pa;ern	  #6:	  	         Useless	  Tests	                                              Copyright	  ©	  2012	  Elisa...
Yes, I have                 actually…   seen this in                            Copyright	  ©	  2012	  Elisabeth	  Hendric...
Fail	  Pa;ern	  #7:	  	             Bad	  Tests	                                                  Copyright	  ©	  2012	  E...
Test Login  Input Text first_name fred                                     Copyright	  ©	  2012	  Elisabeth	  Hendrickson,...
Fail	  Pa;ern	  #8:	  	              Bad	  Code	                                                   Copyright	  ©	  2012	  ...
Copyright	  ©	  2012	  Elisabeth	  Hendrickson,	  Quality	  Tree	  So7ware,	  Inc.	  
Fail	  Pa;ern	  #9:	  	       Only	  specialists	                                           Copyright	  ©	  2012	  Elisabe...
Fail	  Pa;ern	  #10:	  	      Forcing	  devout	  non-­‐                                                     Copyright	  ©	...
Fail	  Pa;ern	  #11:	  	  Tests	  That	  Don’t	  Get	  Run	                                               Copyright	  ©	  ...
Unrun !                                                           tests!                                                  ...
Fail	  Pa;ern	  #12:	  	  Failing	  Tests	  Ignored	                                     Copyright	  ©	  2012	  Elisabeth	...
Copyright	  ©	  2012	  Elisabeth	  Hendrickson,	  Quality	  Tree	  So7ware,	  Inc.	  Image	  of	  the	  1906	  San	  Franc...
#LFMF	                                           Copyright	  ©	  2012	  Elisabeth	  Hendrickson,	  Quality	  Tree	  So7war...
Teema Frisk!                          @hackyourtv!                         Juha Rantanen!                         Jyrki Pu...
Upcoming SlideShare
Loading in...5
×

#LFMF: Tales of Test Automation Gone Wrong

2,897

Published on

This is a "Best Of" list of my (and others) failures over the years in attempting to adopt test automation. In it, you'll find 13 categories of What Not to Do. Presented at Turku Agile Day 2012 (#tad012).

Published in: Technology, Education

#LFMF: Tales of Test Automation Gone Wrong

  1. 1. # L F M F   (and others who shall r anonymous emain even if they deserve it)! don’t Learn  from  My  Fail:    Tales  of  Test  Automa6on  Gone  Wrong     (Plus  a  Li*le  Bit  about  How  to  Do  It  Right)   Elisabeth  Hendrickson   Quality  Tree  So7ware,  Inc.   @testobsessed  
  2. 2. Fail  Pa;ern  #0:    “No  6me  to  automate”   Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.   No  automaFon  means   rapid  growth  in  the  manual  tesFng  burden.   Or  untested  so7ware.  
  3. 3. Velocity!Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.  
  4. 4. Fail  Pa;ern  #1:    Automa6on  for  the  sake   Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.   of  automa6on   If  you  really  want  a  mess,   use  a  count  of  automated   tests  as  a  measure  of   progress.  
  5. 5. Image  of  prinFng  office  from  hJp://www.shorpy.com/node/12652  Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.  
  6. 6. Fail  Pa;ern  #2:    All  automa6on  is  end-­‐to-­‐ Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.   end  through  the  GUI   End-­‐to-­‐end  tests  are  slow  and  fragile.  Some  are  good.   More  are  not  beJer.  
  7. 7. Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.  Flooded  basement  image  courtesy  of  carlpenergy  Showerhead  image  courtesy  of  deusx  Images  from  Flickr  licensed  through  CreaFve  Commons  AJribuFon  
  8. 8. Fail  Pa;ern  #3:     Dispropor6onate   Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.  Amount  of  Time  Spent   Fixing  Broken  Tests   Just  when  you  get  one   working  again,  another   breaks.  
  9. 9. Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.  Image  courtesy  of  cdrussorusso  From  hJp://www.flickr.com/photos/23516192@N08/2631474033/    Licensed  Under  CreaFve  Commons  AJribuFon  
  10. 10. Fail  Pa;ern  #4:     Fragile  Tests   Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.   They  pass,  they  fail,  but   there  seems  to  be  no  correlaFon  between  the  test   results  and  the  actual   so7ware  behavior.  
  11. 11. Hey, the build is red.! year.! full moon this Tuesday with a Yeah, it’s the 2ndCopyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.  
  12. 12. Fail  Pa;ern  #5:    Inter-­‐dependent  Tests   Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.   When  one  fails,     they  all  fail  together.  
  13. 13. Run #1! Run #3! Run #2! Run #4!Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.  
  14. 14. Fail  Pa;ern  #6:     Useless  Tests   Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.  The  tests  can  never  fail,   so  they  yield  no   informaFon.  
  15. 15. Yes, I have actually… seen this in Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.   tests.!assert True But rare ly. More… common: tes rets that ify the s oftwa ver does w hat the softwar e does.!
  16. 16. Fail  Pa;ern  #7:     Bad  Tests   Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.   Obfuscated  intent,  massive  duplicaFon,  and  Fght  coupling  between  the  interface  and  the   tests  makes  for  an   unmaintainable  mess.  
  17. 17. Test Login Input Text first_name fred Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.   Input Text last_name flintstone Input Text user_name fflint Input Text password p@ssw0rd Click Submit Click Logout Click Login Input Text username fflint WAT?!?! Input Text password badpassword Click Submit Input Text ...
  18. 18. Fail  Pa;ern  #8:     Bad  Code   Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.  Smelly  test  automaFon  code   that  violates  SOLID   principles  is  as  much  of  a   problem  as  smelly   producFon  code.  
  19. 19. Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.  
  20. 20. Fail  Pa;ern  #9:     Only  specialists   Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.   automate   They  might  be  good  at  automaFon,  but  they’re  fighFng  an  uphill  baJle.  
  21. 21. Fail  Pa;ern  #10:     Forcing  devout  non-­‐ Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.   coders  to  write   automa6on  If  they  lack  either  interest  or   apFtude  to  automate,  the   results  will  be  bad.  
  22. 22. Fail  Pa;ern  #11:    Tests  That  Don’t  Get  Run   Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.   Someone  writes  them,  no   one  runs  them.  So  when   they  suffer  bitrot,  no  one   can  hear  them  scream.  
  23. 23. Unrun ! tests! Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.   become ! creaky,! dusty,! sad.!Photo  by  crabchick  hJp://www.flickr.com/photos/crabchick/2627370039/  Licensed  under  CreaFve  Commons  AJribuFon  
  24. 24. Fail  Pa;ern  #12:    Failing  Tests  Ignored   Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.   They  run.     They  fail.     No  one  cares.  
  25. 25. Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.  Image  of  the  1906  San  Francisco  Earthquake  a7ermath  from  hJp://www.shorpy.com/node/9866  
  26. 26. #LFMF   Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.  Automate important expectations!Treat test code with the same level ofcare as production code!Run all the automated tests in the CIbuild (no manual kickoff)!Fix failing tests immediately!Foster a “whole team tests” culture!
  27. 27. Teema Frisk! @hackyourtv! Juha Rantanen! Jyrki Pulliainen! Jarkko Kailanto! Many thanks to …!Copyright  ©  2012  Elisabeth  Hendrickson,  Quality  Tree  So7ware,  Inc.  
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×