Building the "right" regression suite using Behavior Driven Testing (BDT)

1,862 views
1,697 views

Published on

Presented in World Conference - Next Generation Testing, Bangalore on 11th July 2013

Published in: Technology, Business
1 Comment
4 Likes
Statistics
Notes
  • Uploaded a new version of the PDF based on my webinar for NY Selenium Meetup Group () and talk at StarEast 2014 ()

    Video of this talk is available at: http://youtu.be/fyCInGDp-Ow

    More information about BDT can be found from my blog: http://essenceoftesting.blogspot.com/search/label/bdt
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
1,862
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
58
Comments
1
Likes
4
Embeds 0
No embeds

No notes for slide

Building the "right" regression suite using Behavior Driven Testing (BDT)

  1. 1. Building  the  “right”   regression  suite  using   Behavior  Driven  Tes5ng   (BDT)   Anand  Bagmar   So#ware  Quality  Evangelist   Principal  Consultant,      
  2. 2. abagmar@gmail.com     Anand.Bagmar@thoughtworks.com              @BagmarAnand     about.me/anand.bagmar   About  me  
  3. 3. What  do  you  expect  from  this   session?  
  4. 4. Case  Study   •  MulHple,  long   running  projects   •  Legacy   applicaHons   •  Integrated  
  5. 5. •  Limited  /  flaky  /  long-­‐running   AutomaHon     •  Long  regression  cycle   •  Huge  cost  of  fixing  defects   FuncHonal  TesHng  Challenges  
  6. 6. •  BriRle   •  LiRle  /  less  value   •  No  visibility  into  what  is  tested   •  Maintenance  nightmare   •  Outdated   •  Expensive   As  a  result  
  7. 7. Biggest  problem   •  Trust  deficit  on  the  team   – In  exisHng  AutomaHon   – In  manual  tesHng   •  Finger  poinHng  /  blame  game  
  8. 8. My  Case  Study  
  9. 9. Concept  #  1     The  Test  Pyramid  
  10. 10. AutomaHon  –  A  Safety  Net  
  11. 11. •  “Succeeding  with  Agile”  –  Mike  Cohn   •  MarHn  Fowler  –  Test  Pyramid   –  hRp://marHnfowler.com/bliki/TestPyramid.html   The  Test  Pyramid  
  12. 12. Func%onal  Tests  (GUI)   Web  Service  Tests   View  Tests   JavaScript  Tests   Integra%on  tests   Component  tests   Unit  Tests   Test  AutomaHon  Pyramid   Manual  /   Exploratory   TesHng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   •  Cost   •  Effort   •  Time  
  13. 13. Test  Pyramid  reality   Development  team   Test  Pyramid   QA  team   Test  Pyramid  
  14. 14. Func%onal  Tests  (GUI)   Web  Service  Tests   …   …     Unit  Tests   Manual  /  Exploratory  TesHng   Test     Ice-­‐cream  cone   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   Test  Pyramid  –  AnH  PaRern   Product     under  test  
  15. 15. Func%onal  Tests  (GUI)   Web  Service  Tests   …   …   Unit  Tests   Manual  /   Exploratory   TesHng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   IdenHfy  the  right  type  of  tests  that  sit   on  top  of  the  Test  Pyramid  
  16. 16. Concept  #  2     BDD  &  BDT  
  17. 17. Behavior  Driven  Development     (BDD)  
  18. 18. RED   GREEN  REFACTOR  
  19. 19. Behavior  Driven  TesHng   (BDT)  
  20. 20. The  Big  Picture   •  How  is  this  funcHonality  going  to  be  used  by   the  end-­‐user?     •  What  is  their  thought  process  going  to  be   when  using  this  product?     •  What  is  the  “core-­‐business-­‐value”  I  am   delivering  to  the  end-­‐user?  
  21. 21. Concept  #  3     Test  Specifica5on  Styles  
  22. 22. Test  specificaHon  styles   •  ImperaHve     •  DeclaraHve  
  23. 23. Search     Search   Results   Contact   Informa%on   An  example   For  SomeAwesomeAirlines  website  –     A  Guest  User  can  …  
  24. 24. ImperaHve  style   Given  I  am  a  guest  user  on  the  home  page   And  I  choose  “round”  trip  opHon   And  I  select  “Chicago”  from  the  origin  dropdown   And  I  select  “San  Francisco”  from  the  desHnaHon  drop  down   And  I  select  departure  date  as  “5  December  2013”   And  I  select  returning  date  as  “25  December  2013”     When  I  click  on  Search     Then  I  should  see  the  search  results  page     And  I  should  see  at  least  1  opHon  for  my  criteria     ...  
  25. 25. …   When  I  select  the  “first”  opHon   Then  I  am  on  the  Contact  InformaHon  page     When  I  enter  first  name  as  “foo”   And  I  enter  last  name  as  “bar”   …   …   And  I  click  the  “Select  and  Con%nue”  buRon     Then  I  should  be  on  the  next  page    
  26. 26. Given  I  am  a  guest  user     When  I  search  for  flight  opHons  for  a  “one-­‐way”  trip   for  “1”  “Adult”  from  “Chicago”  to  “San  Francisco”   And  I  select  the  “first”  flight   And  I  enter  “valid”  contact  details  for  “traveller1”     Then  I  am  able  to  Save  and  ConHnue   DeclaraHve  style  
  27. 27. References   •  Ben  Mabey  -­‐   hRp://benmabey.com/2008/05/19/ imperaHve-­‐vs-­‐declaraHve-­‐scenarios-­‐in-­‐user-­‐ stories.html   •  Alister  ScoR  -­‐   hRp://waHrmelon.com/2010/12/10/ cucumber-­‐imperaHve-­‐or-­‐declaraHve-­‐that-­‐is-­‐ the-­‐quesHon/  
  28. 28. My  Case  Study  
  29. 29. •  Remove  the  ambiguity   •  Same  visibility  for  all  
  30. 30. How  did  we  accomplish  this?  
  31. 31. •  IdenHfy  business  flows  based  on  personas,   and  how  the  personas  use  the  business   funcHonality,  or,   •  IdenHfy  business  flows,  and  how  different  user   personas  can  use  the  business  flows  
  32. 32. 1st  candidate   •  Automate!   •  High  Value,  Low   Cost   2nd  candidate   •  Automate!   •  High  Value,  High   Cost   Can  Automate?   •  Manual   Regression?   •  Low  Value,  Low   Cost   Do  Not  Automate   •  Manual   Regression!   •  Low  Value,  High   Cost   High   Value   High     Cost   Low   Value   Low   Cost  
  33. 33. Who  is  doing  the   acHon?   What  is  the  business   funcHonality?   Depicts  a  user  flow  /   user  journey   Given  auc%oneer  creates  an   aucHon   And  5  vehicles  are  added  to  the   aucHon             When  auc%oneer  starts  the  aucHon   And  auc%oneer  sets  a  starHng  bid   of  5000   And  buyer_1  bids   And  buyer_2  bids     Then  auc%oneer  Sells  the  vehicle   And  buyer_2  wins  the  vehicle  
  34. 34. What  next?   •  IdenHficaHon  of  Tests   •  Automate  it!  
  35. 35. Automate  tests  that  are  valuable,  not   because  you  can  or  like  to  automate!  
  36. 36. Who  is  doing  the   acHon?   What  is  the  business   funcHonality?   Depicts  a  user  flow  /   user  journey   Ac5ons  and   verifica5ons  happen   implicitly   Given  auc%oneer  creates  an   aucHon   And  5  vehicles  are  added  to  the   aucHon             When  auc%oneer  starts  the  aucHon   And  auc%oneer  sets  a  starHng  bid   of  5000   And  buyer_1  bids   And  buyer_2  bids     Then  auc%oneer  Sells  the  vehicle   And  buyer_2  wins  the  vehicle  
  37. 37. Delete profile
  38. 38. 1st  candidate   •  Automate!   •  High  Value,  Low   Cost   2nd  candidate   •  Automate!   •  High  Value,  High   Cost   Can  Automate?   •  Manual   Regression?   •  Low  Value,  Low   Cost   Do  Not  Automate   •  Manual   Regression!   •  Low  Value,  High   Cost   High   Value   High     Cost   Low   Value   Low   Cost  
  39. 39. Final  state   Project  Management  tool   Mingle   Test  case  repository  (manual  +   automated)   Cucumber  .feature  files  with   appropriate  tagging   Feature  files  /  Scenarios   33  /  65   #  of  Manual  Tests   0   Tests  running  on  CI  (Jenkins)   Since  the  1st  test  was  automated   Smoke  tests  execuHon  Hme   15  minutes   Regression  tests  execuHon  Hme   45  minutes  
  40. 40. How  did  this  technique  ever   become  successful?  
  41. 41. Context  is  KING!   Domain   Product-­‐ under-­‐test   Distributed   Teams   Skills   Capability   Tech  Stack   Process   Tools  
  42. 42. BDT  will  ….   •  NOT  work  in  isola5on  
  43. 43. Requires  more  dedicated  and  focused   tesHng  (manual  /  exploratory)  
  44. 44. Requires  lower  layers  of  Test  Pyramid  to  be   automaHon  and  implemented  well!  
  45. 45. Func%onal  Tests  (GUI)   Web  Service  Tests   …   …   Unit  Tests   Manual  /   Exploratory   TesHng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   IdenHfy  the  right  type  of  tests  that  sit   on  top  of  the  Test  Pyramid  
  46. 46. AddiHonally  …   •  Builds  and  deployments  are  automated   •  FuncHonal  Performance  and  Concurrent   TesHng  automated  and  run  on  demand  
  47. 47. BDT  Value  ProposiHon  
  48. 48. ?   Incrementally   build  User  Flows   Regress  required   Product  Business   Value   Tests  in-­‐sync   Effec%ve  Tests       Remove   ambiguity   Understand   new   func%onality   Living   Documenta%on   Onboarding  
  49. 49. Executable   Specs   BDT   Scenarios      
  50. 50. Func%onal  Tests  (GUI)   Web  Service  Tests   …   …   Unit  Tests   BDT  helps  iden5fy  the  “right”  type  of   regression  tests!   Manual  /   Exploratory   TesHng   Product   under  test   Ideal  Test   Pyramid  
  51. 51. Tools  to  enable  BDT   •  Whiteboards   •  Mind  maps   •  Flow  charts   •  Collabora5on!!  
  52. 52. Automate  the  top  layer   Automate  using  -­‐       •  BDD  Tools,  or,     •  Directly  in  any  appropriate  programming   language  /  tools  
  53. 53. abagmar@gmail.com     Anand.Bagmar@thoughtworks.com              @BagmarAnand     about.me/anand.bagmar   Thank  you  

×