Building the "right" regression suite using Behavior Driven Testing (BDT)
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Building the "right" regression suite using Behavior Driven Testing (BDT)

on

  • 1,295 views

Presented in World Conference - Next Generation Testing, Bangalore on 11th July 2013

Presented in World Conference - Next Generation Testing, Bangalore on 11th July 2013

Statistics

Views

Total Views
1,295
Views on SlideShare
1,295
Embed Views
0

Actions

Likes
2
Downloads
34
Comments
1

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Uploaded a new version of the PDF based on my webinar for NY Selenium Meetup Group () and talk at StarEast 2014 ()

    Video of this talk is available at: http://youtu.be/fyCInGDp-Ow

    More information about BDT can be found from my blog: http://essenceoftesting.blogspot.com/search/label/bdt
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Building the "right" regression suite using Behavior Driven Testing (BDT) Presentation Transcript

  • 1. Building  the  “right”   regression  suite  using   Behavior  Driven  Tes5ng   (BDT)   Anand  Bagmar   So#ware  Quality  Evangelist   Principal  Consultant,      
  • 2. abagmar@gmail.com     Anand.Bagmar@thoughtworks.com              @BagmarAnand     about.me/anand.bagmar   About  me  
  • 3. What  do  you  expect  from  this   session?  
  • 4. Case  Study   •  MulHple,  long   running  projects   •  Legacy   applicaHons   •  Integrated  
  • 5. •  Limited  /  flaky  /  long-­‐running   AutomaHon     •  Long  regression  cycle   •  Huge  cost  of  fixing  defects   FuncHonal  TesHng  Challenges  
  • 6. •  BriRle   •  LiRle  /  less  value   •  No  visibility  into  what  is  tested   •  Maintenance  nightmare   •  Outdated   •  Expensive   As  a  result  
  • 7. Biggest  problem   •  Trust  deficit  on  the  team   – In  exisHng  AutomaHon   – In  manual  tesHng   •  Finger  poinHng  /  blame  game  
  • 8. My  Case  Study  
  • 9. Concept  #  1     The  Test  Pyramid  
  • 10. AutomaHon  –  A  Safety  Net  
  • 11. •  “Succeeding  with  Agile”  –  Mike  Cohn   •  MarHn  Fowler  –  Test  Pyramid   –  hRp://marHnfowler.com/bliki/TestPyramid.html   The  Test  Pyramid  
  • 12. Func%onal  Tests  (GUI)   Web  Service  Tests   View  Tests   JavaScript  Tests   Integra%on  tests   Component  tests   Unit  Tests   Test  AutomaHon  Pyramid   Manual  /   Exploratory   TesHng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   •  Cost   •  Effort   •  Time  
  • 13. Test  Pyramid  reality   Development  team   Test  Pyramid   QA  team   Test  Pyramid  
  • 14. Func%onal  Tests  (GUI)   Web  Service  Tests   …   …     Unit  Tests   Manual  /  Exploratory  TesHng   Test     Ice-­‐cream  cone   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   Test  Pyramid  –  AnH  PaRern   Product     under  test  
  • 15. Func%onal  Tests  (GUI)   Web  Service  Tests   …   …   Unit  Tests   Manual  /   Exploratory   TesHng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   IdenHfy  the  right  type  of  tests  that  sit   on  top  of  the  Test  Pyramid  
  • 16. Concept  #  2     BDD  &  BDT  
  • 17. Behavior  Driven  Development     (BDD)  
  • 18. RED   GREEN  REFACTOR  
  • 19. Behavior  Driven  TesHng   (BDT)  
  • 20. The  Big  Picture   •  How  is  this  funcHonality  going  to  be  used  by   the  end-­‐user?     •  What  is  their  thought  process  going  to  be   when  using  this  product?     •  What  is  the  “core-­‐business-­‐value”  I  am   delivering  to  the  end-­‐user?  
  • 21. Concept  #  3     Test  Specifica5on  Styles  
  • 22. Test  specificaHon  styles   •  ImperaHve     •  DeclaraHve  
  • 23. Search     Search   Results   Contact   Informa%on   An  example   For  SomeAwesomeAirlines  website  –     A  Guest  User  can  …  
  • 24. ImperaHve  style   Given  I  am  a  guest  user  on  the  home  page   And  I  choose  “round”  trip  opHon   And  I  select  “Chicago”  from  the  origin  dropdown   And  I  select  “San  Francisco”  from  the  desHnaHon  drop  down   And  I  select  departure  date  as  “5  December  2013”   And  I  select  returning  date  as  “25  December  2013”     When  I  click  on  Search     Then  I  should  see  the  search  results  page     And  I  should  see  at  least  1  opHon  for  my  criteria     ...  
  • 25. …   When  I  select  the  “first”  opHon   Then  I  am  on  the  Contact  InformaHon  page     When  I  enter  first  name  as  “foo”   And  I  enter  last  name  as  “bar”   …   …   And  I  click  the  “Select  and  Con%nue”  buRon     Then  I  should  be  on  the  next  page    
  • 26. Given  I  am  a  guest  user     When  I  search  for  flight  opHons  for  a  “one-­‐way”  trip   for  “1”  “Adult”  from  “Chicago”  to  “San  Francisco”   And  I  select  the  “first”  flight   And  I  enter  “valid”  contact  details  for  “traveller1”     Then  I  am  able  to  Save  and  ConHnue   DeclaraHve  style  
  • 27. References   •  Ben  Mabey  -­‐   hRp://benmabey.com/2008/05/19/ imperaHve-­‐vs-­‐declaraHve-­‐scenarios-­‐in-­‐user-­‐ stories.html   •  Alister  ScoR  -­‐   hRp://waHrmelon.com/2010/12/10/ cucumber-­‐imperaHve-­‐or-­‐declaraHve-­‐that-­‐is-­‐ the-­‐quesHon/  
  • 28. My  Case  Study  
  • 29. •  Remove  the  ambiguity   •  Same  visibility  for  all  
  • 30. How  did  we  accomplish  this?  
  • 31. •  IdenHfy  business  flows  based  on  personas,   and  how  the  personas  use  the  business   funcHonality,  or,   •  IdenHfy  business  flows,  and  how  different  user   personas  can  use  the  business  flows  
  • 32. 1st  candidate   •  Automate!   •  High  Value,  Low   Cost   2nd  candidate   •  Automate!   •  High  Value,  High   Cost   Can  Automate?   •  Manual   Regression?   •  Low  Value,  Low   Cost   Do  Not  Automate   •  Manual   Regression!   •  Low  Value,  High   Cost   High   Value   High     Cost   Low   Value   Low   Cost  
  • 33. Who  is  doing  the   acHon?   What  is  the  business   funcHonality?   Depicts  a  user  flow  /   user  journey   Given  auc%oneer  creates  an   aucHon   And  5  vehicles  are  added  to  the   aucHon             When  auc%oneer  starts  the  aucHon   And  auc%oneer  sets  a  starHng  bid   of  5000   And  buyer_1  bids   And  buyer_2  bids     Then  auc%oneer  Sells  the  vehicle   And  buyer_2  wins  the  vehicle  
  • 34. What  next?   •  IdenHficaHon  of  Tests   •  Automate  it!  
  • 35. Automate  tests  that  are  valuable,  not   because  you  can  or  like  to  automate!  
  • 36. Who  is  doing  the   acHon?   What  is  the  business   funcHonality?   Depicts  a  user  flow  /   user  journey   Ac5ons  and   verifica5ons  happen   implicitly   Given  auc%oneer  creates  an   aucHon   And  5  vehicles  are  added  to  the   aucHon             When  auc%oneer  starts  the  aucHon   And  auc%oneer  sets  a  starHng  bid   of  5000   And  buyer_1  bids   And  buyer_2  bids     Then  auc%oneer  Sells  the  vehicle   And  buyer_2  wins  the  vehicle  
  • 37. Delete profile
  • 38. 1st  candidate   •  Automate!   •  High  Value,  Low   Cost   2nd  candidate   •  Automate!   •  High  Value,  High   Cost   Can  Automate?   •  Manual   Regression?   •  Low  Value,  Low   Cost   Do  Not  Automate   •  Manual   Regression!   •  Low  Value,  High   Cost   High   Value   High     Cost   Low   Value   Low   Cost  
  • 39. Final  state   Project  Management  tool   Mingle   Test  case  repository  (manual  +   automated)   Cucumber  .feature  files  with   appropriate  tagging   Feature  files  /  Scenarios   33  /  65   #  of  Manual  Tests   0   Tests  running  on  CI  (Jenkins)   Since  the  1st  test  was  automated   Smoke  tests  execuHon  Hme   15  minutes   Regression  tests  execuHon  Hme   45  minutes  
  • 40. How  did  this  technique  ever   become  successful?  
  • 41. Context  is  KING!   Domain   Product-­‐ under-­‐test   Distributed   Teams   Skills   Capability   Tech  Stack   Process   Tools  
  • 42. BDT  will  ….   •  NOT  work  in  isola5on  
  • 43. Requires  more  dedicated  and  focused   tesHng  (manual  /  exploratory)  
  • 44. Requires  lower  layers  of  Test  Pyramid  to  be   automaHon  and  implemented  well!  
  • 45. Func%onal  Tests  (GUI)   Web  Service  Tests   …   …   Unit  Tests   Manual  /   Exploratory   TesHng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   IdenHfy  the  right  type  of  tests  that  sit   on  top  of  the  Test  Pyramid  
  • 46. AddiHonally  …   •  Builds  and  deployments  are  automated   •  FuncHonal  Performance  and  Concurrent   TesHng  automated  and  run  on  demand  
  • 47. BDT  Value  ProposiHon  
  • 48. ?   Incrementally   build  User  Flows   Regress  required   Product  Business   Value   Tests  in-­‐sync   Effec%ve  Tests       Remove   ambiguity   Understand   new   func%onality   Living   Documenta%on   Onboarding  
  • 49. Executable   Specs   BDT   Scenarios      
  • 50. Func%onal  Tests  (GUI)   Web  Service  Tests   …   …   Unit  Tests   BDT  helps  iden5fy  the  “right”  type  of   regression  tests!   Manual  /   Exploratory   TesHng   Product   under  test   Ideal  Test   Pyramid  
  • 51. Tools  to  enable  BDT   •  Whiteboards   •  Mind  maps   •  Flow  charts   •  Collabora5on!!  
  • 52. Automate  the  top  layer   Automate  using  -­‐       •  BDD  Tools,  or,     •  Directly  in  any  appropriate  programming   language  /  tools  
  • 53. abagmar@gmail.com     Anand.Bagmar@thoughtworks.com              @BagmarAnand     about.me/anand.bagmar   Thank  you