Building the "right" regression suite using Behavior Driven Testing (BDT)
Upcoming SlideShare
Loading in...5
×
 

Building the "right" regression suite using Behavior Driven Testing (BDT)

on

  • 1,171 views

Presented in World Conference - Next Generation Testing, Bangalore on 11th July 2013

Presented in World Conference - Next Generation Testing, Bangalore on 11th July 2013

Statistics

Views

Total Views
1,171
Views on SlideShare
1,171
Embed Views
0

Actions

Likes
2
Downloads
31
Comments
1

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Uploaded a new version of the PDF based on my webinar for NY Selenium Meetup Group () and talk at StarEast 2014 ()

    Video of this talk is available at: http://youtu.be/fyCInGDp-Ow

    More information about BDT can be found from my blog: http://essenceoftesting.blogspot.com/search/label/bdt
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Building the "right" regression suite using Behavior Driven Testing (BDT) Building the "right" regression suite using Behavior Driven Testing (BDT) Presentation Transcript

    • Building  the  “right”   regression  suite  using   Behavior  Driven  Tes5ng   (BDT)   Anand  Bagmar   So#ware  Quality  Evangelist   Principal  Consultant,      
    • abagmar@gmail.com     Anand.Bagmar@thoughtworks.com              @BagmarAnand     about.me/anand.bagmar   About  me  
    • What  do  you  expect  from  this   session?  
    • Case  Study   •  MulHple,  long   running  projects   •  Legacy   applicaHons   •  Integrated  
    • •  Limited  /  flaky  /  long-­‐running   AutomaHon     •  Long  regression  cycle   •  Huge  cost  of  fixing  defects   FuncHonal  TesHng  Challenges  
    • •  BriRle   •  LiRle  /  less  value   •  No  visibility  into  what  is  tested   •  Maintenance  nightmare   •  Outdated   •  Expensive   As  a  result  
    • Biggest  problem   •  Trust  deficit  on  the  team   – In  exisHng  AutomaHon   – In  manual  tesHng   •  Finger  poinHng  /  blame  game  
    • My  Case  Study  
    • Concept  #  1     The  Test  Pyramid  
    • AutomaHon  –  A  Safety  Net  
    • •  “Succeeding  with  Agile”  –  Mike  Cohn   •  MarHn  Fowler  –  Test  Pyramid   –  hRp://marHnfowler.com/bliki/TestPyramid.html   The  Test  Pyramid  
    • Func%onal  Tests  (GUI)   Web  Service  Tests   View  Tests   JavaScript  Tests   Integra%on  tests   Component  tests   Unit  Tests   Test  AutomaHon  Pyramid   Manual  /   Exploratory   TesHng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   •  Cost   •  Effort   •  Time  
    • Test  Pyramid  reality   Development  team   Test  Pyramid   QA  team   Test  Pyramid  
    • Func%onal  Tests  (GUI)   Web  Service  Tests   …   …     Unit  Tests   Manual  /  Exploratory  TesHng   Test     Ice-­‐cream  cone   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   Test  Pyramid  –  AnH  PaRern   Product     under  test  
    • Func%onal  Tests  (GUI)   Web  Service  Tests   …   …   Unit  Tests   Manual  /   Exploratory   TesHng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   IdenHfy  the  right  type  of  tests  that  sit   on  top  of  the  Test  Pyramid  
    • Concept  #  2     BDD  &  BDT  
    • Behavior  Driven  Development     (BDD)  
    • RED   GREEN  REFACTOR  
    • Behavior  Driven  TesHng   (BDT)  
    • The  Big  Picture   •  How  is  this  funcHonality  going  to  be  used  by   the  end-­‐user?     •  What  is  their  thought  process  going  to  be   when  using  this  product?     •  What  is  the  “core-­‐business-­‐value”  I  am   delivering  to  the  end-­‐user?  
    • Concept  #  3     Test  Specifica5on  Styles  
    • Test  specificaHon  styles   •  ImperaHve     •  DeclaraHve  
    • Search     Search   Results   Contact   Informa%on   An  example   For  SomeAwesomeAirlines  website  –     A  Guest  User  can  …  
    • ImperaHve  style   Given  I  am  a  guest  user  on  the  home  page   And  I  choose  “round”  trip  opHon   And  I  select  “Chicago”  from  the  origin  dropdown   And  I  select  “San  Francisco”  from  the  desHnaHon  drop  down   And  I  select  departure  date  as  “5  December  2013”   And  I  select  returning  date  as  “25  December  2013”     When  I  click  on  Search     Then  I  should  see  the  search  results  page     And  I  should  see  at  least  1  opHon  for  my  criteria     ...  
    • …   When  I  select  the  “first”  opHon   Then  I  am  on  the  Contact  InformaHon  page     When  I  enter  first  name  as  “foo”   And  I  enter  last  name  as  “bar”   …   …   And  I  click  the  “Select  and  Con%nue”  buRon     Then  I  should  be  on  the  next  page    
    • Given  I  am  a  guest  user     When  I  search  for  flight  opHons  for  a  “one-­‐way”  trip   for  “1”  “Adult”  from  “Chicago”  to  “San  Francisco”   And  I  select  the  “first”  flight   And  I  enter  “valid”  contact  details  for  “traveller1”     Then  I  am  able  to  Save  and  ConHnue   DeclaraHve  style  
    • References   •  Ben  Mabey  -­‐   hRp://benmabey.com/2008/05/19/ imperaHve-­‐vs-­‐declaraHve-­‐scenarios-­‐in-­‐user-­‐ stories.html   •  Alister  ScoR  -­‐   hRp://waHrmelon.com/2010/12/10/ cucumber-­‐imperaHve-­‐or-­‐declaraHve-­‐that-­‐is-­‐ the-­‐quesHon/  
    • My  Case  Study  
    • •  Remove  the  ambiguity   •  Same  visibility  for  all  
    • How  did  we  accomplish  this?  
    • •  IdenHfy  business  flows  based  on  personas,   and  how  the  personas  use  the  business   funcHonality,  or,   •  IdenHfy  business  flows,  and  how  different  user   personas  can  use  the  business  flows  
    • 1st  candidate   •  Automate!   •  High  Value,  Low   Cost   2nd  candidate   •  Automate!   •  High  Value,  High   Cost   Can  Automate?   •  Manual   Regression?   •  Low  Value,  Low   Cost   Do  Not  Automate   •  Manual   Regression!   •  Low  Value,  High   Cost   High   Value   High     Cost   Low   Value   Low   Cost  
    • Who  is  doing  the   acHon?   What  is  the  business   funcHonality?   Depicts  a  user  flow  /   user  journey   Given  auc%oneer  creates  an   aucHon   And  5  vehicles  are  added  to  the   aucHon             When  auc%oneer  starts  the  aucHon   And  auc%oneer  sets  a  starHng  bid   of  5000   And  buyer_1  bids   And  buyer_2  bids     Then  auc%oneer  Sells  the  vehicle   And  buyer_2  wins  the  vehicle  
    • What  next?   •  IdenHficaHon  of  Tests   •  Automate  it!  
    • Automate  tests  that  are  valuable,  not   because  you  can  or  like  to  automate!  
    • Who  is  doing  the   acHon?   What  is  the  business   funcHonality?   Depicts  a  user  flow  /   user  journey   Ac5ons  and   verifica5ons  happen   implicitly   Given  auc%oneer  creates  an   aucHon   And  5  vehicles  are  added  to  the   aucHon             When  auc%oneer  starts  the  aucHon   And  auc%oneer  sets  a  starHng  bid   of  5000   And  buyer_1  bids   And  buyer_2  bids     Then  auc%oneer  Sells  the  vehicle   And  buyer_2  wins  the  vehicle  
    • Delete profile
    • 1st  candidate   •  Automate!   •  High  Value,  Low   Cost   2nd  candidate   •  Automate!   •  High  Value,  High   Cost   Can  Automate?   •  Manual   Regression?   •  Low  Value,  Low   Cost   Do  Not  Automate   •  Manual   Regression!   •  Low  Value,  High   Cost   High   Value   High     Cost   Low   Value   Low   Cost  
    • Final  state   Project  Management  tool   Mingle   Test  case  repository  (manual  +   automated)   Cucumber  .feature  files  with   appropriate  tagging   Feature  files  /  Scenarios   33  /  65   #  of  Manual  Tests   0   Tests  running  on  CI  (Jenkins)   Since  the  1st  test  was  automated   Smoke  tests  execuHon  Hme   15  minutes   Regression  tests  execuHon  Hme   45  minutes  
    • How  did  this  technique  ever   become  successful?  
    • Context  is  KING!   Domain   Product-­‐ under-­‐test   Distributed   Teams   Skills   Capability   Tech  Stack   Process   Tools  
    • BDT  will  ….   •  NOT  work  in  isola5on  
    • Requires  more  dedicated  and  focused   tesHng  (manual  /  exploratory)  
    • Requires  lower  layers  of  Test  Pyramid  to  be   automaHon  and  implemented  well!  
    • Func%onal  Tests  (GUI)   Web  Service  Tests   …   …   Unit  Tests   Manual  /   Exploratory   TesHng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   IdenHfy  the  right  type  of  tests  that  sit   on  top  of  the  Test  Pyramid  
    • AddiHonally  …   •  Builds  and  deployments  are  automated   •  FuncHonal  Performance  and  Concurrent   TesHng  automated  and  run  on  demand  
    • BDT  Value  ProposiHon  
    • ?   Incrementally   build  User  Flows   Regress  required   Product  Business   Value   Tests  in-­‐sync   Effec%ve  Tests       Remove   ambiguity   Understand   new   func%onality   Living   Documenta%on   Onboarding  
    • Executable   Specs   BDT   Scenarios      
    • Func%onal  Tests  (GUI)   Web  Service  Tests   …   …   Unit  Tests   BDT  helps  iden5fy  the  “right”  type  of   regression  tests!   Manual  /   Exploratory   TesHng   Product   under  test   Ideal  Test   Pyramid  
    • Tools  to  enable  BDT   •  Whiteboards   •  Mind  maps   •  Flow  charts   •  Collabora5on!!  
    • Automate  the  top  layer   Automate  using  -­‐       •  BDD  Tools,  or,     •  Directly  in  any  appropriate  programming   language  /  tools  
    • abagmar@gmail.com     Anand.Bagmar@thoughtworks.com              @BagmarAnand     about.me/anand.bagmar   Thank  you