0
Building	
  the	
  “right”	
  
regression	
  suite	
  using	
  
Behavior	
  Driven	
  Tes5ng	
  
(BDT)	
  
Anand	
  Bagmar...
abagmar@gmail.com	
  	
  
Anand.Bagmar@thoughtworks.com	
  
	
  
	
  	
  	
  	
  	
  @BagmarAnand	
  
	
  
about.me/anand....
What	
  do	
  you	
  expect	
  from	
  this	
  
session?	
  
Case	
  Study	
  
•  MulHple,	
  long	
  
running	
  projects	
  
•  Legacy	
  
applicaHons	
  
•  Integrated	
  
•  Limited	
  /	
  flaky	
  /	
  long-­‐running	
  
AutomaHon	
  	
  
•  Long	
  regression	
  cycle	
  
•  Huge	
  cost	
 ...
•  BriRle	
  
•  LiRle	
  /	
  less	
  value	
  
•  No	
  visibility	
  into	
  what	
  is	
  tested	
  
•  Maintenance	
 ...
Biggest	
  problem	
  
•  Trust	
  deficit	
  on	
  the	
  team	
  
– In	
  exisHng	
  AutomaHon	
  
– In	
  manual	
  tesH...
My	
  Case	
  Study	
  
Concept	
  #	
  1	
  
	
  
The	
  Test	
  Pyramid	
  
AutomaHon	
  –	
  A	
  Safety	
  Net	
  
•  “Succeeding	
  with	
  Agile”	
  –	
  Mike	
  Cohn	
  
•  MarHn	
  Fowler	
  –	
  Test	
  Pyramid	
  
–  hRp://marHnfow...
Func%onal	
  Tests	
  (GUI)	
  
Web	
  Service	
  Tests	
  
View	
  Tests	
  
JavaScript	
  Tests	
  
Integra%on	
  tests	...
Test	
  Pyramid	
  reality	
  
Development	
  team	
  
Test	
  Pyramid	
   QA	
  team	
  
Test	
  Pyramid	
  
Func%onal	
  Tests	
  (GUI)	
  
Web	
  Service	
  Tests	
  
…	
  
…	
  	
  
Unit	
  Tests	
  
Manual	
  /	
  Exploratory	
...
Func%onal	
  Tests	
  (GUI)	
  
Web	
  Service	
  Tests	
  
…	
  
…	
  
Unit	
  Tests	
  
Manual	
  /	
  
Exploratory	
  
...
Concept	
  #	
  2	
  
	
  
BDD	
  &	
  BDT	
  
Behavior	
  Driven	
  Development	
  	
  
(BDD)	
  
RED	
  
GREEN	
  REFACTOR	
  
Behavior	
  Driven	
  TesHng	
  
(BDT)	
  
The	
  Big	
  Picture	
  
•  How	
  is	
  this	
  funcHonality	
  going	
  to	
  be	
  used	
  by	
  
the	
  end-­‐user?	
...
Concept	
  #	
  3	
  
	
  
Test	
  Specifica5on	
  Styles	
  
Test	
  specificaHon	
  styles	
  
•  ImperaHve	
  
	
  
•  DeclaraHve	
  
Search	
  	
  
Search	
  
Results	
  
Contact	
  
Informa%on	
  
An	
  example	
  
For	
  SomeAwesomeAirlines	
  website	
...
ImperaHve	
  style	
  
Given	
  I	
  am	
  a	
  guest	
  user	
  on	
  the	
  home	
  page	
  
And	
  I	
  choose	
  “roun...
…	
  
When	
  I	
  select	
  the	
  “first”	
  opHon	
  
Then	
  I	
  am	
  on	
  the	
  Contact	
  InformaHon	
  page	
  
...
Given	
  I	
  am	
  a	
  guest	
  user	
  
	
  
When	
  I	
  search	
  for	
  flight	
  opHons	
  for	
  a	
  “one-­‐way”	
...
References	
  
•  Ben	
  Mabey	
  -­‐	
  
hRp://benmabey.com/2008/05/19/
imperaHve-­‐vs-­‐declaraHve-­‐scenarios-­‐in-­‐us...
My	
  Case	
  Study	
  
•  Remove	
  the	
  ambiguity	
  
•  Same	
  visibility	
  for	
  all	
  
How	
  did	
  we	
  accomplish	
  this?	
  
•  IdenHfy	
  business	
  flows	
  based	
  on	
  personas,	
  
and	
  how	
  the	
  personas	
  use	
  the	
  business	
  ...
1st	
  candidate	
  
•  Automate!	
  
•  High	
  Value,	
  Low	
  
Cost	
  
2nd	
  candidate	
  
•  Automate!	
  
•  High	...
Who	
  is	
  doing	
  the	
  
acHon?	
  
What	
  is	
  the	
  business	
  
funcHonality?	
  
Depicts	
  a	
  user	
  flow	
...
What	
  next?	
  
•  IdenHficaHon	
  of	
  Tests	
  
•  Automate	
  it!	
  
Automate	
  tests	
  that	
  are	
  valuable,	
  not	
  
because	
  you	
  can	
  or	
  like	
  to	
  automate!	
  
Who	
  is	
  doing	
  the	
  
acHon?	
  
What	
  is	
  the	
  business	
  
funcHonality?	
  
Depicts	
  a	
  user	
  flow	
...
Delete
profile
1st	
  candidate	
  
•  Automate!	
  
•  High	
  Value,	
  Low	
  
Cost	
  
2nd	
  candidate	
  
•  Automate!	
  
•  High	...
Final	
  state	
  
Project	
  Management	
  tool	
   Mingle	
  
Test	
  case	
  repository	
  (manual	
  +	
  
automated)	...
How	
  did	
  this	
  technique	
  ever	
  
become	
  successful?	
  
Context	
  is	
  KING!	
  
Domain	
  
Product-­‐
under-­‐test	
  
Distributed	
  
Teams	
  
Skills	
  
Capability	
  
Tech...
BDT	
  will	
  ….	
  
•  NOT	
  work	
  in	
  isola5on	
  
Requires	
  more	
  dedicated	
  and	
  focused	
  
tesHng	
  (manual	
  /	
  exploratory)	
  
Requires	
  lower	
  layers	
  of	
  Test	
  Pyramid	
  to	
  be	
  
automaHon	
  and	
  implemented	
  well!	
  
Func%onal	
  Tests	
  (GUI)	
  
Web	
  Service	
  Tests	
  
…	
  
…	
  
Unit	
  Tests	
  
Manual	
  /	
  
Exploratory	
  
...
AddiHonally	
  …	
  
•  Builds	
  and	
  deployments	
  are	
  automated	
  
•  FuncHonal	
  Performance	
  and	
  Concurr...
BDT	
  Value	
  ProposiHon	
  
?	
  
Incrementally	
  
build	
  User	
  Flows	
  
Regress	
  required	
  
Product	
  Business	
  
Value	
  
Tests	
  in-­...
Executable	
  
Specs	
  
BDT	
  
Scenarios	
  
	
  	
  
Func%onal	
  Tests	
  (GUI)	
  
Web	
  Service	
  Tests	
  
…	
  
…	
  
Unit	
  Tests	
  
BDT	
  helps	
  iden5fy	
  the	
...
Tools	
  to	
  enable	
  BDT	
  
•  Whiteboards	
  
•  Mind	
  maps	
  
•  Flow	
  charts	
  
•  Collabora5on!!	
  
Automate	
  the	
  top	
  layer	
  
Automate	
  using	
  -­‐	
  	
  
	
  
•  BDD	
  Tools,	
  or,	
  
	
  
•  Directly	
  ...
abagmar@gmail.com	
  	
  
Anand.Bagmar@thoughtworks.com	
  
	
  
	
  	
  	
  	
  	
  @BagmarAnand	
  
	
  
about.me/anand....
Building the "right" regression suite using Behavior Driven Testing (BDT)
Building the "right" regression suite using Behavior Driven Testing (BDT)
Building the "right" regression suite using Behavior Driven Testing (BDT)
Building the "right" regression suite using Behavior Driven Testing (BDT)
Building the "right" regression suite using Behavior Driven Testing (BDT)
Building the "right" regression suite using Behavior Driven Testing (BDT)
Building the "right" regression suite using Behavior Driven Testing (BDT)
Building the "right" regression suite using Behavior Driven Testing (BDT)
Upcoming SlideShare
Loading in...5
×

Building the "right" regression suite using Behavior Driven Testing (BDT)

1,388

Published on

Presented in World Conference - Next Generation Testing, Bangalore on 11th July 2013

Published in: Technology, Business
1 Comment
3 Likes
Statistics
Notes
No Downloads
Views
Total Views
1,388
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
50
Comments
1
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "Building the "right" regression suite using Behavior Driven Testing (BDT)"

  1. 1. Building  the  “right”   regression  suite  using   Behavior  Driven  Tes5ng   (BDT)   Anand  Bagmar   So#ware  Quality  Evangelist   Principal  Consultant,      
  2. 2. abagmar@gmail.com     Anand.Bagmar@thoughtworks.com              @BagmarAnand     about.me/anand.bagmar   About  me  
  3. 3. What  do  you  expect  from  this   session?  
  4. 4. Case  Study   •  MulHple,  long   running  projects   •  Legacy   applicaHons   •  Integrated  
  5. 5. •  Limited  /  flaky  /  long-­‐running   AutomaHon     •  Long  regression  cycle   •  Huge  cost  of  fixing  defects   FuncHonal  TesHng  Challenges  
  6. 6. •  BriRle   •  LiRle  /  less  value   •  No  visibility  into  what  is  tested   •  Maintenance  nightmare   •  Outdated   •  Expensive   As  a  result  
  7. 7. Biggest  problem   •  Trust  deficit  on  the  team   – In  exisHng  AutomaHon   – In  manual  tesHng   •  Finger  poinHng  /  blame  game  
  8. 8. My  Case  Study  
  9. 9. Concept  #  1     The  Test  Pyramid  
  10. 10. AutomaHon  –  A  Safety  Net  
  11. 11. •  “Succeeding  with  Agile”  –  Mike  Cohn   •  MarHn  Fowler  –  Test  Pyramid   –  hRp://marHnfowler.com/bliki/TestPyramid.html   The  Test  Pyramid  
  12. 12. Func%onal  Tests  (GUI)   Web  Service  Tests   View  Tests   JavaScript  Tests   Integra%on  tests   Component  tests   Unit  Tests   Test  AutomaHon  Pyramid   Manual  /   Exploratory   TesHng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   •  Cost   •  Effort   •  Time  
  13. 13. Test  Pyramid  reality   Development  team   Test  Pyramid   QA  team   Test  Pyramid  
  14. 14. Func%onal  Tests  (GUI)   Web  Service  Tests   …   …     Unit  Tests   Manual  /  Exploratory  TesHng   Test     Ice-­‐cream  cone   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   Test  Pyramid  –  AnH  PaRern   Product     under  test  
  15. 15. Func%onal  Tests  (GUI)   Web  Service  Tests   …   …   Unit  Tests   Manual  /   Exploratory   TesHng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   IdenHfy  the  right  type  of  tests  that  sit   on  top  of  the  Test  Pyramid  
  16. 16. Concept  #  2     BDD  &  BDT  
  17. 17. Behavior  Driven  Development     (BDD)  
  18. 18. RED   GREEN  REFACTOR  
  19. 19. Behavior  Driven  TesHng   (BDT)  
  20. 20. The  Big  Picture   •  How  is  this  funcHonality  going  to  be  used  by   the  end-­‐user?     •  What  is  their  thought  process  going  to  be   when  using  this  product?     •  What  is  the  “core-­‐business-­‐value”  I  am   delivering  to  the  end-­‐user?  
  21. 21. Concept  #  3     Test  Specifica5on  Styles  
  22. 22. Test  specificaHon  styles   •  ImperaHve     •  DeclaraHve  
  23. 23. Search     Search   Results   Contact   Informa%on   An  example   For  SomeAwesomeAirlines  website  –     A  Guest  User  can  …  
  24. 24. ImperaHve  style   Given  I  am  a  guest  user  on  the  home  page   And  I  choose  “round”  trip  opHon   And  I  select  “Chicago”  from  the  origin  dropdown   And  I  select  “San  Francisco”  from  the  desHnaHon  drop  down   And  I  select  departure  date  as  “5  December  2013”   And  I  select  returning  date  as  “25  December  2013”     When  I  click  on  Search     Then  I  should  see  the  search  results  page     And  I  should  see  at  least  1  opHon  for  my  criteria     ...  
  25. 25. …   When  I  select  the  “first”  opHon   Then  I  am  on  the  Contact  InformaHon  page     When  I  enter  first  name  as  “foo”   And  I  enter  last  name  as  “bar”   …   …   And  I  click  the  “Select  and  Con%nue”  buRon     Then  I  should  be  on  the  next  page    
  26. 26. Given  I  am  a  guest  user     When  I  search  for  flight  opHons  for  a  “one-­‐way”  trip   for  “1”  “Adult”  from  “Chicago”  to  “San  Francisco”   And  I  select  the  “first”  flight   And  I  enter  “valid”  contact  details  for  “traveller1”     Then  I  am  able  to  Save  and  ConHnue   DeclaraHve  style  
  27. 27. References   •  Ben  Mabey  -­‐   hRp://benmabey.com/2008/05/19/ imperaHve-­‐vs-­‐declaraHve-­‐scenarios-­‐in-­‐user-­‐ stories.html   •  Alister  ScoR  -­‐   hRp://waHrmelon.com/2010/12/10/ cucumber-­‐imperaHve-­‐or-­‐declaraHve-­‐that-­‐is-­‐ the-­‐quesHon/  
  28. 28. My  Case  Study  
  29. 29. •  Remove  the  ambiguity   •  Same  visibility  for  all  
  30. 30. How  did  we  accomplish  this?  
  31. 31. •  IdenHfy  business  flows  based  on  personas,   and  how  the  personas  use  the  business   funcHonality,  or,   •  IdenHfy  business  flows,  and  how  different  user   personas  can  use  the  business  flows  
  32. 32. 1st  candidate   •  Automate!   •  High  Value,  Low   Cost   2nd  candidate   •  Automate!   •  High  Value,  High   Cost   Can  Automate?   •  Manual   Regression?   •  Low  Value,  Low   Cost   Do  Not  Automate   •  Manual   Regression!   •  Low  Value,  High   Cost   High   Value   High     Cost   Low   Value   Low   Cost  
  33. 33. Who  is  doing  the   acHon?   What  is  the  business   funcHonality?   Depicts  a  user  flow  /   user  journey   Given  auc%oneer  creates  an   aucHon   And  5  vehicles  are  added  to  the   aucHon             When  auc%oneer  starts  the  aucHon   And  auc%oneer  sets  a  starHng  bid   of  5000   And  buyer_1  bids   And  buyer_2  bids     Then  auc%oneer  Sells  the  vehicle   And  buyer_2  wins  the  vehicle  
  34. 34. What  next?   •  IdenHficaHon  of  Tests   •  Automate  it!  
  35. 35. Automate  tests  that  are  valuable,  not   because  you  can  or  like  to  automate!  
  36. 36. Who  is  doing  the   acHon?   What  is  the  business   funcHonality?   Depicts  a  user  flow  /   user  journey   Ac5ons  and   verifica5ons  happen   implicitly   Given  auc%oneer  creates  an   aucHon   And  5  vehicles  are  added  to  the   aucHon             When  auc%oneer  starts  the  aucHon   And  auc%oneer  sets  a  starHng  bid   of  5000   And  buyer_1  bids   And  buyer_2  bids     Then  auc%oneer  Sells  the  vehicle   And  buyer_2  wins  the  vehicle  
  37. 37. Delete profile
  38. 38. 1st  candidate   •  Automate!   •  High  Value,  Low   Cost   2nd  candidate   •  Automate!   •  High  Value,  High   Cost   Can  Automate?   •  Manual   Regression?   •  Low  Value,  Low   Cost   Do  Not  Automate   •  Manual   Regression!   •  Low  Value,  High   Cost   High   Value   High     Cost   Low   Value   Low   Cost  
  39. 39. Final  state   Project  Management  tool   Mingle   Test  case  repository  (manual  +   automated)   Cucumber  .feature  files  with   appropriate  tagging   Feature  files  /  Scenarios   33  /  65   #  of  Manual  Tests   0   Tests  running  on  CI  (Jenkins)   Since  the  1st  test  was  automated   Smoke  tests  execuHon  Hme   15  minutes   Regression  tests  execuHon  Hme   45  minutes  
  40. 40. How  did  this  technique  ever   become  successful?  
  41. 41. Context  is  KING!   Domain   Product-­‐ under-­‐test   Distributed   Teams   Skills   Capability   Tech  Stack   Process   Tools  
  42. 42. BDT  will  ….   •  NOT  work  in  isola5on  
  43. 43. Requires  more  dedicated  and  focused   tesHng  (manual  /  exploratory)  
  44. 44. Requires  lower  layers  of  Test  Pyramid  to  be   automaHon  and  implemented  well!  
  45. 45. Func%onal  Tests  (GUI)   Web  Service  Tests   …   …   Unit  Tests   Manual  /   Exploratory   TesHng   Product   under  test   Ideal  Test   Pyramid   Business-­‐ facing  Tests   Technology-­‐ facing  Tests   IdenHfy  the  right  type  of  tests  that  sit   on  top  of  the  Test  Pyramid  
  46. 46. AddiHonally  …   •  Builds  and  deployments  are  automated   •  FuncHonal  Performance  and  Concurrent   TesHng  automated  and  run  on  demand  
  47. 47. BDT  Value  ProposiHon  
  48. 48. ?   Incrementally   build  User  Flows   Regress  required   Product  Business   Value   Tests  in-­‐sync   Effec%ve  Tests       Remove   ambiguity   Understand   new   func%onality   Living   Documenta%on   Onboarding  
  49. 49. Executable   Specs   BDT   Scenarios      
  50. 50. Func%onal  Tests  (GUI)   Web  Service  Tests   …   …   Unit  Tests   BDT  helps  iden5fy  the  “right”  type  of   regression  tests!   Manual  /   Exploratory   TesHng   Product   under  test   Ideal  Test   Pyramid  
  51. 51. Tools  to  enable  BDT   •  Whiteboards   •  Mind  maps   •  Flow  charts   •  Collabora5on!!  
  52. 52. Automate  the  top  layer   Automate  using  -­‐       •  BDD  Tools,  or,     •  Directly  in  any  appropriate  programming   language  /  tools  
  53. 53. abagmar@gmail.com     Anand.Bagmar@thoughtworks.com              @BagmarAnand     about.me/anand.bagmar   Thank  you  
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×