Your SlideShare is downloading. ×

Executable Specifications Agile Palooza

2,467

Published on

Published in: Technology, Business
2 Comments
3 Likes
Statistics
Notes
No Downloads
Views
Total Views
2,467
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
85
Comments
2
Likes
3
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Executable  Specifica/ons Documents  for  So7ware  that   Chris  Sterling Partner  at  Sterling  Barton Cer/fied  Scrum  Trainer Agile  Consultant   Thursday, January 14, 2010 1
  • 2. Executable  Specifica/ons  -­‐ Topics  to  Discuss • What  are  Executable  Specifica/ons? • Why  Create  Them? • Acceptance  Tests • When  to  Use • An  Approach  with  Example • Available  Tools 2 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 2
  • 3. The  second-­‐most  detailed  specifica/on  of  the  customer’s  request   WHAT  ARE  EXECUTABLE  SPECIFICATIONS? 3 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 3
  • 4. Executable  Specifica/ons • Unambiguous  defini/on  of  desired  so7ware  behavior • Executable  documenta/on • Repeatable  and  specific  tests • Regression  tests  to  validate  exis/ng  so7ware  behavior • The  second-­‐most  detailed  specifica/on  of  the   customer’s  request   4 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 4
  • 5. Building  the  “right”  so7ware WHY  CREATE  EXECUTABLE  SPECIFICATIONS? 5 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 5
  • 6. Increasing  Cost  of  Release  Stabiliza/on   Periods 500,000 375,000 250,000 125,000 Release 1 Release 2 Release 3 0 Release 4 Release 5 Release 6 Cost of Fixing Defects Cost for Feature Dev 6 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 6
  • 7. Costly  Regression  Test  Phases 7 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 7
  • 8. Managing  So7ware  Debt  –  an  Overview 8 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 8
  • 9. Effect  of  Project  Constraints  on  Quality 9 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 9
  • 10. Effect  of  Project  Constraints  on  Quality 9 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 9
  • 11. How  does  the  Team  know  they  have  implemented  desired  behavior  successfully? ACCEPTANCE  TESTS 10 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 10
  • 12. Agile  tes/ng  happens  at  two  levels   • Acceptance  Tests  tell  us  whether  the   system  does  what  the  customer   expects  (“building  the  right  code”) • Programmer  Tests  define  whether  the   system  does  what  the  developers   expect  (“building  the  code  right”) 11 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 11
  • 13. Agile  tes/ng  happens  at  two  levels   • Acceptance  Tests  tell  us  whether  the   system  does  what  the  customer   expects  (“building  the  right  code”) • Programmer  Tests  define  whether  the   system  does  what  the  developers   expect  (“building  the  code  right”) 11 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 11
  • 14. Acceptance  Tests • Are  also  called  customer  tests  or  func/onal  tests • Tell  us  whether  the  system  does  what  the  customer   expects • Enable  Developers  to  know  they’ve  sa/sfied   requirements • Can  be  run  automa/cally  at  any  /me  by  anyone • Helps  us  build  the  “right”  so7ware • “Running,  Tested  Features”* * “A Metric Leading to Agility” – Ron Jeffries http://www.xprogramming.com/xpmag/jatRtsMetric.htm 12 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 12
  • 15. User  Stories Story 14 As a customer I want to check my order status online so that I can know when to expect my package 13 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 13
  • 16. User  Stories Story 14 As a customer I want to check my order status online so that I can know when to expect my package A small piece of business value that can be delivered in an iteration 13 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 13
  • 17. What  is  a  User  Story?* • CARD – Token  represen4ng  the  requirement.  It's  used  in  planning.   Notes  are  wriDen  on  it,  reflec4ng  priority  and  cost • CONVERSATION – The  requirement  itself  is  communicated  from  customer  to   programmers  through  conversa4on  (The  conversa4on  is   largely  verbal,  but  is  oMen  supplemented  with  documents) • CONFIRMATION – The  confirma4on  provided  by  the  acceptance  test  is  what   makes  possible  the  simple  approach  of  card  and  conversa4on – When  the  conversa4on  about  a  card  gets  down  to  the  details   * “Essential XP: Card, Conversation, Confirmation” – Ron Jeffries http://www.xprogramming.com/xpmag/EXPCardConversationConfirmation.htm 14 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 14
  • 18. Confirma/on  through  Acceptance  Tests • Product  Owner  makes  first  pass  at  Acceptance  Criteria   • During  Itera/on  Planning  Acceptance  Criteria  are  discussed • Final  Acceptance  Criteria  for  each  User  Story  is  product  of   nego/a/on  between  Delivery  Team  and  Product  Manager • Should  be  short,  easy  to  understand  statements Story 14 As a customer, I want to check my order status online so that I can know when to expect my package 15 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 15
  • 19. Confirma/on  through  Acceptance  Tests • Product  Owner  makes  first  pass  at  Acceptance  Criteria   • During  Itera/on  Planning  Acceptance  Criteria  are  discussed • Final  Acceptance  Criteria  for  each  User  Story  is  product  of   nego/a/on  between  Delivery  Team  and  Product  Manager • Should  be  short,  easy  to  understand  statements Story 14 Acceptance Criteria •View status as “waiting for pickup”, As a customer, I want to check “en route” or “delivered” my order status online so that I •Date of each step in route can know when to expect my •Estimated time of delivery package 15 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 15
  • 20. Take  on  the  Role  of  a  User  -­‐  “What   should  the  so7ware  do  next  for  me?” This  ques/on  helps  you  to  decide  what  the   next  acceptance  test  should  model 16 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 16
  • 21. Acceptance  Test-­‐Driven  Development 17 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 17
  • 22. Legacy  so7ware,  so7ware  test  automa/on,  and  commercial  off-­‐the-­‐shelf WHEN  TO  USE  EXECUTABLE  SPECIFICATIONS? 18 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 18
  • 23. Tradi/onal  balance  of  tests Manual GUI Easy to Create Acceptance Very familiar – what we always do Tests Typically tedious How do we know coverage? Need Automation specialists Automated Automation good for performance GUI Tests Seems like we always rewrite Sometimes fragile Unit Tests What is Development testing? How do we know what these are? How do we know when they fail? (Originally discussed by Mike Cohn) 19 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 19
  • 24. Mike  Cohn’s  Tes/ng  Pyramid Small number Automate most GUI Acceptance Tests Drive development and acceptance Executable Specifications Do the most Create Test Driven Design Unit Tests (Also “borrowed” from Mike Cohn) 20 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 20
  • 25. Working  with  Legacy  So7ware Define Acceptance Criteria for Requirement 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21
  • 26. Working  with  Legacy  So7ware Define Analyze What Acceptance Might Be Criteria for Affected by Requirement Requirement 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21
  • 27. Working  with  Legacy  So7ware Define Analyze What Write Acceptance Might Be Executable Criteria for Affected by Specifications Requirement Requirement for How it Works Now 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21
  • 28. Working  with  Legacy  So7ware Define Analyze What Write Acceptance Might Be Executable Criteria for Affected by Specifications Requirement Requirement for How it Works Now Write Failing Executable Specifications for How it Should Work 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21
  • 29. Working  with  Legacy  So7ware Define Analyze What Write Acceptance Might Be Executable Criteria for Affected by Specifications Requirement Requirement for How it Works Now Carefully Write Failing Modify Source Executable Code to Specifications Implement for How it Requirement Should Work 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21
  • 30. Working  with  Legacy  So7ware Define Analyze What Write Acceptance Might Be Executable Criteria for Affected by Specifications Requirement Requirement for How it Works Now Execute Carefully Write Failing Executable Modify Source Executable Specifications Code to Specifications to Verify Implement for How it Change Requirement Should Work 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21
  • 31. Working  with  Legacy  So7ware • Focus  on  Wri/ng  Executable  Specifica/ons  for  New   Requirements  and  Func/onality  that  may  be  Affected • Stabilize  Legacy  So7ware  with  Black-­‐box  Tests   (Executable  Specifica/ons) • Incrementally  Improve  the  Internal  So7ware  Design • Try  to  Iden/fy  where  Programmer  Tests  can  be  Added • Execute  Regression  Test  Suite  that  Includes  Executable   Specifica/ons  Frequently  (more  than  once  per  day  if   possible) 22 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 22
  • 32. So7ware  Test  Automa/on Manual Test Script Login as Step 1 Administrato r Create Step 2 Order with 1 Item Click “Check Step 3 Out” Button Verify Item in Step 4 Shopping Cart 23 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 23
  • 33. So7ware  Test  Automa/on Manual Test Script Login as Step 1 Administrato Automate r Create Step 2 Order with 1 Item Click “Check Step 3 Out” Button Verify Item in Step 4 Shopping Cart 23 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 23
  • 34. Commercial  Off-­‐the-­‐Shelf  (COTS) Version 1: COTS Software 24 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 24
  • 35. Commercial  Off-­‐the-­‐Shelf  (COTS) Create Version 1: Executable Specifications for COTS Configurations & Customizations Software 24 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 24
  • 36. Commercial  Off-­‐the-­‐Shelf  (COTS) Create Version 1: Executable Specifications for COTS Configurations & Customizations Software Version 2: COTS Software 24 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 24
  • 37. Commercial  Off-­‐the-­‐Shelf  (COTS) Create Version 1: Executable Specifications for COTS Configurations & Customizations Software Execute Existing Version 2: Executable Specifications to COTS Find Upgrade Path Software 24 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 24
  • 38. Commercial  Off-­‐the-­‐Shelf  (COTS) Create Version 2: Executable Specifications for COTS Configurations & Customizations Software 24 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 24
  • 39. Walk  through  the  crea/on  of  executable  specifica/ons  and  how  to  execute  them  effec/vely AN  EXAMPLE  APPROACH 25 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 25
  • 40. A  simple  example  of  a  FIT  test *Source: http://fit.c2.com/ 26 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 26
  • 41. Fit  Workflow* • Starts  with  whiteboard  conversa/on • Customer  (SME,  business  person,  product  manager,   etc.)  informally  describe  new  feature • Programmers  and  testers  ask  ques/ons *Source: http://fit.c2.com/ 27 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 27
  • 42. Fit  Workflow* • Customer,  working  with  delivery  team,  refines   examples  into  tables • Use  business-­‐friendly  tools,  such  as  Microso7  Excel  and   Word,  to  capture  test  tables *Source: http://fit.c2.com/ 28 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 28
  • 43. Fit  Workflow* • Delivery  team  suggest  addi/onal  areas  to  cover   *Source: http://fit.c2.com/ 29 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 29
  • 44. Fit  Workflow* • Delivery  team  format  tables  for  use  with  Fit *Source: http://fit.c2.com/ 30 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 30
  • 45. Fit  Overview *Source: http://fit.c2.com/ 31 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 31
  • 46. Fit  Workflow* • Delivery  team  creates  Fit  “fixtures”  (small  piece  of  code   that  translates  test  tables  into  execu/on  tests  against   the  so7ware) *Source: http://fit.c2.com/ 32 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 32
  • 47. Fit  Workflow* • Execute  Fit  document   • Click  to  edit  Master  text  styles against  so7ware • At  first  some  tests  should   be  failing  (red) *Source: http://fit.c2.com/ 33 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 33
  • 48. Fit  Workflow* • Delivery  team   • Click  to  edit  Master  text  styles collaborates  with   customer  to  incrementally   enhance  test  tables • Delivery  team  implements   so7ware  to  meet  test   execu/on  (green) *Source: http://fit.c2.com/ 34 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 34
  • 49. Fit  Workflow* • Document  is  kept  for  regression  tes/ng • Document  is  included  in  automated  build  to  ensure   everything  keeps  working   • As  ques/ons  arise  about  func/onality  an  example  is   added  and  Fit  reports  the  answer   *Source: http://fit.c2.com/ 35 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 35
  • 50. Con/nuous  Integra/on 36 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 36
  • 51. Organizing  FIT  tests • Maintain  suite  of  regression  tests  from  past  itera/ons   that  always  pass • Run  “regression”  tests  with  build. • Maintain  a  suite  of  “in  progress”  tests  for  the  current   itera/on – Begin  the  itera4on  with  all  tests  failing – End  the  itera4on  with  most  tests  passing • At  the  end  of  the  itera/on,  move  newly  passing  tests   into  regression  suite – Beware  the  Fitnesse  “Refactor/Move”  command 37 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 37
  • 52. Automated  Regression  Tes/ng Application Under Development 38 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 38
  • 53. Automated  Regression  Tes/ng Application Under Development New Feature 38 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 38
  • 54. Automated  Regression  Tes/ng Application Under Development New Design Changes Feature 38 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 38
  • 55. Automated  Regression  Tes/ng Application Under Development Automated Design Changes Regression New Test Run Feature 38 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 38
  • 56. Automated  Regression  Tes/ng Application Under Development Automated Design Changes Regression New Test Run Feature 38 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 38
  • 57. Cost  reduc/on  using  Fit  for  test  automa/on  and  data  conversion A  FIT  CASE  STUDY 39 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 39
  • 58. Manual  Regression  Tes/ng • Tes/ng  was  taking  75  person  hours  during  2  full  test   runs  consis/ng  of: – Comprehensive  manual  regression  tes4ng – Data  conversion  and  valida4on • Cost  for  tes/ng  was  $17,000  each  itera/on 40 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 40
  • 59. Introducing  Fit  into  Tes/ng  Process • A7er  8  itera/ons  team  had  introduced  healthy  amount   of  Fit  fixtures  and  automated  tests • Reduced  70+  hour  test  run/me  down  to  6  hours  which   now  included: – Fit  automated  regression  tes4ng   – Data  conversion  and  valida4on  automated  with  Fit  fixtures   • Reduced  cost  of  tes/ng  each  itera/on  from  $17,000  to   $7,000 41 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 41
  • 60. Tools  that  Teams  can  use  and  what  context  they  are  best  used  in AVAILABLE  TOOLS 42 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 42
  • 61. Table-­‐based  Acceptance  Tes/ng  Tools • Fit  –  hlp://fit.c2.com/   • FitNesse  –  hlp://www.fitnesse.org/ • RobotFramework  -­‐  hlp://code.google.com/p/ robonramework/   43 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 43
  • 62. Programmer  Acceptance  Tes/ng  Tools • Cucumber  -­‐  hlp://cukes.info/   • Rspec  -­‐  hlp://rspec.info/   • JBehave  -­‐  hlp://jbehave.org/   • JMeter  -­‐  hlp://jakarta.apache.org/jmeter/   • xUnit  (JUnit/NUnit/CppUnit/etc…) 44 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 44
  • 63. Web  UI  Agile  Tes/ng  Tools • Open  Source  Web  UI  Tes/ng  tools – StoryTestIQ – WATiR – Selenium – Canoo  Web  Test • GUI  tests  tend  to  be  more  brille • Tests  wrilen  and  maintained  incrementally • Most  of  these  tools  have  code  integra/on 45 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 45
  • 64. What  about  tradi/onal  tes/ng  tools? • Why  don’t  we  use  commercial  products  such  as  SilkTest,   TestDirector,  QuickTest  Pro,  etc.  for  Agile  tes/ng? – Expensive – Automated  tools  are  record-­‐and-­‐playback;  briDle – Ties  our  tests  to  the  UI  implementa4on – Manual  tools  (TestDirector)  take  too  long  to  run. – Work  fine  as  an  interim  strategy  (especially  if  you  already   have  the  licenses) – Consider  adding  open  source  Agile  Tes4ng  tool  as  a   component  of  your  tes4ng  strategy 46 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 46
  • 65. Ques/ons  and  Answers THANK  YOU 47 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 47
  • 66. Presenter:  Chris  Sterling Partner  at  Sterling  Barton • Technology  Consultant,  Cer/fied  Scrum   Trainer  and  Agile  Coach • Consults  on  so7ware  technology  across  a   spectrum  of  industries • Consults  organiza/ons  on  Agile   development,  management,  and   enterprise  prac/ces • Founder  of  Interna/onal  Associa/on  of   So7ware  Architects  (IASA)  Puget  Sound   chapter • University  of  Washington  Lecturer:  Agile   Developer  Cer/ficate  Program Email:  chris@SterlingBarton.com Our  Blog:  hlp://www.getngagile.com Follow  me  on  Twiler:  @csterwa 48 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 48

×