Managing Software Debt - Quality Debt Focus - QASIG Kirkland


Published on

  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Managing Software Debt - Quality Debt Focus - QASIG Kirkland

  1. 1. Managing  So)ware   Debt  in  Prac2ce     Quality  Debt  Focus  
  2. 2. First  Things  First...   Get Rally! @csterwa #swdebt 2  
  3. 3. Chris  Sterling   • Director  of  Engineering  at  Rally   So)ware  in  Kirkland,  WA  office   • Author  of  Book   Managing  So)ware   Debt:  Building  for  Inevitable  Change   • Consults  on  so)ware  technology,  Agile   technical  prac2ces,  Scrum,  and   effec2ve  management  techniques   • Innova2on  Games®  Trained  Facilitator   Email:       Web:  hWp://     • Cer2fied  Scrum  Trainer   Follow  me  on  TwiWer:  @csterwa     Blog:  hWp://   Hashtag  for  presenta2on:  #swdebt   • Open  Source  Developer   3  
  4. 4. Agenda   •  Managing  So)ware  Debt   •  The   3  Amigos  PaWern   Overview   •  Test-­‐Driven  Design   •  So)ware  Debt  Types   •  Monitoring  Quality   •  Technical   •  The  Power  of  2  Scripts   •  Quality   •  Con2nuous  Integra2on   •  Configura2on  Management   •  Automated  Promo2on   •  Design   •  Advanced  Quality  Metrics   •  Pla]orm  Experience   Trending  and  Analysis   •  Asser2ng  Quality   •  Wrap  Up   •  Defini2on  of  Done   •  So)ware  Debt  Management   Strategy   •  Test  Automa2on   •  The   No  Defect  Mindset   4  
  5. 5. Managing  So)ware  Debt  Overview   Let  me  tell  you  a  poten2ally   familiar  story…  
  6. 6. Lack  of  emphasis  on  so)ware  quality  aWributes  contributes  to  decay  
  7. 7. Principle:     No  maWer  what,  the  cost  of   addressing  so)ware  debt   increases  with  2me.  
  8. 8. Types  of  So)ware  Debt   Technical,  Quality,   Configura2on  Management,   Design,  and  Pla]orm   Experience  
  9. 9. Why  not  just  call  it  all   Technical  Debt   • Technical  debt  tended  to  focus  more  on   programming  aspects  of  so)ware  delivery  and   le)  out  full  so)ware  development  life  cycle   • Each  type  of  so)ware  debt  can  be  managed  and   monitored  using  different  tools  and  approaches   • Focusing  on  managing  each  type  of  so)ware  debt   simplifies  crea2on  of  an  overall  strategy  that   promotes  holis2c  perspec2ve   10  
  10. 10. Types  of  So)ware  Debt   •  Technical  Debt:  These  are  the  ac2vi2es  that  a  team  or  team  members   take  shortcuts  on  now  that  will  impede  future  development  if  le)  as   is.   •  Quality  Debt:  There  is  a  diminishing  ability  to  verify  the  func2onal  and   technical  quality  of  so)ware:  the   Break/Fix  mentality.   •  Configura2on  Management  Debt:  Integra2on  and  release   management  become  more  risky,  complex,  and  error-­‐prone.   •  Design  Debt:  The  cost  of  adding  features  is  increasing  toward  the   point  where  it  is  more  than  the  cost  of  wri2ng  from  scratch.   •  Pla]orm  Experience  Debt:  The  availability  and  alignment  of  people  to   business  objec2ves  that  involve  so)ware  changes  is  becoming  more   limited  or  cost-­‐prohibi2ve.   8  
  11. 11. Asser2ng  Quality   Teams  must  focus  on  asser2ng   sustainable  quality  to  support   future  customer  needs  in  a   2mely  manner  
  12. 12. Defini2on  of  Done   So)ware  developments  assert   what  finished,  working   so)ware  entails  to  support   predictable  delivery  into  the   future  
  13. 13. Defini2on  of  Done  -­‐  Assert  Quality   " Acceptance defined criteria for each " Tested on FE user story " Integration test written & passes " Unit tests written and passed " Test code reviewed " Code compiles with no errors and no " Environment requirements documented warnings " New code doesn t break existing code " Interface document updated/added and checked in to SVN " Test case review (Dev to review test " Acceptance criteria verified complete case written) " Architectural impact assessed and " All P1-P3 bugs for the story are closed artifacts updated if necessary " Test approves user story " Comments in code " Story demonstrated to product owner and accepted on Target Platform " Error codes added " Code reviewed by peer " Code checked in with reference to US#/Task# 14  
  14. 14. Release  Defini2on  of  Done   • Every  release  should  have  clear  quality  criteria   • With  a   Release  Defini2on  of  Done  you  can   understand  targets  beWer   • Measure  the  gap  between  the  teams  Defini2on   of  Done  and  a  Release  Defini2on  of  Done.   •  This  gap  is  a  source  of  quality  issues  and  represents   significant  risk  to  schedule  
  15. 15. Case  Study:     Test  Automa2on  Reduces  Cost   of  Change  
  16. 16. Manual  Regression  Tes2ng   • Tes2ng  was  taking  75  person  hours  during  2  full   test  runs  consis2ng  of:   •  Comprehensive  manual  regression  tes2ng   •  Data  conversion  and  valida2on   • Cost  for  tes2ng  was  $17,000  each  itera2on   17  
  17. 17. Introducing  Fit  into  Tes2ng  Process   • A)er  8  itera2ons  team  had  introduced  healthy   amount  of  Fit  fixtures  and  automated  tests   • Reduced  70+  hour  test  run2me  down  to  6  hours   which  now  included:   •  Fit  automated  regression  tes2ng     •  Data  conversion  and  valida2on  automated  with  Fit   fixtures     • Reduced  cost  of  tes2ng  each  itera2on  from   $17,000  to  $7,000   18  
  18. 18. The  Agile  Regression  Tes2ng  Triangle*   Smoke++  Tests   Risk-­‐based  UI  &   API  Automated   Integra2on  Tests   Tests   Automated  &   Exploratory   Automated  Unit  Tests   Make  up  largest  por2on  of   regression  tests  and  are   developed  by  programmers   *  The  Agile  Triangle  has  been  modified  from  Mike  Cohn’s  original  version   19  
  19. 19. The   3  Amigos  PaWern*   Quickly  get  testers,  coders,   and  business  on  the  same  page   before  building  based  on  a   requirement   *  The  Three  Amigos  paWern  originally  coined  by  George  Dinwiddie   hWp://    
  20. 20. The  Three  Amigos  PaWern   As a Shopper I want to receive updates on incredible deals that are located near my home so that I can save money on my purchases Acceptance Criteria: • Save Shopper’s location • Ask Shopper if they want to receive localized deals daily • Send notification of incredible deals to Shoppers located within 10 miles each morning • Allow Shopper to stop receiving localized deals 21  
  21. 21. The  Three  Amigos  PaWern   As a Shopper I want to receive updates on incredible deals that are located near my home so that I can save money on my purchases • What  areas  of  the  applica2on  will  this  affect?   • What  is  the  overall  design?  (UI,  API,  UX,  etc…)   • What  are  the  details  test  cases  for  this  user  story   and  it’s  acceptance  criteria?   • What  about  nega2ve  test  condi2ons?   • What  about  boundary  condi2ons?   • How  might  we  put  exis2ng  func2onality  at  risk?   22  
  22. 22. The  Three  Amigos  PaWern   • At  minimum  include   tester,  coder  &   business  rep  in   discussion   • Should  only  take  30   minutes  to  1  hour  for   user  stories   • Focus  on  clarifica2on   and  design  through   testable  inputs/ outputs   23  
  23. 23. Acceptance  Test-­‐Driven  Development   24  
  24. 24. Release  Management   If  releases  are  like  giving   birth,  then  you  must  be  doing   something  wrong.  -­‐  Robert   Benefield  
  25. 25. Case  Study:  Enterprise  Agile  Adop2on   •  180+  person   Web  2.0  product  organiza2on   •  Waterfall  SDLC  that  development  uses  to  deliver  in  6  month  release   cycles   •  Want  to  use  Agile  methods  to  be  more  responsive  to  users  and  keep  up   with  other   Web  2.0  companies   •  Transi2oned  to  Agile  methods  on  15  teams  in  3  months   •  Changed  release  management  strategy,  added  XP  technical  prac2ces,   and  implemented  Scrum  product  development  framework  for  scaled   coordina2on   •  Able  to  release  every  week  to  users  within  4  months   •  Used  streamlined  deployment  environment  process  to  validate  product   changes  daily  using  Con2nuous  Integra2on  and  automated  promo2ons   26  
  26. 26. The  Power  of  2  Scripts:  Deploy  &  Rollback   27  
  27. 27. Tradi2onal  Source  Control  Management   Code   Complete   Version  1   Integrate  for   Branch   Version  2   Debt   Main  Branch   Death  March   {   Debt  accrues  quickly  within  stabiliza2on  periods   28  
  28. 28. Flexible  Source  Control  Management   Version 1 Version 2 Main Branch { Not Easy! Must have proper infrastructure to do this. 29  
  29. 29. Scaling  Con2nuous  Integra2on   End-­‐to-­‐End  &   Load/Stress   Integrated   Component   Valida2on   Component   Valida2on   30  
  30. 30. Automated  Promo2on  to  Environments   31  
  31. 31. Advanced  Quality  Asser2ons     Using  Automated  Tools  and   Dashboards  
  32. 32. Con2nuous  Integra2on   33  
  33. 33. Quality  Dashboard  -­‐  Sonar   34  
  34. 34. Quality  Dashboard  -­‐  Sonar   35  
  35. 35. Quality  Dashboard  -­‐  Sonar   36  
  36. 36. Quality  Dashboard  -­‐  Sonar   37  
  37. 37. Early  Warning  Signs   Early  Warnings:   • Broken  Builds   • Broken  Automated  Tests   • Broken  Custom  Thresholds   38  
  38. 38. Early  Warning  on  Quality  Dashboard   Early  Warnings:   • Design  Debt  in  Duplica2on  (DRY)   • Technical  Debt  in  Code  Complexity   • Quality  Debt  in  Bug  DB  (Break/Fix)   • Other  Custom  Thresholds   39  
  39. 39. The   No  Defect  Mindset   What  he  needs  is  some  way   to  pay  back.  Not  some  way  to   borrow  more.  -­‐-­‐  Will  Rogers   39  
  40. 40. Ken  Schwaber     For  every  [dollar]  of  compe22ve   advantage  gained  by  cuYng  quality,  it   costs  $4  to  restore  it;  and  so)ware  is   an  organiza2onal  asset  and  decisions   to  cut  quality  must  be  made  by   execu2ve  management  and  reflected   in  the  financial  statements.   hWp://­‐quality-­‐canary-­‐coalmine  
  41. 41. Case  Study:  Field  Support  Applica2on   •  2000+  users  access  applica2on  each  day   •  Applica2on  supports  mul2ple  perspec2ves  and  workflows   from  Field  Support  Opera2ons  to  Customer  Service   •  Team  of  5  people  delivering  features  on  exis2ng  Cold  Fusion   pla]orm  implementa2on   •  Migra2ng  Architecture  to  Spring/Hibernate  in  slices  while   s2ll  delivering  valuable  features   •  36  2-­‐week  Sprints,  33  produc2on  releases,  and  only  1  defect   found  in  produc2on   •  So,  what  was  the  defect  you  say?  Let  me  tell  you…   40  
  42. 42. Can  We  Afford  a   No  Defect  Policy?   •  This  team  worked  on  legacy  codebase  inherited  from  another   vendor   •  Other  vendor  had  been  slowing  down  month  a)er  month  and   cost  of  development  was  increasing   •  In  first  itera2on  this  team  was  able  to  deliver  more  than  other   vendor  was  able  to  in  previous  2  months   •  A)er  24  itera2ons  this  team  was  10  2mes  faster  delivery  than1st   itera2on   •  Acceptance  Test-­‐Driven  Development  and  Con2nuous  Integra2on   were  greatest  technical  factors  to  support  team  in  these  results   •  Can  you  afford  not  to  have  a   No  Defect  policy?   41  
  43. 43. Thank  you!   Ques2ons  and  Answers   [Time  permiYng]  
  44. 44. Chris  Sterling  Director  of  Engineering  at  Rally  So)ware  in  Kirkland,  WA  office  Author  of  Book   Managing  So)ware  Debt:  Building  for  Inevitable  Change  Consults  on  so)ware  technology,  Agile  technical  prac2ces,  Scrum,  and  effec2ve  management  techniques  Innova2on  Games®  Trained  Facilitator   Email:       Web:  hWp://    Cer2fied  Scrum  Trainer   Follow  me  on  TwiWer:  @csterwa     Blog:  hWp://   Hashtag  for  presenta2on:  #swdebt  Open  Source  Developer