Your SlideShare is downloading. ×
Using flow approaches to effectively manage agile testing at the enterprise level
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Using flow approaches to effectively manage agile testing at the enterprise level

2,115

Published on

Slides from my upcoming LSSC11 talk: …

Slides from my upcoming LSSC11 talk:

More and more organizations want to become more agile these days. When the theory hits the shores of reality, few organizations can get to an idealistic agile feature team that does all testing within sprints, has no need for release-level processes, and where everything is fully automated continuous deployment style. Usually the testing organization is in the eye of the storm when talking about Big Batches, Wastes, ineffective handoffs and mountains of rework, as well as high transaction costs. I’ve recently been using Lean/Kanban flow based approaches to provide a way to evolve testing organizations to a more effective way of working, so that they can better support earlier feedback and higher flexibility. I will present this work as well as case studies from enterprise-level product development companies that are starting to use these approaches.
We will deal with the following challenges:

* Complex environments when it is not realistic to finish all required work within a sprint
* How to visualize and reduce testing batch sizes within sprints/releases using CFD
* How to deal with the testing bottleneck so common in product development organizations – practical suggestions and how to deal with the mindset issues
* How to run stabilization/hardening periods using Flow-based thinking

http://lssc11.crowdvine.com/talks/18074

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,115
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
65
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Recipe for Continuous Stabilization How to deal with Elasticity in Testing (and in general...)How to deal with Shared Test LabsClasses of treatment
  • Recipe for Continuous Stabilization How to deal with Elasticity in Testing (and in general...)How to deal with Shared Test LabsClasses of treatment
  • http://www.flickr.com/photos/andreasnilsson1976/1307180278/sizes/l/in/photostream/
  • TODO Highlight where I’m going to focus today.
  • Can do it either way – but last few examples we see are a “Burning platform” so willing to sacrifice in order to see quick improvementsInteresting though – even under distress, going Scrum all the way is usually not chosen. Several theories why? Offline...
  • What does it mean to manage flow? Managers take action to stop the line in case of high WIPIdentify bottlenecks and divert resources / focusManagement attention
  • Vote:Who here is working on V<1.0?Of those – who knows up front the majority of the scope of V1.0? The rest should care.On those working
  • Recipe for Continuous Stabilization How to deal with Elasticity in Testing (and in general...)How to deal with Shared Test LabsClasses of treatment
  • Empty after testing, development in done testing busy  bottleneck in testingThis is a classic bottleneck in an R&D team.Testing are at their work in progress limit, meaning they cannot take on more work. Acceptance has no work in progress, what we call a “bubble”Development are at their limit as well. Nothing from Testing is DONE waiting to be pulled, which explains why Acceptance has a bubble
  • Automation – not just test automation!How can we help you spend more time actually testing (compared to setup, and other wastes) (http://theoryofconstraints.blogspot.com/2007/06/toc-stories-2-blue-light-creating.html) How often do we need to retest? Why?ATDD - drives better code into testing, as well as offload some testing workAgree on “READY for Testing” criteria for stories, setup relevant team rules and processes.
  • http://www.flickr.com/photos/basf/4837105835/sizes/l/in/photostream/High variability – you never know what you get...High elasticity – You can always do more – Which is a special kind of Variability...
  • http://www.flickr.com/photos/basf/4837105835/sizes/l/in/photostream/High variability – you never know what you get...High elasticity – You can always do more – Which is a special kind of Variability...
  • What does it mean to manage flow? Managers take action to stop the line in case of high WIPIdentify bottlenecks and divert resources / focusManagement attention
  • Explicit expectation – can be via uniform size, or via some level of estimations.
  • When there is an external due date – sure, put it on there, and it will create purposeBut what if there isn’t really an external due date? Put the estimated due date based on your current cycle time predictability control chart.
  • Testers should not feel like the “last line of defense” – they should really get higher quality stuff to begin with. Everyone should support doing the right amount of testing, sometimes it will be more, sometimes it will be lessEveryone should start to think about the right economic decision, not just perfect quality. With history of low quality, it is hard for testers to trust this will happen. When you start building quality in, trust can build, and risks can start to be taken. Talk about my experience in SANRAD about this? 51? What happens when Whole Team approach starts to kick in. Testers should
  • Sometimes fixing all defects is NOT the right economic decision...
  • Main point is to have a discussion about the policies, share the view about why they should work for you, and be clear about the expectations. Talk about Testing Yield
  • Main point is to have a discussion about the policies, share the view about why they should work for you, and be clear about the expectations.
  • Recipe for Continuous Stabilization How to deal with Elasticity in Testing (and in general...)How to deal with Shared Test LabsClasses of treatment
  • TODO – summarize the tools/tips until nowPut the tips on the screen
  • TODO – emphasize the shared resources
  • Red– performance team Must be involved hands onYellow– performance team Advise/Consult, but most work is in TeamsGreen– don’t need any involvement from performance teamDrives collective ownership of performance (and other “Ilities”) – And is another form of subordinationto the constraint
  • Recipe for Continuous Stabilization How to deal with Elasticity in Testing (and in general...)How to deal with Shared Test LabsClasses of treatment
  • Recipe for Continuous Stabilization How to deal with Elasticity in Testing (and in general...)How to deal with Shared Test LabsClasses of treatment
  • Transcript

    • 1. #LSSC11 /CC @yuvalyeret
      Testing Flow
      Using flow approaches to effectively manage agile testing at the enterprise level
      Yuval Yeret –
      Kanban Lead - AgileSparks
    • 2. Repeat after me:VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE-VALUE
      UE-VAL == YUVAL
    • 3.
    • 4. What we covered
      Testing
      Elasticity
      Test-specific
      Challenges
      To
      Flow
      Why Flow for Testing
      Issues at Scale
    • 5. Distress call!
      “We need to release much more frequently”
      “the business cannot live with months of hardening”
      We want Agile to save us!
    • 6. So why not just do Scrum+XP?
      Sufficient?Not for Enterprise scale
      Necessary?
      Not all of it
      Pragmatic?
      So so
    • 7. DONE DONE– The Enterprise Testing Challenge
      “We are doing agile iterations for development and testing and it is going quite well. One of our greatest challenges is dealing with the stuff we cannot finish within the iteration – currently its one big pile at the end”
      R&D GM
      Large Enterprise Software Group
    • 8. What are the alternatives?
      Longer Sprints
      Sprint
      Synchronized
      Handoffs
      Flow-based
      Handoffs
    • 9. Flowing all the way to DONE DONE
      DONE DONE
      READY
      DONE
      9
    • 10. Complement
      Solutions
      Focus
      Pragmatic Pace
    • 11. So... We started to use a recipe based on Flow
    • 12. Current Reality Tree
      Let’s dive into why this is a problem
      LOW PREDICTABILITY
      HIGH STABILIZATION COSTS
      QUALITY @ THE END
      LATE FEEDBACK
      HEAVY MANUAL TESTING
      LONG CYCLE TIMES
      LARGE WIP
      BIG BATCHES
      BIG FEATURES
      FEATURE PER PERSON
      DONE DONE
      LIMIT FEATURES IN PROGRESS
      AUTOMATE AS PART OF DONE
      CONTINUOUS INTEGRATION
      SPECIALIZAT-ION
      LONE WOLFS
      NOT EFFECTIVE
      FEATURE FREEZE POLICY TOO STRICT
      HIGH COST
      OUR BUILD IS NOT STABLE
      LIMIT OPEN DEFECTS
      SMALLER FEATURES
      COLLECTIVE OWNERSHIP
      LIMIT AT THE FEATURE LEVEL
      WHOLE TEAM
      AUTOMATION PYRAMID
      INVEST IN REMOVING INSTABILITY
      IMPROVEMENT BACKLOG
      START AS IS, SHORTEN WHEN PREDICTABLE
      FIX M/L DEFECTS?
      LOSE SIGHT OF THE BIG PICTURE
      LARGE OVERHEAD
      EXISTING GAP IS TOO BIG
      EFFECTIVE SLICING
      PRAGMATIC RISK PROFILING
      TRANSITION BASED ON NECESSITY
      elad@agilesparks.com
      http://blog.crisp.se/henrikkniberg
      http://www.flickr.com/photos/librarygoblin/4307610546/lightbox/
    • 13. Is this Evolution or Revolution? k/K?
      Performance
      Distress Calls +
      Israeli Impatience?
      Evolution
      Time
      This is what we see in Israel...
    • 14. Lets see how it looks like in practice
    • 15. “What is the gapbetween dev and test? What are we doing to close it?”
    • 16. BAD
      GOOD
      Limit Defects in Process – Early Quality is Cheap Quality
    • 17. “We used to feel comfortable to release quarterly and anxious to release every two weeks. “
      “Now releasing every two weeks is natural. We are anxious when we can’t do it [due to holiday freeze]”
      FiftyOne.com
    • 18. Always go for Continuous Deployment???
      Not necessarily
    • 19. What we covered
      Testing
      Elasticity
      Test-specific
      Challenges
      To
      Flow
    • 20. Limiting WIP – pull according to the bottleneck
      DONE
      R
      D
      C
      Hardening
      R
      D
      C
      T
      R
      D
      C
      T
      R
      D
      C
      T
      R
      D
      C
      T
      R
      D
      C
      T
      Time
    • 21.
    • 22. WIP Limits SLACKTIME ???
      * No coaches were idle due to the preparation of this presentation
    • 23. Feature Freeze
      Coding Done
      DONE
      Hardening
      R
      D
      C
      T
      R
      D
      C
      T
      R
      D
      C
      T
      R
      D
      C
      T
      R
      D
      C
      T
      R
      D
      C
      T
    • 24. What LIMITED WIP Does
      Fix open defects on current work
      3
      WIP Limit!
      Can’t startnew DEV work!
      Help us with Blocker
      T
      D
      T
      D
      T
      D
      Help us automate tests for current work
      How can we help current work?
    • 25. What LIMITED WIP Does
      Automate Setups and Test Data
      Improve quality you deliver to QA!
      Half of our work is not core test work. Maybe you can take some of it, or help us reduce waste there
      3
      T
      D
      T
      D
      Ways to share the test load – e.g. automation/setups work
      T
      D
      How can I help you be more efficient?
    • 26. Developers
      Developers
      Testers
      Where Kanbanis used, Whole Team, ATDD and other Agile Testing approaches emerge MUCH faster
    • 27. problem solved?
    • 28. “Testing was always like a pillow – it took whatever space was left. We used to have a cap on testing because there was a deadline...
      ...Now with the new Flow mode the inmates are running the asylum [and we have no cap on how long testing takes]”
      A worried Dev Manager
    • 29. Testing/Verification is a highly Elastic activityLets look at a few ways to cope ...
    • 30. Cadence of releases/drops, will drive completion
    • 31. Have an explicit policy regarding the desired/acceptable size of work
    • 32. Care about Cycle Time & Done
      Testing
      End to End
    • 33. Due Datescan help(Especially if doable and agreed upon with delivery team)
      http://yuvalyeret.com/2010/09/19/kanban-early-warning-using-a-predictive-variant-of-spc/
    • 34. No REAL external Due Date?
      tryInternal Cycle Time Soft Goals based on learned capabilities
      Avoidlocal due dates!
      http://yuvalyeret.com/2010/09/19/kanban-early-warning-using-a-predictive-variant-of-spc/
    • 35. Try...In-Flight Control Chart
    • 36. Energizing Purpose and Whole Team Accountability
    • 37. Can testing be overdone?
    • 38. Talk about what is DoneEstablish Explicit Policies
    • 39. Experiment with using an Explicit Policies board documenting your process
    • 40. Different work types / CoS can have different Done/Release Criteria
    • 41.
    • 42. 1
      2
      3
      Goals+Visibility enhance Purpose
    • 43. What we covered
      Testing
      Elasticity
      Issues at Scale
    • 44. Will visualizing and managing flow in each of those roads be enough?
    • 45. Need to visualize and manage the global end to end flow across sharedresources
    • 46. Sharing
    • 47. Manage flow at the level at which resources are shared
    • 48. 3
    • 49. Example Policy - Classes of Service for Downstream Involvement
      Classes of Treatment
      Risk Profiling  Involvement mode of the shared resource
    • 50. What we covered
      Testing
      Elasticity
    • 51. yuval@agilesparks.com
      @yuvalyeret
      Get the slides at http://www.slideshare.net/yyeret
      Questions?
      Blogging at
      http://yuvalyeret.com
    • 52. Images Attribution
      http://www.flickr.com/photos/yuvalyeret/265568342/in/set-72157594323037021/
      www.slideshare.net/wnazzaro/agile-it-and-the-business-community
      http://www.freefoto.com/images/41/03/41_03_4---Weight-Limit-Sign_web.jpg?&k=Weight+Limit+Sign
      http://www.flickr.com/photos/maisonbisson/267746261/sizes/o/in/photostream/
      http://www.flickr.com/photos/malavoda/4195215934/sizes/l/in/photostream/
      http://www.flickr.com/photos/njj4/5044361592/sizes/l/in/photostream/
      http://www.flickr.com/photos/grafixer/3527166081/sizes/l/in/photostream/
      http://www.flickr.com/photos/librarygoblin/4307610546/lightbox/
      http://www.flickr.com/photos/orinrobertjohn/2188280583/sizes/l/in/photostream/
      http://www.flickr.com/photos/kwl/4809326028/
      http://www.flickr.com/photos/justin_case/54087334/sizes/o/in/photostream/
      http://www.flickr.com/photos/noneck/869906174/sizes/l/in/photostream/
      http://www.flickr.com/photos/wendycopley/1332359314/
      http://www.flickr.com/photos/frumbert/307532180/sizes/l/in/photostream/
      http://www.flickr.com/photos/tgillin/4599272187/
      http://www.flickr.com/photos/rolleh/3475746680/sizes/o/in/photostream/
      http://www.flickr.com/photos/pamilne/3401930511/sizes/l/in/photostream/
      http://www.flickr.com/photos/batintherain/112826092/sizes/l/in/photostream/
      http://sherisays.files.wordpress.com/2010/08/drivers-stuck-in-traffic-jam-for-9-days-in-china.jpg
      http://www.flickr.com/photos/slidesf/2648616262/sizes/l/in/photostream/
      http://www.flickr.com/photos/jongales/391648530/sizes/l/in/photostream/
      http://www.flickr.com/photos/andreasnilsson1976/1307180278/sizes/l/in/photostream/
      http://www.flickr.com/photos/basf/4837105835/sizes/l/in/photostream/

    ×